We currently use the kubernetes nginx-ingress-controller and would like to add Linkerd in a basic service mesh configuration (deloyed as a daemonset). I’ve read the guide at https://buoyant.io/2016/11/18/a-service-mesh-for-kubernetes-part-v-dogfood-environments-ingress-and-edge-routing/ which explains how to configure nginx to proxy to Linkerd and I think this is more or less what I am aiming for, except I would like to continue to use the Ingress objects to configure nginx-ingress-controller, rather than create a custom nginx deployment and config.
The first thing I have tried is using the Linkerd config from the article above, and routing traffic via the backend configuration in the Ingress definition to point at the Service for Linkerd. This sends requests to a Linkerd with the expected service hostname. E.g
With the correct dtabs in place I can route traffic to the backend service successfully and see requests coming through the ingress and incoming routers. But I’m not sure exactly whether what I’ve configured is totally sound.
The main concern is that I am not forwarding requests from nginx through an outbound local linkerd (I assume this is why I see no traffic on the outbound router?). However, I think this is also the case with the article above, where nginx is configured to proxy_pass to http://l5d.default.svc.cluster.local, which resolves to the cluster VIP of the Kubernetes Service not the node-local linkerd IP.
If I am correct, what are the implications of this? And is it a suitably reliable configuration to use?
@andyhume sorry for the delay, I’ve double-checked with Risha, who wrote the article, and your setup seems fine.
The main concern is that I am not forwarding requests from nginx through an outbound local linkerd (I assume this is why I see no traffic on the outbound router?).
That is correct, since we want nginx to just send traffic to the VIP of the k8s service, and k8s will load balance over the linkerds.
You don’t want to send traffic to the linkerd outbound router (the service to service linkerd setup will do that for you) - the inbound/outbound routers there are set up purely for service to service routing in the daemonset (basically we’re only adding one more router to the setup explained in https://buoyant.io/2016/10/14/a-service-mesh-for-kubernetes-part-ii-pods-are-great-until-theyre-not/). If you’re sending traffic from service to service (and not external to service) you should see traffic on the outbound routers too.
Thank-you. The configuration I’ve described has been running in a stage cluster with no issues for a week or so. Having you confirm we’re not doing anything off-the-wall is a big step towards us moving forward with this in production.
Andy can you share your configs? I’m thinking of a similar setup wherein nginx-ingress represents the outermost layer. This terminates TLS and matches on subdomains, one of which I’d like to be linkerd managed.
Thus, I’d like to have nginx-ingress-controller handle SSL (with kube-cert-manager) and then forward traffic to https://api.example.com into the linkerd service, which then may be able to redirect grpc requests to various internal services.