Can't get mTLS to work

Hi there, I’m testing out Linkerd2, for which I started injecting the proxy to a couple of deployments in our cluster to see how it goes.

I’m using AWS Kubernetes (EKS) 1.12. Linkerd 2.4.0

My setup is the following:

ALB Ingress Controller --> Service A --> Service B.

both Services are being injected and I can see them along with some basic stats in Linkerd’s dashboard. However when I tap them, there’s no mTLS turned on: “not_provided_by_remote”

this is a ‘tap’ of the service B. (which is called by service A):

req id=11:1 proxy=in src=xx.xx.yy.136:40430 dst=xx.xx.zz.193:5050 tls=not_provided_by_remote :method=POST :authority=ingress.actual.public-hostname :path=/api/v1/myservice

One thing I noticed, is that “:authority” is set to the original FQDN of the ingress, not the serviceA identity created by the Identity component (serviceb.mynamespace.serviceaccount.identity.linkerd.cluster.local)

Requests work fine. I get the right responses, so Service A is able to talk to Service B. I can see its being done via the proxies as well
I’m not really sure where to go nor what to do, I haven’t found a solution in the documentation either.
Any ideas what might be the problem?

Thanks in advance

Fede

@fedetl

Thanks for your message. There’s a good explanation of the message in to github issue: https://github.com/linkerd/linkerd2/issues/2970#issuecomment-504447024

This issue might be relevant as well: https://github.com/linkerd/linkerd2/issues/2270

Does the FQDN match a record that the proxy will try to resolve?

Thanks @cpretzer for the answer. I’ll read those links today.

As for the FQDN, no, that’s the thing. The FQDN is the external hostname that resolves to the public ingress, like app.mypublichost.com . This Ingress is not meshed so it doesn’t know anything about proxies.

The Ingress talks to the Service A (which is meshed). Service A talks to Service B (also meshed).

Both Services (A and B), are pointing to the actual containers (not the proxies). I understand the routing through the proxy for both incoming and outgoing traffic, is done automatically via iptables. Is that right?

One thing I’m not yet clear. When I installed Linkerd, I used the regular ‘install’ command. I’m using AWSs flavour of Kubernetes (EKS), which uses a CNI plugin for the VPC networking. Should I have installed linkerd using ‘install-cni’ instead ?

Hi there, I believe I have found the problem.

Service A, is basically a nginx with some custom lua logic. After reading https://linkerd.io/2/tasks/using-ingress/ , I added the following headers to the nginx response

    proxy_set_header l5d-dst-override my-svc:80;
    proxy_hide_header l5d-remote-ip;
    proxy_hide_header l5d-server-id;

and now tls is returning ‘true’.

I’ll keep an eye on it, testing it more.
Thanks for the help!

@fedetl That’s great news, I’m glad to hear that TLS is working now.

Please let me know if you have any additional questions. Also, please join our community at https://slack.linkerd.io

Charles