Targeting a specific Kubernetes pod


#1

Hi there, I’m new to linkerd and my team has the following use case:

We have a set of approx 10 microservices running on k8s. One of the microservices is the persistency layer - a document store that sync’s over http - and is in a StatefulSet with 3 replicas. Each pod from the document store needs to access the other pods directly (gossip based scheme), but all three need to be accessed via a service also (i.e. round robin load balanced). We would like to have linkerd secure all communication with TLS (pod to pod for stateful set, and service to service).

I’ve deployed the hello world sample on our k8s cluster and that is working, and I can access apps via their service names as expected. Now I need to tackle the pod to pod communication. Instead of using a pod IP directly I’m hoping we can use a DNS name such as 100-100-0-6 and have linkerd route traffic to the pod dns 100-100-0-6.default.pod.cluster.local or something similar.

Any chance somebody could point me in the right direction to target a specific pod via the linkerd proxy?


Targeting Pod IPs with TLS enabled linkerd routers in k8s
#2

Hey @carpenterm this is an interesting case. Linkerd routes to a set of addresses and applies load balancing to those said addresses. If you plan on on talking to only one Pod IP you would need to have an address set with that single Pod IP. You mention that you can use a DNS name like 100-100-0-6, is that something kubernetes provides? If so, one thing you could do is use that DNS name in a Linkerd DTab in order to route addresses to that specific pod.


#3

Thanks @deebo91, yes the pod DNS name is provided natively by kubernetes. You can read about it at https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#a-records-1

I read a little about the DTabs and they seem pretty complex at first glance. Do you know of a similar example I could follow somewhere?


#4

Hi @carpenterm, I am having the exact same problem:
A statefulSet with N pods and I want to do some routing to some specified pods with the name but I am unsure how to do it.
For sure if you go to your service name you will hit one of the 3 pods but no idea on how to specifically target one ?
If anybody has an idea, I would be glad to hear it :slight_smile:
Cheers guys !
P.S: @deebo91 yes k8s provides a dns entry for each pod in a StatefulSet, for me it is
podname.servicename.default.svc.cluster.local
So what you are suggesting is creating a dtab rule for each pod ? But this requires manual configuration and we do not want that ? (or am I mistaken?)
cheers


#5

@carpenterm, @sangui: a custom dtab is the way to go here.

If you have stable pod hostnames, your dtab would look something like:

/svc           => /#/io.l5d.k8s/default/http;
/svc/stateful3 => stateful3.default.pod.cluster.local
/svc/stateful2 => stateful2.default.pod.cluster.local
/svc/stateful1 => stateful1.default.pod.cluster.local

(rules are evaluated from the bottom up)

If you do not have stable pod hostnames, you could use labels instead:

namers:
- kind: io.l5d.k8s
  host: localhost
  port: 8001
  labelSelector: statefulId
/svc           => /#/io.l5d.k8s/default/http;
/svc/stateful3 => /#/io.l5d.k8s/default/http/stateful3;
/svc/stateful2 => /#/io.l5d.k8s/default/http/stateful2;
/svc/stateful1 => /#/io.l5d.k8s/default/http/stateful1;

Some more detail on using label selectors:
https://linkerd.io/config/1.3.2/linkerd/index.html#kubernetes-service-discovery


#6

Thanks for the reply, I will not have stable pod hostnames so I want to use the second solution.

namers:

  • kind: io.l5d.k8s
    host: localhost
    port: 8001
    labelSelector: statefulId

No problem I understand the configuration, however if I have a dtab like this:

/svc => /#/io.l5d.k8s/default/http;

If hostnames are not stable I do not want to put them in dtab, just want linkerd to route to the correct hostname/labeled pod.
When I call a route /svc/servicename/labelvalue, I reach the endpoints but if I have 2 different values possible for 1 same label, linkerd should be able to route to the correct pod no ?
Because if I use value1 or value2 I always end up with both IP of both Pods as possible resolution

Thanks a lot


#7

Hi @sangui. Currently labelSelector only works on services, not pods. We have an open issue to address this:
https://github.com/linkerd/linkerd/issues/1658

In the meantime, as a workaround, you could deploy 3 services, each pointing at a different pod. You would need to set unique selectors on each instance of the pod, either via multiple resource controllers, deployments, or dynamically after the pods are running.