Websocket linkerd-tcp linker-to-linker with TLS

Hey all, this is not a question, I just wanted to share our configuration to have linkerd-tcp passing secured TLS connections between a socket server and a client hosted on Kubernetes. Here are the main parts to achieving this:

For each of our websocket applications we set up a sidecar container containing linkerd-tcp and it’s own port. We used 7474 for the port on the server side and 7575 for the port on the client side.

- name: linkerd-tcp
  image: linkerd/linkerd-tcp:0.1.1
  command: [ "/usr/local/bin/linkerd-tcp"]
  args:
	/io.buoyant/linkerd/config/config.yaml
  volumeMounts:
	name: l5d-tcp-config-ws-client
	mountPath: /io.buoyant/linkerd/config/config.yaml
	subPath: config.yaml
  ports:
	name: tcp-admin
	containerPort: 9989
	name: tcp
	containerPort: 7575

We also set up an environment variable in the client side code, and is replaced by localhost:7575 to direct to the local sidecar pod.

env:
- name: linkerd_proxy # for linkerd
  value: localhost:7575 # tcp proxy port

namerd is set up as a seperate deployment with a dtab routing from service name directly to linkerd-tcp port tcp on sidecar container in server application pod

/svc/server => /#/io.l5d.k8s/default/tcp/server

linkerd-tcp client configuration includes it’s port with a dstName of the application it will be sending requests to.

routers:
...
	servers:
	- ip: 0.0.0.0
	  port: 7575
	  dstName: /svc/server

linkerd-tcp server configuration sets its dstName to localhost on the web port so that the connection will leave linkerd-tcp and reach it’s final destination.

routers:
...
	servers:
	- ip: 0.0.0.0
	  port: 7474
	  dstName: /$/inet/127.1/80

This is all that’s needed to get linkerd-tcp working in linker-to-linker mode.

We also implemented TLS, with certificates that are configured with hosts set to our server application as well as a wildcard node name.

Here is our client linkerd-tcp configuration, now with TLS!

routers:
...
  servers:
  - ip: 0.0.0.0
    port: 7575
    dstName: /svc/server
  client:
    kind: io.l5d.static
    configs:
    - prefix: /svc/server
      connectTimeoutMs: 400
      tls:
        dnsName: "server"
        trustCerts:
        - /io.buoyant/linkerd/certs/tls.chain

And our server linkerd-tcp configuration:

routers:
...
  servers:
  - ip: 0.0.0.0
    port: 7474
    dstName: /$/inet/127.1/80
    tls:
      defaultIdentity:
        privateKey: /io.buoyant/linkerd/certs/tls.key
        certs:
        - /io.buoyant/linkerd/certs/tls.crt
        - /io.buoyant/linkerd/certs/tls.chain

I hope that this helps anyone trying to implement linkerd-tcp on their cluster.

Cheers!

2 Likes

Thank you very much for sharing this!