Linkerd-tcp not detecting basic websocket app


#1

Hello,
I’ve been attempting to configure linkerd-tcp to work with a simple websocket and client with no luck. I’m not getting any sort of errors anywhere however linkerd-tcp is not logging any sort of tcp connections with RUST_LOG set to ‘trace’.

The following configuration is what I’m using for a linkerd-tcp daemonset and namerd deployment with dtabs custom resources in kubernetes.

kind: CustomResourceDefinition
apiVersion: apiextensions.k8s.io/v1beta1
metadata:
  name: dtabs.l5d.io
spec:
  scope: Namespaced
  group: l5d.io
  version: v1alpha1
  names:
    kind: DTab
    plural: dtabs
    singular: dtab
---
apiVersion: l5d.io/v1alpha1
dentries:
- dst: /#/io.l5d.k8s/default/http
  prefix: /svc
kind: DTab
metadata:
  namespace: linkerd
  name: l5d
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    app: namerd
  name: namerd
  namespace: linkerd
spec:
  selector:
    matchLabels:
      app: namerd
  strategy:
    rollingUpdate:
      maxSurge: 2
      maxUnavailable: 1
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: namerd
    spec:
      volumes:
      - name: l5d-tcp-namerd
        configMap:
          name: l5d-namerd-config
          items:
          - key: namerd.yaml
            path: namerd.yaml
      containers:
      - name: namerd
        image: buoyantio/namerd:1.3.5
        args:
        - /io.buoyant/namerd/1.3.5/config/namerd.yaml
        volumeMounts:
        - name: l5d-tcp-namerd
          mountPath: /io.buoyant/namerd/1.3.5/config/namerd.yaml
          subPath: namerd.yaml
        ports:
        - name: namerd-admin
          containerPort: 9991
      restartPolicy: Always
      securityContext: {}
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: l5d-namerd-config
  namespace: linkerd
data:
 namerd.yaml: |-
    admin:
      port: 9991
      ip: 0.0.0.0

    storage:
      kind: io.l5d.k8s
      host: localhost
      port: 8001
      namespace: linkerd
    interfaces:
    - kind: io.l5d.httpController
      ip: 0.0.0.0
      port: 4180
    telemetry:
    - kind: io.l5d.prometheus
      prefix: tcp_

    namers:
    - kind: io.l5d.k8s
      host: localhost
      port: 8001
---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  labels:
    app: l5d
  name: l5d
  namespace: linkerd
spec:
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: l5d
    spec:
      hostNetwork: true
      dnsPolicy: ClusterFirstWithHostNet
      volumes:
      - name: l5d-tcp-config
        configMap:
          name: l5d-tcp-config
          items:
          - key: config.yaml
            path: config.yaml
      containers:
      - name: linkerd-tcp
        image: linkerd/linkerd-tcp:0.1.1
        command: [ "/usr/local/bin/linkerd-tcp"]
        args:
        - /io.buoyant/linkerd/config/config.yaml
        volumeMounts:
        - name: l5d-tcp-config
          mountPath: /io.buoyant/linkerd/config/config.yaml
          subPath: config.yaml
        ports:
        - name: tcp-admin
          containerPort: 9989
          hostPort: 9989
        - name: default
          containerPort: 4180
        env:
        - name: RUST_LOG # for debugging
          value: "trace"
        - name: RUST_BACKTRACE # for debugging
          value: "1"
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
          ######################################################
      - name: kubectl
        image: buoyantio/kubectl:v1.8.5
        args:
        - "proxy"
        - "-p"
        - "8001"
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: l5d-tcp-config
  namespace: linkerd
data:
  config.yaml: |-
    admin:
      ip: 0.0.0.0
      port: 9989
      metricsIntervalSecs: 10
    routers:
    - label: default
      interpreter:
        kind: io.l5d.namerd.http
        namespace: default
        baseUrl: http://localhost:4180
        periodSecs: 20
      servers:
      - port: 4180
        dstName: /svc/default
---
apiVersion: v1
kind: Service
metadata:
  name: l5d
  namespace: linkerd
  labels:
    k8s-app: l5d
    app: l5d
spec:
  selector:
    app: l5d
  type: LoadBalancer
  ports:
  - name: tcp-admin
    port: 9989
  - name: default
    port: 4180
  - name: namerd-admin
    port: 9991

as mentioned my test applications are a simple websocket server and client that make consistent requests to each other. Both are set up as regular deployments with the http_proxy set to $(NODE_NAME):4180

I’ve seen examples where linkerd-tcp is set up with a routing server set up on port 7474 as well as having the httpController interface on 4180, what is the difference between those two ports?


#3

Hi @a-kinder,

Thanks for providing your config for additional context. At a first glance it looks like you might be running into a port conflict issue in this section:

routers:
- label: default
  interpreter:
    kind: io.l5d.namerd.http
    namespace: default
    baseUrl: http://localhost:4180
    periodSecs: 20
  servers:
  - port: 4180
    dstName: /svc/default

The interpreter section defines on what port the interpreter is connecting to namerd. Given that namerd is running on localhost, it’s connecting to namerd on port 4180, but not listening for any traffic.

Under servers we define a port and that is the port we configure linkerd to listen on for any incoming requests, so it seems there could be a conflict here. If you look at our example (https://github.com/linkerd/linkerd-tcp) in the README.md, we’ve got the servers listening on different ports, such as 7575 and 7474.

Let us know if you’re running into any other issues. In that case it would be helpful to see the examples you were referring to.


#4

Thanks @franzi! I’ve been plugging away at it since yesterday and did figure out that bit. My colleague has opened an issue with our full, and somewhat updated config at github issue #90. At one point we were getting traffic as far as namerd but it didn’t seem to be able to pass it on to the actual service we were trying to reach.


#5

I was mainly using the linkerd/linkerd-examples linkerd-tcp example from github as a reference (discourse isn’t allowing a direct link for some reason)


#6

Thanks for the update!
Just to wrap up this issue and circle back - your colleague reported the issue as fixed and closed it: (https://github.com/linkerd/linkerd-tcp/issues/90)

I’m glad everything is working now!