THRIFT client connectivity issues

I am trying out a THRIFT test client against the service container running in marathon.
when i try with linkerd, the service does not respond.

# java -cp service-test-1.0-SNAPSHOT-jar-with-dependencies.jar TestClient
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

it gets stuck at that point with out any response or error.
but when i try the same pointing the clinet directly to the container (IP:PORT on the docker host), the service responds fine.

# java -cp service-test-1.0-SNAPSHOT-jar-with-dependencies.jar TestClient
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
pong

i have the dtabs like bellow

/marathonId =>/#/io.l5d.marathon ;
/svc =>/$/io.buoyant.http.domainToPathPfx/marathonId ;
/grpc/staging =>/#/io.l5d.marathon ;
/svc =>/$/io.buoyant.http.domainToPathPfx/grpc ;
/svc/model-server =>/$/io.buoyant.http.domainToPathPfx/svc/staging/model-server ;
/svc/grpc.health.v1.Health =>/$/io.buoyant.http.domainToPathPfx/svc/staging/model-server ;
/srv =>/#/io.l5d.marathon/default/grpc ;
/grpc =>/srv ;
/svc =>/$/io.buoyant.http.domainToPathPfx/grpc ;
/grpc/model-server =>/srv/model-server ;
/grpc.health.v1.Health/check =>/$/io.buoyant.http.domainToPathPfx/svc/staging/model-server ;
/svc/service =>/#/io.l5d.marathon/staging/service ;
/thrift =>/svc/thrift-buffered ;
/svc =>/$/io.buoyant.http.domainToPathPfx/thrift-buffered ;
/svc/service =>/$/io.buoyant.http.domainToPathPfx/svc/staging/service ;

when i try to resolve the service /svc/service from the dtab playground i am able to resolve to the container host:port
any ideas as to why it fails when i try the client via linkerd? am i missing something basic? I have a GRPC based client which is working fine with the above config.
my linkerd config :

admin:
port: 9990

telemetry:

  • kind: io.l5d.zipkin
    host: zipkin.stg.com
    port: 9410
    sampleRate: 0.02

routers:

  • protocol: http
    servers:
  • port: 4140
    ip: 0.0.0.0
    label: linkerd_proxy
    interpreter:
    experimental: true
    kind: io.l5d.namerd.http
    dst: /$/inet/namerd1.stage-mesos.stg.net/4180
  • protocol: thrift
    servers:
  • port: 8081
    ip: 0.0.0.0
    thriftFramed: true
    interpreter:
    kind: io.l5d.namerd
    dst: /$/inet/namerd1.stage-mesos.stg.net/4100
  • protocol: h2
    experimental: true
    label: grpc
    identifier:
    kind: io.l5d.header.path
    segments: 1
    servers:
  • port: 4142
    ip: 0.0.0.0
    interpreter:
    experimental: true
    kind: io.l5d.mesh
    dst: /$/inet/namerd1.stage-mesos.stg.net/4321

namerd config :

admin:
port: 9001

storage:
kind: io.l5d.zk
zkAddrs:

  • host: zookeeper2
    port: 2181
  • host: zookeeper1
    port: 2181
  • host: zookeeper3
    port: 2181
    pathPrefix: /dtabs
    sessionTimeoutMs: 10000
    namers:
  • kind: io.l5d.marathon
    host: marathon.mesos
    port: 8080
    ttlMs: 5000

interfaces:

  • kind: io.l5d.thriftNameInterpreter
    ip: 0.0.0.0
    port: 4100
  • kind: io.l5d.httpController
    ip: 0.0.0.0
    port: 4180
  • kind: io.l5d.mesh
    ip: 0.0.0.0
    port: 4321

I enabled the recent requests but i do not see any requests updated. Am I missing something very basic.
Any pointers will be very helpful.

Hey @krishnaghatti,

The thrift protocol doesn’t have any metadata fields to store routing destination so all thrift requests will have the service name of just /svc by default. You can change this default by setting the dstPrefix property on the thrift router config. So this is the name (/svc or whatever you change it to) that you should be testing in your dtab playground. You will probably have to change your dtab to account for this.

Hope this helps!

Hi Alex,

Thank you for the response.
I was able to resolve to the container endpoint (HOST:PORT) in the dtab playground. My doubt was that I should see the test request under the recent requests tab and some messages in the logs though it fails. I see nothing in these places.
So I was not sure if Linkerd was actually getting the request.
Also I tried adding dstPrefix in the config as /svc and the dtab playground now resolves to just the service name and not the container port. Is this correct?

I’m not sure that I understand all the details here. Could you provide a screenshot of the dtab playground that demonstrates the issue?

I can resolve the service from the dtab playground. But when i try from my test client, I do not see any response/error. Thus I am unable to know if the request is actually reaching Linkerd and failing due to some routing/dtab issue or the client query is not reaching Linkerd itself.
Hope I made my question clear. Please let me know otherwise.

Hi @krishnaghatti, sorry for the delay in getting back to you. Based on the screenshot, it looks like you’re putting /svc/XXXXXXservice as the name to resolve in the playground. As I mentioned above, you need to just put /svc alone as the name to resolve.

Hi @krishnaghatti, does my suggestion help? Are you still experiencing this issue?