Three Ways to Trace End-to-end

On the left, we see a trace only with client and server side spans for a Lambda function trigger. On the right, we see more spans including ALB, and outgoing requests to S3 and to a Redis server.
  • Being able to accept and/or propagate the distributed tracing context
  • Participating into the incoming trace and producing spans

Transforming the trace context

  • It is simple to implement if downstream services consistently are supporting a different header.
  • If the converter is implemented as a proxy server, it doesn’t require any changes to the existing services.
  • When combined with a vendor-agnostic collection pipeline like the OpenTelemetry collector, services can keep publishing trace spans in vendor specific formats, and the collector can transform and send them to the same service. For example, if service A is using Jaeger and B is using Zipkin, OpenTelemetry collector can both accept spans coming from the services and transform all the data to be sent to Jaeger or Zipkin.
  • If trace headers are not convertible due to reasons like different identifier lengths or different fields, this option is not possible to implement.
  • If downstream services support a variety of different headers, this option is very complicated to implement and maintain.

Linking traces

A trace collected for serviceA.Lookup makes a request to serviceB.Query. The client span links to another trace that contains the serviceB.Query’s server-side traces.
  • It doesn’t require any significant changes to the existing instrumentation and propagation formats.
  • Allows different data access levels. For example, sensitive traces can be split as a linked trace with different access levels.
  • Requires custom ways to propagate back the “linked trace” and record it.
  • Not many distributed tools support visualizing or querying links. Even when querying supported, querying is very limited in comparison to having all the spans under the same trace.
  • Hard to produce automatic service maps based on trace data, you may need to build custom solutions.
  • Propagating the downsampling decision is still a challenge unless all components are using the same trace header.

Partial traces

--

--

--

See rakyll.org for more.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Motivational Story : He got his first placement success after being rejected by 22 companies

First encounter with celery: tasks

Deploying Machine Learning Model On The Top Of Docker Container

Three types of Agile adoption according to Alan Ford

Tari Protocol Discussion #9

Active Record Explained

Mobile App Testing Strategies

CS371p Spring 2022: Joshua Cho

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Jaana Dogan

Jaana Dogan

See rakyll.org for more.

More from Medium

The Art of Automating Automation

Redis — the optimal solution for high-performance data retrieval

Title pic

HayStack: Paper Summary

Ace the System Design Interview — Distributed ID Generator