How To Emit Traces

Overview

Fullsend emits OpenTelemetry traces for every agent run. By default, traces are written to the run-telemetry.jsonl file that gets uploaded into an artifact in the workflow. Fullsend is able to send traces to a remote OpenTelemetry-compatible endpoint.

When the same OTEL_EXPORTER_OTLP_* configuration is set, fullsend eval-measure also exports newly written scores as gen_ai.evaluation.result span events on that endpoint (fail-open; local eval-measurements.jsonl always wins). See Eval Measurements for the score contract and carrier details.

Follow this guide to configure a GitHub repository or organization to send traces to a backend like MLflow, Jaeger, Grafana Tempo, etc.

Before you begin

• An OTLP/HTTP-compatible endpoint and its URL (e.g. https://mlflow.example.com:4318/v1/traces).
• Authentication credentials for the endpoint (bearer token, basic auth, or API key) if required.
• Network reachability from where runs execute (your machine or CI runners) to the backend endpoint.
• The gh CLI installed and authenticated, or access to the GitHub UI for setting variables and secrets.

Send traces from a repository

1. Determine the endpoint URL where your backend receives traces.

   For most systems this is <host>:4318/v1/traces.

2. Set the endpoint as a GitHub Actions variable on the repository.

   gh variable set OTEL_EXPORTER_OTLP_ENDPOINT --body "https://example.com:4318" --repo <owner/repo>

   OTEL_EXPORTER_OTLP_ENDPOINT gets /v1/traces appended automatically by the SDK. If your backend uses a different path, use OTEL_EXPORTER_OTLP_TRACES_ENDPOINT and specify the full URL instead.

3. Build the authentication header for your endpoint.

   The header format depends on your backend. Common patterns include:

   • Bearer token: Authorization=Bearer%20<token>
   • Basic auth: Authorization=Basic%20<base64 of user:password>

   %20 is the encoding for space, don't use spaces directly.

   Use = not :

   OpenTelemetry headers must be written in key=value format, not standard HTTP header syntax (key: value). Entries with : instead of = (e.g. Authorization: Bearer <token>) will fail fullsend's header validation and prevent traces from exporting to the remote endpoint.

4. Set the authentication header as a GitHub Actions secret on the repository.

   gh secret set OTEL_EXPORTER_OTLP_TRACES_HEADERS --body "<authentication-header>" --repo <owner/repo>

5. (Optional) Add backend-specific routing headers to the same secret.

   Some backends require additional headers. For MLflow, append the experiment ID:

   gh secret set OTEL_EXPORTER_OTLP_TRACES_HEADERS \
     --body "<authentication-header>,x-mlflow-experiment-id=<id>" \
     --repo <owner/repo>

6. (Optional) Set resource attributes to tag traces with metadata.

   gh variable set OTEL_RESOURCE_ATTRIBUTES \
     --body "deployment.env=prod,category=a" \
     --repo <owner/repo>

   Resource attributes may become filterable trace tags depending on your backend.

Send traces to an endpoint from a GitHub organization

1. Set the endpoint as an organization-level variable.

   To make it visible to all repositories:

   gh variable set OTEL_EXPORTER_OTLP_ENDPOINT \
     --body "https://example.com:4318" \
     --org <org> --visibility all

   To restrict visibility to specific repositories:

   gh variable set OTEL_EXPORTER_OTLP_ENDPOINT \
     --body "https://example.com:4318" \
     --org <org> --repos repo1,repo2,repo3

2. Set the authentication header as an organization-level secret with the same visibility.

   gh secret set OTEL_EXPORTER_OTLP_TRACES_HEADERS \
     --body "<authentication-header>" \
     --org <org> --visibility all

   To restrict visibility to specific repositories:

   gh secret set OTEL_EXPORTER_OTLP_TRACES_HEADERS \
     --body "<authentication-header>" \
     --org <org> --repos repo1,repo2,repo3

Capture conversation content

Set OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT to add the agent's text, reasoning, tool calls, and their arguments and results — when the runtime's stream provides them; Claude runs do — to each agent span, in the local file and at the endpoint. A retry that carries validation feedback also records the prompt the runner composed. Content is redacted for secrets and bounded per iteration; a string that held a terminal colour or title code or tag characters, as most coloured CLI output does, or that was not valid UTF-8 or held the replacement character (U+FFFD), is recorded as *** whole, and so is a retry prompt the runner sanitized, or cut at its size bound while it held compatibility characters or a URL with userinfo, a query or a fragment; feedback cut otherwise is still recorded. Content may still contain proprietary code or PII: tool arguments carry the commands an agent ran and the paths and patterns it used. A Write, Edit or NotebookEdit body, an Agent or WebFetch prompt and the input of MCP tools are not recorded as arguments, though a tool's result can quote what it wrote and a call's summary keeps the first 120 characters of its input; a Bash command is recorded whole (up to 8 KiB) and can embed the file content it writes. A URL in the record has its userinfo, query and fragment masked, as written in a command, text or JSON; the developer guide's tracing page lists the forms it does not read. Make sure your backend's access controls fit before enabling it.

gh variable set OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT --body "true" --repo <owner/repo>

With the claude runtime, id-bearing tool calls are visible at every level as execute_tool child spans of each agent span (tool name, call id, timing, error) — that is metadata, not content, and this variable does not affect it. The pi and codex runtimes emit none yet (see Runtimes).

Disable trace export

Remove the endpoint variable and header secret from the repository or organization to stop sending traces to the endpoint. The local run-telemetry.jsonl file continues to be produced.

gh variable delete OTEL_EXPORTER_OTLP_ENDPOINT --repo <owner/repo>
gh secret delete OTEL_EXPORTER_OTLP_TRACES_HEADERS --repo <owner/repo>

Disable the local trace file

Set OTEL_SDK_DISABLED to suppress all telemetry output, including the local file and any configured endpoint export.

gh variable set OTEL_SDK_DISABLED --body "true" --repo <owner/repo>

To disable at the organization level and then re-enable individual repositories:

gh variable set OTEL_SDK_DISABLED --body "true" --org <org> --visibility all
gh variable set OTEL_SDK_DISABLED --body "false" --repo <owner/repo>

See also

• Eval Measurements: local score JSONL and optional OTLP gen_ai.evaluation.result export on the same OTEL_EXPORTER_OTLP_* path as agent traces
• Tracing with MLflow: experiment routing, Basic auth encoding, org-level organization, and cost column caveats
• Tracing Reference: span schemas, attribute tables, and environment variable reference
• Tracing Development Guide: implementation details for contributors