# Run Artifacts and Private Data

Limit what an agent run publishes in its CI artifact, so a public repository
can take work from a private issue tracker without publishing what the agent
read there.

Every agent run uploads a run artifact: `fullsend-<agent>` on GitHub Actions,
the job artifact on GitLab CI. By default it holds the whole conversation,
which can quote a private Jira issue. On a public repository, any signed-in
user can download it. [Background](#background-what-the-artifact-holds) has
the details.

## Before you start

- A repository with fullsend installed, and write access to its
  `.fullsend/config.yaml`.
- A fullsend release that has the `fullsend prepare-artifacts` command. See
  [Upgrading](#upgrading).

## Stop Jira text from reaching the artifact

These steps use Jira. For runs started somewhere else, use that system's
source name (`github` or `gitlab`) in place of `jira`.

1. Open `.fullsend/config.yaml` in your repository and add:

   ```yaml
   artifacts:
     by_source:
       jira: metadata      # runs started from a Jira issue
   ```

   `metadata` keeps run health (cost, turns, traces) and removes the
   conversation. Use `none` to upload nothing. See
   [Choose a level](#choose-a-level).

2. Commit the change to your default branch.

3. Start an agent run from a Jira issue the way you normally do (see
   [Jira integration](jira-integration.md)).

4. Open the run's job log. On GitHub Actions, look in the
   **Prepare fullsend artifacts** step; on GitLab CI, look near the end of the
   agent job log. You should see:

   ```text
   Run artifact level metadata: run source "jira" from the normalized event
   ```

   The normalized event is the record fullsend keeps of what started the
   run. The line confirms the run was recognized as a Jira run and got
   `metadata`.

5. Download the run artifact and unpack it. Each run directory holds only
   these files:

   ```text
   fs-cod-abc123/metrics.json
   fs-cod-abc123/run-telemetry.jsonl
   ```

   `eval-measurements.jsonl` is there too when the run was scored. There is
   no `iteration-<N>/` directory and no `security/findings.jsonl`.

The log line and file list above come from running
`fullsend prepare-artifacts` locally on a sample run directory with this
config and a Jira event. If you see something else, go to
[Troubleshooting](#troubleshooting).

## Choose a level

A **level** says how much of the run's `output/` directory goes into the
artifact.

| Level | Choose it when | Artifact holds |
|---|---|---|
| `all` (default) | The run reads only public data. | Everything in `output/`. |
| `metadata` | You want run health (cost, turns, traces) without the conversation. | `metrics.json`, `eval-measurements.jsonl` and a stripped `run-telemetry.jsonl` per run directory. |
| `none` | Even metrics, span names and identifiers must stay private. | Nothing; no artifact is uploaded. |

Set the level in one of two places:

- `artifacts.content` applies to every run.
- `artifacts.by_source.<source>` applies to runs started from one **source**:
  the system where the work began. Today that is `github`, `gitlab` or
  `jira`.

A run gets the stricter of the two. `none` is stricter than `metadata`, which
is stricter than `all`. A `by_source` entry can tighten a run but never
loosen `content`.

Write source keys in lowercase: `jira`, not `Jira`.

> [!NOTE]
> **Manual and scheduled runs get your strictest level.** They have no
> source, so fullsend cannot tell which entry applies and uses the strictest
> of `content` and every `by_source` entry. With `by_source: {jira: none}`,
> a scheduled run uploads nothing. No `by_source` key can exempt these runs
> today.

## `by_source` looks only at where the run started

A run started from a GitHub issue, pull request or comment uses the `github`
level, even if its harness gives the agent Jira access on that run. The
shipped code and triage harnesses attach read-only Jira access only to
Jira-started runs, so `by_source.jira` covers them.

A custom agent that reads a private system on runs from every source is not
covered by `by_source`. Set a stricter `content` for the whole repository
instead:

```yaml
artifacts:
  content: metadata
```

### How the run's source is found

fullsend describes what started a run in a normalized event: one JSON record
with the same shape for every tracker and forge. Its `source.system` field
names the source.

1. A run with a valid normalized event uses its `source.system`, lowercased
   first, so an event that says `Jira` still matches your `jira` key.
2. A GitHub Actions run started directly by a GitHub issue, pull request,
   comment or review event is `github`. The standard GitHub stage jobs work
   this way.
3. Everything else is unknown and gets the strictest level you configured:
   - manual and scheduled runs;
   - a dispatch with no valid normalized event;
   - a dispatch whose payload names a source that its event does not back
     up;
   - a GitLab event too large to pass along.

The job log says which rule applied. A GitHub comment run logs:

```text
Run artifact level all: run source "github" (native issue_comment event)
```

## What `metadata` keeps and removes

**Removed:**

- every `iteration-<N>/` directory: conversation stream, transcripts and the
  agent's output files;
- `security/findings.jsonl`;
- every conversation-content attribute and event in `run-telemetry.jsonl`;
- free error text in telemetry: span status messages and the message and
  stack trace of exception events (the status code and exception type stay);
- pre-script text: the skip reason a pre-script prints is removed from the
  telemetry, and every skipped row in `eval-measurements.jsonl` gets a fixed
  explanation instead of its own (whether the run was skipped stays);
- any file not on the metadata list, including files added in later releases.

A telemetry or measurements file that cannot be parsed is left out of the
artifact.

**Still published:**

- `metrics.json`, copied as written: counts, token use and costs, model
  fallback reasons and runner paths;
- eval measurements (`eval-measurements.jsonl`);
- telemetry span names and timings;
- identifiers in span attributes, such as the work item id, the branch name,
  the repository URL and the harness path.

If an identifier such as a branch name is itself private, use `none`.

## What later workflows lose

The retro agent, session resumption and
[follow-up workflows](chaining-follow-up-workflows.md) download the run
artifact. Under `metadata` they get no transcripts, agent results or security
findings. Under `none` they get no artifact at all. A follow-up that reads an
agent's result needs `all` for the runs it consumes.

## Keep conversation content privately

The withheld files exist only on the runner and are gone when the job ends;
fullsend keeps no private copy. To keep conversation content, export traces
over OTLP to a backend you control
([How to emit traces](how-to-emit-traces.md)). OTLP export works at every
level.

## Keep Jira text out of the job log

Set `progress` to keep Jira issue text out of the public job log and its
annotations; `artifacts` filters only the uploaded artifact, not the log. By
default the log prints the model's reasoning and replies, tool lines, error
messages and validation output, and on GitHub Actions some of these become
annotations on the run summary page. Anyone can read the job log of a public
repository.

1. In `.fullsend/config.yaml`, next to the `artifacts` block, add:

   ```yaml
   progress:
     by_source:
       jira: quiet         # heartbeat, counts, error types and the result only
   ```

2. Commit the change and merge it the way you merge other config changes.

3. Start a run from a Jira issue.

### Check that it worked

Open the run's job log. Near the top, before the agent starts, you should see:

```text
Progress log: quiet (run source "jira" from the normalized event)
```

A run started from a GitHub issue in the same repository prints no
`Progress log` line, because it still logs at `full`.

While the agent runs, a quiet log looks like this (a triage run, trimmed):

```text
→ Agent: claude-sonnet-5-5 (v2.1.306)
  ! Retry 1/10 (delay 522ms)
  ⏳ Agent running (30s elapsed, 9m30s remaining)
→ Result: success
    Turns: 7
    Cost: $0.1794
    Tokens: in=10 out=1871 reasoning=49 cache_create=62561 cache_read=42757

  ✓ Agent exited with code 0 (33.7s)
  ...
  ✓ Validation passed (1.5s)
```

The log keeps the heartbeat, event states such as `Task — running`, token
counts, error types, `Validation passed` or `Validation failed`, and the
result. It leaves out reasoning, replies, tool lines and validation output. A
stuck or failed run still shows the heartbeat, the error type and the exit
code.

If the agent CLI wrote anything to stderr, you also see one line like:

```text
Agent CLI stderr: 3 line(s) kept out of the log at quiet, in agent-stderr.log
```

The stderr itself is in `agent-stderr.log` in the iteration directory, next
to `output.jsonl`. The run artifact includes that file only when
`artifacts` is `all` for the run, just like the transcripts. With
`artifacts.by_source.jira: metadata` from
[Stop Jira text from reaching the artifact](#stop-jira-text-from-reaching-the-artifact),
it stays on the runner.

### What else to know

- `tools` hides only reasoning and replies. Tool lines and task labels can
  still carry text from the source, so use `quiet` for a source that isn't
  public.
- The level changes only what is printed. The next iteration still gets the
  full validation feedback, and OTLP export and Level 3 content capture (the
  tracing level that adds conversation content to spans, see
  [Content capture](../infrastructure/distributed-tracing.md#content-capture-level-3))
  still receive everything ([Keep conversation content privately](#keep-conversation-content-privately)).
- The level covers the live job log only. The status comment fullsend posts
  on the issue or pull request when a run ends still carries the full run
  error, and output from your own pre-scripts and post-scripts prints as
  before.
- For every level and for how the level is chosen, see
  [`progress`](../../reference/config-reference.md#progress).

## Troubleshooting

Find the message in the job log. On GitHub Actions the warnings also show as
annotations on the run.

| Message | What it means | What to do |
|---|---|---|
| `Run started from jira: the run artifact publishes the full transcript (artifacts level "all"). If jira holds data that is not public, set artifacts.by_source.jira to metadata or none in config.yaml; set it to all to silence this warning.` | A run from an external tracker uploaded everything, and your config makes no choice for that source. | Set `artifacts.by_source.jira` to `metadata` or `none`. To keep full transcripts on purpose, set it to `all`. See [the full transcript warning](#the-full-transcript-warning). |
| `Run artifact level none: run source unknown (no valid normalized event); using the strictest of artifacts.content and every artifacts.by_source entry` | The run has no source (manual, scheduled, or a dispatch without a valid event), so it got your strictest level. The level in the message is the one applied. | Nothing, if that level is what you want. Otherwise loosen the strictest `by_source` entry; no key exempts these runs. |
| `invalid artifacts.by_source key "Jira": source keys are lowercase with no surrounding spaces; use "jira"` | A source key is not lowercase. The run uploads no artifact. | Rename the key as the message says. |
| `invalid artifacts.content "meta": must be one of all, metadata, none` | A level is misspelled. The run uploads no artifact. | Use `all`, `metadata` or `none`. |
| `Run artifact level could not be resolved; no run artifact is uploaded` (on GitLab: `WARNING: run artifact level could not be resolved; no run artifact is uploaded`) | The config or the event could not be read. The line above it in the log gives the cause. | Fix the cause named in the log. |
| `The installed fullsend CLI predates prepare-artifacts, so no run artifact is uploaded. Set the version input to at least the fullsend release that ships this action.` | The workflow pins an older fullsend CLI through the action's `version` input. | Raise `version`. See [Upgrading](#upgrading). |
| `The fullsend CLI is not installed (check the install step), so no run artifact is uploaded.` | The install step failed earlier in the job. | Fix the install step. |
| No artifact, no warning | The level is `none`, or on GitLab the job was cancelled or timed out before the end of the job script. | Check the `Run artifact level` line in the log. |
| `Progress log: quiet (the progress setting could not be resolved: invalid progress.by_source key "Jira": ...)` | A `progress` level or source key is invalid, so the run logs at `quiet`. The text after `resolved:` names the cause. | Fix the key or level it names; see the next two rows. |
| `invalid progress.by_source key "Jira": source keys are lowercase with no surrounding spaces; use "jira"` | A `progress.by_source` key is not lowercase. | Rename the key as the message says. |
| `invalid progress.verbosity "loud": must be one of full, tools, quiet` or `invalid progress.by_source.jira "loud": must be one of full, tools, quiet` | A `progress` level is misspelled. | Use `full`, `tools` or `quiet`. |
| `Progress log: quiet (run source unknown: no valid normalized event)` | The run has no source (manual, scheduled or local, or a dispatch without a valid event), so it got the strictest of `progress.verbosity` and every `progress.by_source` entry. The level in the line is the one applied. | Nothing, if that level is what you want. Otherwise loosen the strictest `by_source` entry; no key exempts these runs. |

When fullsend cannot work out the level, it uploads nothing rather than the
raw `output/`.

### The full transcript warning

The warning appears when all of these are true:

- the run started from an external tracker: any source other than `github`,
  `gitlab`, `forgejo`, `manual` or `schedule`;
- the run uploads at `all`;
- your config sets neither `artifacts.content` nor a `by_source` entry for
  that source;
- the repository is not known to be private. An `internal` repository or
  project still warns, because every member of the enterprise or signed-in
  user of the instance can read it. When visibility is unknown, it warns.

To silence it, make an explicit choice in `.fullsend/config.yaml`, either for
the source or for every run:

```yaml
artifacts:
  by_source:
    jira: all           # or metadata / none
```

```yaml
artifacts:
  content: all          # explicit, so no warning for any source
```

## Upgrading

- **GitLab artifact path.** After you re-sync the `.fullsend` scaffold, the
  agent job uploads `fullsend-artifact/` instead of `output/`. Anything that
  downloads the job artifact by path must look under `fullsend-artifact/`.
  Add `fullsend-artifact/` to your `.gitignore` next to `output/`.
- **GitHub Actions `version` input.** If your workflow pins the fullsend CLI
  through the fullsend action's `version` input, set it to this release or
  later. An older CLI has no `prepare-artifacts` command, so the run uploads no
  run artifact and logs the "predates prepare-artifacts" warning.
- **GitLab jobs killed mid-run.** The artifact is prepared at the end of the
  job script. A job that is cancelled or times out before then uploads no
  run artifact.

## Background: what the artifact holds

By default the run artifact holds the run's whole `output/` directory:

- the raw conversation stream and transcripts of every iteration;
- `agent-stderr.log` beside them, written only at progress level `quiet`;
- the agent's output files;
- `run-telemetry.jsonl`, with conversation content when content capture is on;
- `security/findings.jsonl`.

Tool results in those files can quote what the agent read, such as the text
of a private Jira issue. On a public repository, any signed-in user can
download the artifact until it expires. The output scan masks secret
literals; it does not remove personal data, customer names or other private
business text.

`fullsend run` leaves `output/` untouched. At the end of the job,
`fullsend prepare-artifacts` builds a filtered copy at the level you set, and
CI uploads only that copy. The field reference is
[`artifacts`](../../reference/config-reference.md#artifacts).
