Automate a workflow

Use customer credentials, immutable inputs, JSON and safe submission semantics in scripts.

Automate the same workflow you first verified interactively. Pin the inputs, save each returned run ID and make ambiguous outcomes explicit.

Authenticate without exposing credentials

Use a user-scoped credential supplied through your CI secret manager. For an opaque personal access token, the current CLI also requires its owner’s UUID:

export SYNTH_ACCESS_TOKEN="<customer-token>"
export SYNTH_USER_ID="<customer-user-uuid>"
synth status --json

A JWT carries its owner. These placeholders illustrate environment variables; do not put real values in source files, shell history or logs. Interactive users should use browser sign-in.

Check the intended service and identity before submitting work. A locally valid credential is not proof of remote access. Verify an authenticated read against the resource you intend to use.

Pin the inputs

Keep the exact task or dataset version, profile version or local profile bytes, judge selection and plan in the experiment record. A mutable name can resolve to a newer version on the next invocation.

Inspect first:

synth run start \
  --dataset <dataset-ref> \
  --actor @reviewer.json \
  --plan @eval-plan.json \
  --dry-run \
  --json

A dry run describes its validation boundary. It is not proof of inference availability or a successful task.

Give each submission a stable identity

For a new experiment, choose a unique idempotency key and retain it with the exact request:

synth run start \
  --dataset <dataset-ref> \
  --actor @reviewer.json \
  --plan @eval-plan.json \
  --idempotency-key "<unique-experiment-key>" \
  --json

If the response is uncertain, inspect before retrying. Where idempotent replay is supported, retry the same request with the same key. A new key represents new work, not recovery of an ambiguous submission.

Do not assume every mutating command has the same replay guarantees.

Read machine output, not a human table

Use --json where the command supports it. Check process exit status and the documented result fields. For example, a completed status --json inspection may report signed_in: false; a zero process exit is not a claim that authentication succeeded.

Run readers and Registry commands have different JSON shapes. Do not assume every object uses a top-level state or the same pagination envelope. The CLI reference identifies supported flags.

synth run events --follow --json emits one event per line. Preserve durable event identities; filtering can produce sequence gaps without missing data.

Preserve evidence and failure meaning

Export terminal run evidence into a new directory. Verify its manifest and coverage before treating it as complete. Missing scores, unavailable reads, cancellation and assessed task failure are different states.

A script should report an uncertain submission or incomplete assessment, not silently resubmit or turn it into a score of zero. See retry safety and result interpretation.