Skip to main content
The downloadable launcher is ordinary Python calling the released synth CLI. Inspect tutorial.py before running it. There is no browser credential form, hidden paid backend, polling daemon or automatic resubmission.

​
Work and spending

The course’s python3 tutorial.py ... launch command is a preview unless you add --execute. Direct synth run start and synth train start commands submit work immediately unless you pass --dry-run; they do not use the course launcher’s preview or workspace ceiling. The workspace’s trial limit counts reserved attempts, including uncertain submissions. Preview larger workloads before launching them and keep the returned run IDs. The reference qualification and delivery used 205 reserved trial slots, including alternative-model qualification, failed attempts, the cancellation canary and the retained-baseline invocation. This is not a required bill for repeating the main path. The core comparison/qualification/training/exam plan uses 136 slots within its explicit 160-slot workspace limit; starter, six calibration controls and adaptations are separate authorized work. Never silently borrow allowance from another workspace. Observed actor medians ranged from 21 to 73 seconds across the matched learning arms; judge medians ranged from 55 to 116 seconds. Allow several minutes for a judged cohort and additional queue/setup time. The recorded two-update training took about 15½ minutes and had 13 missing scores; these observations are not a service-level guarantee. No dollar invoice was available, so the report exposes measured per-role token counts and leaves dollar cost unknown. The core tasks prescribe one CPU, 4 GiB memory, a four-minute actor deadline and a three-minute verifier deadline. Setup, queueing, artifact transfer and cleanup add wall time. These are task-level limits; the course does not override them at run level. A model/harness deadline is not an assurance that a queued run finishes by then.

​
Continue without starting again

​
Direct CLI starter

The first lesson uses the CLI directly. It does not create the course launcher’s state.json or operations/ directory. Keep the upload/start receipts, exact task reference, profile, plan and idempotency key together. With the returned run ID, read progress without submitting anything:
After it is terminal, export to a new directory with synth run export <run-id> --output ./first-review-evidence. To stop your confirmed run, use synth run cancel <run-id> --yes, then read show until it is terminal. Cancellation is not an instantaneous settlement.

​
Course launcher

state.json, operations/ and exported evidence belong to this experiment. Keep them private. Status reads the exact recorded runs. Reloading a web page, closing a terminal or rerunning status does not submit a job. A completed operation reuses its receipt; the launcher refuses changed inputs under the same name and overlapping local writers.

​
A submission lost its receipt

If a command times out or loses its response, the operation is uncertain, not failed. Its reserved work remains counted. Preserve the original arguments and idempotency key (the launcher saves them in operations/). synth run list and synth run show are read-only investigation tools, but a similar name, dataset or timestamp alone does not establish that a run belongs to this submission. Have your organization’s operator reconcile the original request and exact run identity if the receipt is missing. Do not create a new name/key, rerun the start command, or edit state.json to bypass the stop. The launcher has no command to adopt an uncertain submission. Its status, cancel and export operations only address runs whose receipts it recorded. Once the exact run ID is confirmed, use the direct CLI read/cancel/export commands above to settle and preserve that run; keep the uncertain workspace as evidence. The launcher will continue to refuse cleanup until its uncertainty is resolved through supported assistance. This is a safe stop, not an automatic recovery claim.

​
Read the phase before fixing anything

  • No actor output and setup failure: investigate environment/harness setup, not the review prompt.
  • Actor output but missing score: inspect the verifier/judge and reward contract.
  • Low score with a completed review: check the actual review and assessment before blaming infrastructure.
  • Training progress without saved checkpoints: inspect synth run updates and synth run checkpoints; progress alone is not a usable model.
The reference starter encountered both a setup problem and a numeric reward-file packaging error before its successful run. Those attempts remain in the evidence; they are not counted as successful reviews.

​
Verify that the evaluation used the checkpoint

Save the final checkpoint ID from your training run, then check both its origin and downstream use:
Require the training checkpoint entry’s checkpoint_id to equal the chosen ID, and checkpoint show’s checkpoint.source_run_id to equal your training run. Check the completed update count and saved time. In the evaluation’s show response, job.model must equal that checkpoint ID—not just the base model name. This last check establishes the requested model, not actual consumption by itself. For actual-use readback, the checkpoint’s runs response must include your evaluation run ID with role: "sampled". Its groups response must include all expected sampling groups for that run. Follow pagination when has_more is true. synth run groups and synth run trials default to the latest attempt: use --attempt <attempt-number> from synth run show and join on the full sampling-group ID when checking earlier attempts. synth run trials <evaluation-run-id> --attempt <attempt-number> --group <sampling-group-id> --json reads that exact group. Retries are physical work and may repeat tasks; do not sum them as new logical exam tasks. For each matched group, compare expected_trial_count, settled_trial_count and missing_reward_count with its trial records and your frozen task/sample plan. Missing groups, missing scores, a different ID or an unavailable response mean this check has not passed. Do not substitute an aggregate usage count that might belong to another run. Retain these responses with the export. Checkpoint use proves which model was sampled; only the fixed comparison and inspection of actual outputs can establish improvement.

​
Cancel only your named run

The first command requests cancellation; the second must show a terminal state before you call it settled. A terminal run is not cancelled again. Export finished evidence before discarding local files. In the recorded early-cancellation exercise, cancellation was requested one second after admission and became terminal about 15 minutes later. Treat that as an observed recovery delay, not a promise or an instantaneous stop. Keep the exact run ID and continue status checks; never submit a replacement merely because the display still says cancelling.
Cleanup checks every owned recorded run and refuses an active or uncertain operation. It does not delete immutable private task/dataset versions or retained run records: the released CLI has no registry-delete command. It touches neither global configuration nor other runs. Managed execution handles its temporary environments; preserve the experiment’s local receipts as the recovery record. Do not advertise this as an account wipe or a spend refund. For an explicit future smoke check, run just the one-trial starter with a new recorded experiment identity and authorized budget. This is the course canary, not a scheduled service. The documentation maintainers own updates to its pinned assets; a failed canary should block a new success claim until repaired and exercised.