Skip to main content
Your terminal and the remote run have separate lifetimes. Closing a terminal, losing a connection or reaching a local event-follow timeout does not establish that remote work stopped.

​
Keep the submission receipt

Save the run ID, exact input references, request and idempotency key. Inspect the same run after reconnecting:
Event following is bounded. Reaching its timeout stops waiting locally, not the remote run. Use the command’s documented output and exit behavior in automation.

​
An unclear submission is not permission to resubmit

If submission times out after sending a request:
  1. Preserve the original request and idempotency key.
  2. Read the returned run ID, or inspect recent runs for the original submission.
  3. If an idempotent retry is needed, reuse the exact request and key.
  4. Do not create a new key merely because the first response was unclear.
A new key is a new intent and can create duplicate paid work. A failed dry run, a rejected request and a lost submission response are different situations.

​
Stop a run deliberately

Cancellation requests a stop; read back the final state. Do not assume cancellation deletes completed trials or saved checkpoints.

​
Recover training evidence

A failed training run can still have real optimizer updates and saved checkpoints. Inspect them before deciding what to do next. An input checkpoint, updates made by the current run, and records from a recovered execution segment are not the same facts. Training explains the distinction between evaluating a saved model and resuming its training state. Do not blindly restart an entire plan or count recovered work twice.

​
Preserve results

Export a settled run into a new destination and read its coverage report. Missing artifacts are not empty artifacts. If a signed download link expires, request fresh authorized access; do not share or preserve the signed URL. Recovery is complete when the original work, its cost and its evidence are accounted for—not when a replacement happens to succeed.