> ## Documentation Index
> Fetch the complete documentation index at: https://synthlabs.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Build a reviewer you can evaluate

> Run useful work, challenge its assessment, make one change, and decide whether training helped.

Imagine you are building a code reviewer for **DispatchDesk**, a fictional application with permissions, billing and queue policies. You want it to catch policy regressions without inventing bugs.

In this course, you will run the reviewer, inspect its findings, define what counts as correct, and compare changes on work it did not train on. The complete teaching codebase is supplied under the MIT license.

Code review is the example throughout. For a support agent, the task might be a customer case and a resolution policy; for a document analyst, a document and its source records; for an engineering assistant, project documents and requirements. The task, agent profile and assessment change, while the workflow stays the same.

| You bring | Synth does | You get |
| - | - | - |
| Work instructions, source and an output contract | Runs the chosen agent in each task environment | Review files, execution records and failures |
| A qualified verifier or judge and a task collection | Repeats the work and records assessments | Evidence for comparing agents |
| Practice data, a trainable agent and an explicit plan | Optimizes the model and saves checkpoints | A model to evaluate—not a guaranteed improvement |

**Agent = Model + Harness + Context.** The task environment is separate. A profile pins the agent; a task supplies work and an environment; a dataset selects task versions; a run records what happened. You remain responsible for relevant tasks, lawful data use, meaningful assessment and the promotion decision.

## Choose how to begin

<CardGroup cols={2}>
  <Card title="Explore the recorded work" icon="magnifying-glass" href="/tutorials/first-review">
    See the source, a real review and its assessment.
  </Card>

  <Card title="Run the course" icon="terminal" href="/tutorials/first-review#run-it-yourself">
    Download the complete inputs and use the released CLI in a fresh workspace.
  </Card>
</CardGroup>

<span id="live-prerequisites" />

## Before you start

[Create a Synth account](https://app.synthlabs.ai), [install the CLI](/getting-started/install) and [sign in](/getting-started/authentication).

You will also need Python 3.10 or later and a macOS or Linux terminal for the example scripts. The downloads include the tasks, agent profiles and assessment tools.

Start with one trial. Later lessons show how to preview larger experiments and keep their inputs and results together.

## Six connected lessons

1. [Get a useful review](/tutorials/first-review): read an actual finding and reproduce it.
2. [Challenge the score](/tutorials/assessment): expose a verifier blind spot and calibrate a judge.
3. [Bring your own failure](/tutorials/adapt): build a complete, inspectable task.
4. [Compare one deliberate change](/tutorials/compare): freeze inputs and inspect paired evidence.
5. [Train and test transfer](/tutorials/train): distinguish an update, a checkpoint and improvement.
6. [Use or reject the result](/tutorials/use): apply the decision and retain a rollback path.

Every lesson uses the same [course project](/downloads/review-lab.project.json) and [manifest](/downloads/tutorial-manifest.json). You can start later with the supplied files; recorded run IDs are provenance, not prerequisites or resources in your account. [Operation and recovery](/tutorials/operate) covers interruptions and cleanup.

## Download and unpack a project

The [versioned example release](https://github.com/SynthLabsAI/synth-examples/releases/tag/v2026.09.27.2) contains ordinary ZIP downloads and SHA-256 checksums. Inspect the [complete source](https://github.com/SynthLabsAI/synth-examples) before executing it. Download the starter into a new directory:

```bash theme={null}
mkdir synth-starter
cd synth-starter
curl --fail --location --remote-name https://github.com/SynthLabsAI/synth-examples/releases/download/v2026.09.27.2/review-permissions.zip
curl --fail --location --remote-name https://github.com/SynthLabsAI/synth-examples/releases/download/v2026.09.27.2/checksums.txt
shasum --algorithm 256 --check --ignore-missing checksums.txt
unzip review-permissions.zip -d review-permissions
cd review-permissions
```

The checksum check must say `review-permissions.zip: OK`. For the complete course, use `review-lab.zip` and extract into a new `review-lab` directory instead. Downloading and extracting does not authenticate or submit work. Keep the release tag and checksum file with your inputs; do not extract over an existing project.

### Alternative: inspect or unpack the readable JSON project

Download either the [starter project](/downloads/review-permissions.project.json) or the [complete course](/downloads/review-lab.project.json) into a new folder, preserving its `.project.json` filename. These are readable JSON files containing the complete source and a checksum for every file—not executable installers or encoded archives.

In that folder, run the command below. It validates all file paths and checksums, refuses to replace an existing project, and writes the supplied files. It does not authenticate, submit work or run downloaded code.

```bash theme={null}
python3 - <<'PY'
"""Unpack a public JSON project into a new directory; never execute its files."""
import hashlib
import json
import shutil
from pathlib import Path, PurePosixPath


def unpack(source, parent):
    if source.stat().st_size > 20 * 1024 * 1024:
        raise ValueError('Project exceeds the 20 MiB input limit')
    project = json.loads(source.read_text(encoding='utf-8'))
    if project.get('schema') != 'synth.tutorial-project.v1':
        raise ValueError('Unsupported project format')
    if project.get('name') not in {'review-lab', 'review-permissions'}:
        raise ValueError('Unexpected project name')
    files = project['files']
    if not isinstance(files, dict) or not files or len(files) > 2000:
        raise ValueError('Invalid project file inventory')
    checked = []
    for name, record in files.items():
        path = PurePosixPath(name)
        if path.is_absolute() or '..' in path.parts or '\\' in name or str(path) != name or name == '.' or any(ord(c) < 32 for c in name):
            raise ValueError('Unsafe project path: ' + name)
        data = record['text'].encode('utf-8')
        if hashlib.sha256(data).hexdigest() != record['sha256']:
            raise ValueError('File checksum mismatch: ' + name)
        checked.append((path, data))
    paths = {path for path, _ in checked}
    if any(parent in paths for path in paths for parent in path.parents):
        raise ValueError('Conflicting file and directory paths')
    destination = parent / project['name']
    destination.mkdir(exist_ok=False)
    try:
        for path, data in checked:
            target = destination / str(path)
            target.parent.mkdir(parents=True, exist_ok=True)
            with target.open('xb') as stream:
                stream.write(data)
    except BaseException:
        shutil.rmtree(destination)
        raise
    return destination


if __name__ == '__main__':
    projects = list(Path('.').glob('*.project.json'))
    if len(projects) != 1:
        raise SystemExit('Use a new folder containing exactly one downloaded .project.json file')
    print('Project ready:', unpack(projects[0], Path('.')))
PY
```

For the starter, continue from the new `review-permissions` directory. For the full course, continue from `review-lab`. Keep the downloaded JSON and [shared manifest](/downloads/tutorial-manifest.json) as the record of your inputs.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.