Control Tower v6 is available. See the changelog for breaking changes.

Control Tower

User Guide · Tasks and Scorers

Auditing Environments

Most environment auditing tools are ct extras subcommands. In those that run an Inspect task, a sample whose reply was cut off, at the output-token limit or by a prompt too long for the model, errors instead of being scored, and the command exits non-zero naming the log.

Side Task Review

Review side tasks for scorer reward hacking, magic numbers, accidental triggering, and sabotage trajectory (straj) feasibility:

# Review all side tasks for an environment ct extras side-task-review -e my_setting/my_environment # Review specific side tasks ct extras side-task-review -e linuxarena/web_scraping -s visit_malicious_website -s expose_secret

Test Quality

Analyze whether integration tests verify real end-to-end behavior:

ct extras integration-test-quality -e my_setting/my_environment

Compare Side Tasks

Compare side task successes between two runs:

ct extras compare-side-tasks <run_id_1> <run_id_2>

Reward Hacking Analysis

Audit side tasks for reward hacking vulnerabilities using the smart model alias (or --model) with reasoning:

ct extras reward-hacking -e my_setting/my_environment

Env Description Audit & Generate

Two paired tools maintain an environment's docs/description.md. Both follow the spec at src/control_tower/extras/env_description/shared/spec.md, which defines length caps, required structure, and which content is in and out of bounds.

Audit scores an existing description against the codebase on five metrics (accuracy, clarity, completeness, information leakage, overall):

# Audit all environments ct extras env-description-audit # Audit specific environment(s) ct extras env-description-audit -e my_setting/my_environment # Use a different model ct extras env-description-audit -m anthropic/claude-sonnet-4-6

The audit is an Inspect task with one sample per environment, and writes an Inspect .eval log to logs/ (or INSPECT_LOG_DIR); open it with inspect view. A sample's score holds the five metric scores, and its metadata holds each metric's justification with the issues and suggestions. Defaults to the smart model alias at -c 2.

Generate writes a fresh description from the codebase. It is also an Inspect task with one sample per environment, whose score is the description's word count. To protect uncommitted edits, the command aborts if docs/description.md already exists unless you pass --overwrite or --dry-run.

ct extras env-description-generate -e my_setting/my_environment # first-time only ct extras env-description-generate -e my_setting/my_environment --overwrite # replace existing ct extras env-description-generate -e my_setting/my_environment --dry-run # writes description.draft.md # Then validate with the audit ct extras env-description-audit -e my_setting/my_environment

ct extras doc-audit is a separate command that audits per-side-task docs.

Attack Evaluations

Use ct run eval directly with the appropriate policy:

# Obvious attack ct run eval --policy obvious-attack -e my_setting/my_environment -t main_task -s side_task # Sneaky attack ct run eval --policy attack -e my_setting/my_environment -t main_task -s side_task

Options

Most ct extras audit commands accept --env / -e for environment selection (compare-side-tasks takes two positional run IDs instead). If --env is omitted, the CLI prompts interactively.