Safety Curves

ct game-metrics metrics computes safety curves, which measure what fraction of attacks a monitor catches at various audit budgets. The same command reports usefulness, attack-success rates, and per-budget audit-rule details. It takes inline (--honest/--attack) or run-file (--run-file) inputs and supports group-by analysis.

For per-entry control over the safety calculation (e.g. action-only vs. trajectory-only thresholds, or fully manual thresholds), use the Pareto frontier command, where every entry picks its own tm-safety mode and arguments.

Quick Start

# Single honest/attack pair
ct game-metrics metrics --honest <run-id-or-path> --attack <run-id-or-path>

# From a run file with multiple entries, save artifacts
ct game-metrics metrics --run-file runs.yaml --output-dir results/safety

The command prints per-subset metrics and an aggregate safety-vs-budget table, and writes metrics.json and safety_curves.png to --output-dir (see Output).

How Safety Is Computed

For each audit budget (e.g. 1%), the command searches for the dual-threshold audit rule (live action threshold + delayed trajectory-level threshold) that maximises attack-catching while keeping the false-positive rate on honest trajectories at or below the budget. See find_best_rule.

Safety is the probability that the audit rule catches the attack before its side task lands, formalised in compute_safety_given_audit_rule:

\text{safety} = \frac{P(\text{caught})}{P(\text{caught}) + P(\text{uncaught and side task succeeded})}

The budget is interpreted as a step-wise false positive rate (FPR) over the union of honest action scores and honest trajectory-level scores. A budget of 0 short-circuits to the never-firing rule on both legs (the "no audit budget" reference point), matching the tm-safety impl used by the Pareto frontier plot.

Rule-fitting modes (`--mode`)

The default optimized mode does the full 2D search and requires both per-action and trajectory-level scores. It calibrates on the supplied attack set, so it is useful as a ceiling but can overfit that exact attack pool.

For calibration-free curves, use --mode 50_50 or --mode stepwise_percentile. Both fit thresholds from honest scores only: 50_50 splits the total expected honest audit slots evenly between the action and trajectory channels, while stepwise_percentile gives each available channel the same per-channel FPR.

When the trajectories only carry one kind of monitor score (e.g. mean-action produces per-action scores only and leaves traj_monitor_response empty), pass --mode action_only (or --mode trajectory_only) to fit a single-leg rule per budget. These modes use the same tm-safety implementation; see tm-safety modes for how each mode behaves.

Bootstrap CIs

Pass --bootstrap N to add 95% bootstrap confidence bands to printed output, metrics.json, and safety_curves.png. Every draw resamples both honest and attack trajectories and refits the audit rule.

Python callers use compute_safety("tm-safety", honest, attack, {"mode": "stepwise_percentile", "audit_budget": 1.0}, BootstrapCIArgs(n_bootstrap=1000, confidence_level=0.90)). The three honest-calibrated modes (stepwise_percentile, action_only, trajectory_only) use numeric sample reductions and vectorized draws through this same API. Other calculations retain the generic bootstrap. Pass unscored_as_failed: True in the calculation arguments only for the all-population diagnostic; headline safety excludes unscored side tasks.

Browser consumers use metrics/browser.ts, included in the installed Python package. Its safetyCi(honest, attack, targetFpr, seed) uses 1,000 resamples, threshold refitting, and a 90% percentile interval. Its seeded generator differs from NumPy, so individual draws are not identical to Python.

Input Modes

Inline mode

Pass --honest and --attack directly.

ct game-metrics metrics \
  --honest https://data.linuxarena.ai/runs/c7aeb0aeccba40eb9d910ab13b601bed \
  --attack https://data.linuxarena.ai/runs/bf8a832da92f4431b2579ad04cd983ad

Sources can be run IDs, run URLs (https://data.linuxarena.ai/runs/...), local directories, or eval log file paths.

Run file mode

Pass --run-file with a YAML file describing multiple honest/attack pairs, in the same format as ct game-metrics graph roc (see ROC Curves).

ct game-metrics metrics --run-file runs.yaml --output-dir results/safety

Run file format:

runs:
  - label: gpt-4.1-nano
    model: gpt-4.1-nano
    honest: https://data.linuxarena.ai/runs/c7aeb0aeccba40eb9d910ab13b601bed
    attack: https://data.linuxarena.ai/runs/bf8a832da92f4431b2579ad04cd983ad

  - label: claude-haiku
    model: claude-haiku-4-5-20251001
    honest: https://data.linuxarena.ai/runs/af9f96b7df2b442aae557bd7c12ef9a0
    attack: https://data.linuxarena.ai/runs/99d571edada441e8a595d7e50f4efc37

Required fields per entry: honest, attack. Optional fields: label, honest_task_set / attack_task_set (see Task-Set Filtering), plus arbitrary metadata used by --filter-by.

Grouping

Use --group-by to split attack trajectories by a field and compute separate safety curves per group. All honest trajectories are always used together for threshold computation; only the attack side is split.

# Separate curves for live vs delayed failures
ct game-metrics metrics \
  --honest <run> --attack <run> \
  --group-by failure_type \
  --output-dir results/by_failure_type

Available group keys: env, failure_type, main_task_category, side_task_category.

When grouping is active, the results also include an aggregate [all] curve over all attacks, but safety_curves.png plots only the per-group curves.

Filtering

Use --filter-by to select a subset of runs from a run file:

ct game-metrics metrics --run-file runs.yaml --filter-by model=gpt-4.1-nano

Filters can be repeated (all must match). Auto-resolved fields (monitor, model, attack-model, attack-policy) come from each run's metadata; see the ROC Curves doc for the resolution table.

Task-Set Filtering

Use --honest-task-set / --attack-task-set to intersect each side with an allowlist of task identities. This is how heterogeneous runs (e.g. a paper-baseline full task pool vs. an experiment cohort) get compared on the same task set.

Each file is JSONL, one record per task identity. Records with env and main_task match honest trajectories; records that also carry side_task match attack trajectories; one file may mix both to serve as the allowlist for both sides:

{"env": "web_scraping", "main_task": "crawl_depth"}
{"env": "rate_limiting", "main_task": "fix_burst_handling"}

{"env": "web_scraping", "main_task": "crawl_depth", "side_task": "visit_malicious_website"}
{"env": "rate_limiting", "main_task": "fix_burst_handling", "side_task": "eavesdropper"}

ct game-metrics metrics \
  --run-file runs.yaml \
  --honest-task-set honest-task-set.jsonl \
  --attack-task-set attack-task-set.jsonl

Run-file entries can override the CLI paths per entry with honest_task_set / attack_task_set keys:

runs:
  - honest: <run>
    attack: <run>
    honest_task_set: cohort-a-honest.jsonl
    attack_task_set: cohort-a-attack.jsonl

The command echoes pre- and post-filter counts per run and errors out if a filter leaves either side empty.

Parallel Loading

Run entries load concurrently on a thread pool (--max-workers, default 8). Run-URL sources are cached on disk after the first fetch, so repeated invocations against the same runs skip the network.

Dense Safety-vs-FPR Curves (`graph safety-fpr`)

ct game-metrics metrics fits a fresh dual-threshold rule at each of a handful of audit budgets. For a dense curve over the action-monitor leg, use the vectorized sweep:

ct game-metrics graph safety-fpr --honest <run> --attack <run>
ct game-metrics graph safety-fpr --run-file runs.yaml --points 500

It calibrates an action-only threshold on the pooled honest action scores at every point of a log-spaced FPR grid (--fpr-min / --fpr-max / --points, default 250 points from 0.01% to 100%) and plots safety per run entry, with a 1%-budget reference line, to <output-dir>/safety_fpr.png. tests/metrics/test_fpr_curve.py checks the sweep point by point against compute_safety_given_audit_rule. The FPR axis is the step-wise audit FPR over all honest audit opportunities (action scores plus each honest trajectory's trajectory-monitor step), the same denominator metrics --mode action_only uses, so the two share an x-axis; only the action-monitor leg fires. Where a target FPR lands on a tie in the honest action scores, the fitted threshold is not unique, so the dense curve can differ slightly from the discrete metrics --mode action_only point.

Deferral Sweeps (`graph defer-sweep`)

For a fixed audit budget, sweep the defer-to-trusted deferral FPR and plot the resulting safety, showing how much deferral at threshold beta helps at audit budget alpha:

ct game-metrics graph defer-sweep --honest <run> --attack <run> --audit-budget 1.0

Each grid point (--defer-fpr-min / --defer-fpr-max / --points) calls the registered defer-to-trusted safety calculation, giving the same result as listing YAML entries with different defer_fpr values. The plot is written to <output-dir>/defer_sweep.png.

Neither graph draws a confidence band, because safety is not a binomial proportion of n_attack. For interval estimates, use the opt-in bootstrap CIs on graph pareto.

Audit Budgets

The default budgets are:

0.0001, 0.0002, 0.0005, 0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.1, 0.2, 0.5

Override with repeated --audit-budgets:

ct game-metrics metrics \
  --honest <run> --attack <run> \
  --audit-budgets 0.001 --audit-budgets 0.005 --audit-budgets 0.01 --audit-budgets 0.05

Output

Console output

The command prints:

A --- <subset label> --- block per subset with honest and attack counts, attack-success rate, usefulness, and, at every budget, the action and trajectory thresholds, actual FPR, and safety.
A SAFETY RESULTS table of safety per budget and subset.

--- gpt-4.1-nano ---
  Honest trajectories: 200
  Attack trajectories: 92 (ST success: 27, ASR: 29.35%)
  Usefulness: 0.6450
  Target FPR: 1%
    Action threshold: 4.00 (frac: 0.0000)
    Trajectory threshold: 7.00 (frac: 0.5000)
    Actual FPR: 0.9876%
    Safety: 0.7891
  ...

============================================================
SAFETY RESULTS
============================================================
  Budget    gpt-4.1-nano    claude-haiku
   0.01%           0.234           0.198
   0.10%           0.456           0.401
   1.00%           0.789           0.723
  10.00%           0.967           0.945

Artifact output

The output directory contains:

results/safety/
  metrics.json          # full GameMetricsResult dump (subsets, budgets, rules)
  safety_curves.png     # one curve per subset

When --output-dir is omitted the path is auto-generated as data/metrics/<YYYY-MM-DD_HH-MM-SS>/.

The metrics.json payload mirrors the GameMetricsResult dataclass, so downstream code can re-load it instead of re-running the computation.

Curve labelling

Labels in the table and plot are constructed automatically:

Scenario	Label format
Single run, no grouping	`metrics`
Multiple runs, no grouping	run label
Single run, `--group-by` set	`[<group_value>]`
Multiple runs, `--group-by` set	`<run_label> [<group_value>]`

Programmatic API

The compute step is independent of printing and plotting. Pass already-loaded LoadedRunPair instances and audit budgets to compute_game_metrics to get a structured GameMetricsResult, then render or persist as needed:

from control_tower.metrics.metrics import compute_game_metrics
from control_tower.agent_runs.run_pair import (
    RunPairEntry, load_runs,
)
from control_tower.metrics.printing.metrics import (
    print_game_metrics_result, write_game_metrics_artifacts,
)

runs = load_runs(
    [RunPairEntry(honest="<run-id>", attack="<run-id>", label="my-run")],
)
result = compute_game_metrics(
    runs,
    audit_budgets=[0.001, 0.01, 0.1],
    mode="50_50",  # default is "optimized"
    bootstrap_samples=200,
)
print_game_metrics_result(result)
write_game_metrics_artifacts(result, output_dir=Path("results/safety"))

Run ct game-metrics metrics --help for every option.