ct run monitor
Score a finished eval log with more monitors, writing a monitored copy
Usage
ct run monitor [OPTIONS] SOURCE
Arguments
| Argument | Type | Required |
|---|---|---|
SOURCE | text | Yes |
Options
| Option | Description |
|---|---|
-m, --monitors TEXT | The monitors, as YAML: a list of packaged prompt names or mappings of Monitor fields, e.g. '[simple, {prompt: strong, model: judge, reads: [tool_calls, tool_results, text], samples: 2}]'. model names a role bound with --model-role. simple asks about an action, trajectory about a whole run, so pick the one --kind is for. [required] |
--kind [action|trajectory] | Score every action before it ran, or the whole trajectory once. [default: action] |
--model-role TEXT | Bind a model role, as name=model or name={model: ..., <generate config>, model_args: {...}} in YAML or JSON (multiple allowed). A monitor answers on the role its model names, monitor by default, which a bound trusted binds too unless monitor is given. |
-o, --save-dir PATH | Where to write a base log's monitored copy (default: ./data/monitoring); a monitored copy given as SOURCE is updated in place. |
--upload / --no-upload | Upload the monitored copy as a run, linked to the run it was derived from. |
--hawk | Score a hawk://<eval-set-id> SOURCE in a Hawk scan instead of locally; the scores stay in the scan's results unless --save-dir or --upload asks for the monitored copy. |
--hawk-arg TEXT | A Hawk option as key=value (repeatable); use=<repo>@<ref>,... picks the versions the scan clones, and <setting>=<owner>/<repo>@<ref> also names a setting's repo. |
--help | Show this message and exit. |