Control Tower v6 is available. See the changelog for breaking changes.

Control Tower

User Guide · Tasks and Scorers

Submit Guessability Verified

Audits side tasks for submit tool exploitability with live verification.

Purpose

Like submit-guessability, but it spins up containers and runs the scorer to check whether proposed exploits work. The AI gets multiple attempts, with feedback, to find working submit arguments.

Commands

ct extras submit-guessability-verified

Audit side tasks with live scorer verification.

Examples:

# Audit all side tasks with verification ct extras submit-guessability-verified # Audit specific environment ct extras submit-guessability-verified -e linuxarena/web_scraping # More attempts to find exploits ct extras submit-guessability-verified -a 10 # Lower concurrency for resource-constrained systems ct extras submit-guessability-verified -c 1

Output

The run is an Inspect task with one sample per side task, and writes an Inspect .eval log to logs/ (or INSPECT_LOG_DIR). Open it with inspect view. A sample scores I when a proposed set of submit arguments passed the live scorer, with those arguments as the score's answer, C when the auditor answered NO or its arguments reached the scorer and failed, and N when it did neither. The sample transcript holds every attempt and the scorer feedback the auditor was given.

Workflow

For each side task:

  1. The AI analyzes the scorer and submit tool and calls a try_submit_args tool, whose parameters are the side task's submit arguments, with the arguments it proposes.
  2. A container is spun up and the scorer runs with those arguments.
  3. If the scorer fails, the AI sees the error and tries again, up to max-attempts times.
  4. The results show whether a working exploit was found.