Useful prompts for task work

Act as a senior Terminal-Bench task author.

Review the task idea and propose a hard-but-fair implementation plan.

Include:
1. The user-facing objective
2. The environment and starter state
3. The core technical trap that makes the task difficult
4. What instruction.md must disclose for fairness
5. What the hidden tests should verify independently
6. The oracle strategy
7. How to prove nop receives reward 0
8. Likely reward-hacking or answer-leakage risks

Do not write a step-by-step solution into the task instructions.

Task idea:
[PASTE TASK IDEA HERE]

Review this Terminal-Bench task as a strict expert reviewer.

Inspect instruction.md, task.toml, environment/, solution/, and tests/.

Check:
- Every tested requirement is disclosed in instruction.md
- The task is solvable without hidden assumptions
- The oracle solves the task and should score exactly 1
- Nop should score exactly 0
- Tests verify behavior independently instead of copying oracle logic
- Tests cannot be bypassed with hardcoded outputs or fixture leakage
- The Docker environment is deterministic
- Paths, permissions, timeouts, and dependencies are correct
- The task remains difficult without becoming ambiguous

Return:
1. Blocking issues
2. Non-blocking improvements
3. Exact files and changes needed
4. A final submit / do-not-submit verdict

Task folder:
[PASTE PATH OR ATTACH TASK FILES]

Debug why the Terminal-Bench oracle did not receive reward 1.

Use the attached task folder and complete agent/job logs. Trace the failure from container build through solution execution and test verification.

Check:
- Docker build and runtime errors
- Missing packages, files, permissions, or executable bits
- Incorrect paths or working directory assumptions
- Oracle commands that silently fail
- Tests that exit before writing /logs/verifier/reward.txt
- Tests that reject valid output or depend on unavailable state
- Timeout, network, architecture, or line-ending issues

Make the smallest correct fix. Do not weaken a legitimate task requirement merely to make the oracle pass.

After editing, run the oracle again and report:
1. Root cause
2. Files changed
3. Verification evidence

Task folder:
[PASTE PATH]

Logs:
[PASTE OR ATTACH COMPLETE LOGS]

Debug why the Terminal-Bench nop agent received reward 1.

Inspect the task, starter state, test script, fixtures, and reward-writing logic.

Look for:
- Expected output already present in the image or starter files
- Tests that do not execute the required behavior
- Checks that accept empty, stale, or hardcoded output
- Incorrect success conditions or shell exit-code handling
- Fixtures or oracle artifacts visible to the agent
- Tests that always write reward 1

Strengthen the verifier so nop scores 0 while the legitimate oracle still scores 1. Prefer independent behavioral checks over exact implementation matching.

Run both oracle and nop after the fix and report the evidence.

Task folder:
[PASTE PATH]

Logs:
[PASTE OR ATTACH COMPLETE LOGS]

This Terminal-Bench task was rated TRIVIAL or EASY. Make it meaningfully harder while keeping it fair and solvable.

Analyze the task files and agent trajectories to identify why agents succeed.

Consider:
- Removing solution leakage and excessive hand-holding
- Adding realistic edge cases or hidden instances
- Requiring integration across multiple files or tools
- Strengthening independent verification
- Adding one technically meaningful trap
- Preventing hardcoded or shortcut solutions

Do not:
- Add ambiguity
- Hide requirements that tests enforce
- Depend on flaky network behavior
- Make the oracle unreliable
- Add arbitrary busywork

Propose the smallest high-impact changes first. Then implement them and verify oracle=1 and nop=0.

Task folder and evaluation report:
[ATTACH FILES AND REPORT]

This Terminal-Bench task is marked not solvable because some tests were not passed by any agent.

Determine whether each failing test indicates:
- An oracle bug
- An environment or dependency failure
- An undisclosed requirement
- An over-strict or incorrect assertion
- A genuine agent failure on a fair requirement

Fix correctness and fairness defects while preserving the core challenge. Every required behavior must be disclosed, and the oracle must pass every test.

Return a test-by-test diagnosis, implement the necessary changes, and verify oracle=1 and nop=0.

Task folder:
[PASTE PATH]

Evaluation and logs:
[PASTE OR ATTACH]

Review the attached Terminal-Bench quality report against the task files.

Classify every finding as:
1. Blocking correctness/security issue
2. Missing instruction detail
3. Test-quality weakness
4. Soft suggestion
5. False positive

Critical or Vulnerable findings must be fixed. For a Quality Check failure, add any tested-but-undisclosed requirement to instruction.md without revealing the solution. Do not change solid tests solely because a report says Warning / Test Quality Robust.

Implement only justified changes, then summarize each finding and its resolution.

Task folder:
[PASTE PATH]

Quality report:
[PASTE REPORT]

Revise this Terminal-Bench task from the reviewer feedback.

For each reviewer note:
1. Explain the underlying issue
2. Identify the exact file(s) involved
3. Make the smallest complete fix
4. Add or update verification when needed
5. Confirm the change does not make the task trivial or unsolvable

After all changes, run the relevant checks plus oracle and nop. Return a concise response mapping every reviewer note to the implemented fix and verification evidence.

Do not create a replacement submission. This revision will update the existing NEEDS_REVISION submission.

Task folder:
[PASTE PATH]

Reviewer feedback:
[PASTE FEEDBACK]