Plan mode is dead?
Forward
Recently Claude Code and others have released a series of guides recommending their users to shift from structured plan mode to setting verifiable goals and allowing the AI to find its way to the goal on its own.
Representatives of AmpCode have said on X, "Do people know they don't have to use plan mode?"
This is further reflected by many thought leaders in the tech industry promoting the method of allowing users to directly submit issues which are automatically built by AI with a PR awaiting a professional developer.
The destination for /plan and /goal methods is the same, the question is whether AI has been sufficiently well-trained to self plan, iterate, and persistently deliver a quality solution.
This article tests the goal method on 3 verifiable backlog items for a personal AI assistant (Sidekick), captured by myself during use of the assistant. The goal results are compared against results from a structured written plan.
Contents
- Summary
- Hypothesis and Test Setup
- Results
- Technical Notes
- Future Tests
1 - Summary
In order to test the hypothesis 3 tasks were gathered from my personal AI assistant backlog of increasing difficulty:
- Expand email read capability to include those that arrive in HTML format
- Allow the assistant to delete and rename google task lists safely - including adhering to project security policy conventions
- Add the ability for the assistant to open online property listing photo libraries, then navigate and pass photos to the multi modal model for interpretation as part of a property scanning cron job.
Claude Code was tested as part of a Opus5.5 / Sonnet 5.5 builder reviewer setup without and without a plan. The same setup was tested through Opencode with a locally hosted Qwen Flash Next / Deepseek Flash Cloud reviewer in place of Claude models to test the same theory with less capable models.
Results - Goal wins for Claude Code, Plan wins for OpenCode
| # | Cell | Median tests | Median cost | Median min | Tests by sitting (1–5) |
|---|---|---|---|---|---|
| 1 | OpenCode, plan | 54/54 | $0.73 | 401.1 | 54, 54, 54, 53, 54 |
| 2 | Claude Code, plan | 54/54 | $29.63 | 88.3 | 54, 54, 54, 54, 54 |
| 3 | OpenCode, goal | 53/54 | $0.38 | 332.3 | 54, 54, 48, 50, 53 |
| 4 | Claude Code, goal | 53/54 | $22.52 | 75.2 | 54, 53, 53, 53, 53 |
The key finding is that Claude Code delivered significantly more consistently and faster in the goal approach, whereas OpenCode with Qwen Flash Next delivered materially the same quality performance but only in plan mode.
While the Claude Code goal cells did pass 1 less test on average this was a minor naming convention issue. In contrast, OpenCode with Qwen Flash Next in the goal approach delivered multiple app-breaking defects.
For practitioners this may indicate staying on plan mode for Claude Code will lead to poorer results over time as their models are trained into goal-oriented execution - consider shifting towards more goal-based workflows now!
Next article we'll explore graph engineering
Once a month you'll join us and get deep into the weeds of the latest AI engineering setups.
Get the next article2 - Hypothesis and Test Setup
The experiment cell block
| Harness | Handed | Models used | Sittings |
|---|---|---|---|
| Claude Code + Adventure Party | Six-step plan | claude-opus-5-5, claude-sonnet-5-5 | 5 |
| Claude Code + Adventure Party | Goal and guardrails | claude-opus-5-5, claude-sonnet-5-5 | 5 |
| OpenCode + local party | Six-step plan | local/qwen-flash-next, local/qwen-guide, openrouter/~deepseek/deepseek-flash-latest | 5 |
| OpenCode + local party | Goal and guardrails | local/qwen-flash-next, local/qwen-guide, openrouter/~deepseek/deepseek-flash-latest | 5 |
“Plan” cells were handed the six-step plan, “goal” cells only its goal and guardrails.
Scoring the runs
The following groupings encapture the tests the cells were measured against:
- Email reading (11 tests): an HTML-only email comes back as clean text, with entities decoded, line breaks and links kept, and styles, scripts, tracking pixels and quoted replies trimmed. An email that also has a plain-text part still uses it.
- Task-list tools (10 tests): new tools delete a task, delete a list and rename a list. Deletes are marked destructive, a list with tasks in it won't delete without being forced, and the approval card shows the list's name and how many tasks will go, read live from Google.
- Photos to the model (10 tests): view_page opens a page or a photo gallery, with a cap on how many images it takes. The photos reach the model as real images inside the tool's answer, not as text; the cache marker never sits on an image; a reopened chat re-sends exactly the same request; and a bot wall leaves the tab open with a link.
- The property job (4 tests): a playbook that runs view_page, scores the house with the rubric, writes to the tracker sheet by name and caches its verdict by listing URL.
- Security of the new tools (12 tests): destructive tools always ask, are refused when nobody is watching and can never be set to always-allow; nothing new arrives pre-approved; only a person at the keyboard can grant always-allow; previews are live reads; and view_page counts as sending data out, asks before a new site and keeps its guard against private addresses.
- The backlog board (6 tests): each of the three items is archived as done, gone from the live board, with a claim and a close in the log, and the rest of the board is left alone.
- The run's own tests (1 test): the test suite the run wrote must pass.
- Security gate (pass/fail, outside the 54): Sidekick's original security tests, run from a clean copy so a run can't weaken them. All 20 runs passed.
Prompt approach
Both cells got the same goal, contract and constraints. The plan cell got a detailed execution plan.
| Part of the file | Goal cell | Plan cell |
|---|---|---|
| Opening line | “Here’s the goal and its guardrails; plan and execute however you see fit” | “Scope is fixed to these three items; anything else you find goes on the board for later” |
| The goal: the three backlog items and how each must be proven | âś… same | âś… same |
| Resume contract: keep progress in files and commits so a long run can pick up again | âś… same | âś… same |
| Pinned constraints: the project’s known traps (image caching, a 420-line file cap, tool safety classes, which Chrome to use, no trafilatura) | ✅ same | ✅ same |
| Six steps: what to build in which order, which files each step touches, how to check each one | ❌ none | ✅ about 125 lines |
| Who does what: a builder per step, a reviewer after every build, a senior reviewer optional at the hardest step | ❌ none | ✅ |
| Size | 742 words | 2,312 words |
3 - Results
Tests passed by cell chart
The main finding here is that OpenCode with a local smaller model did not perform well with the goal approach.
Cost by cell charts
Interestingly we see that Claude Code in plan mode actually had much larger variation than the goal approach - potentially indicating Claude models are becoming more tuned towards the goal approach.
Opencode is largely run through a local model and incremental costs are not-material.
Time by run charts
Claude Code runs were very consistently ~80minutes. As expected the local model runs were ~6 hours filling the "run during the workday" or "run overnight" time window.
Of interest the plan mode execution was very consistent for the less intelligent local model, however goal mode swung wildly.
Aggregate results
| Cell | Turns | Tool calls | Party agents spawned | Commits | Lines added |
|---|---|---|---|---|---|
| Claude Code, plan | 456 | 475 | 12 | 10 | 4,594 |
| Claude Code, goal | 342 | 363 | 6 | 3 | 4,423 |
| OpenCode, plan | 811 | 1042 | 12 | 17 | 3,867 |
| OpenCode, goal | 641 | 772 | 8 | 5 | 3,306 |
Results by sitting
| Cell | Sitting | Tests | Cost | Active min | Commits | Interruptions |
|---|---|---|---|---|---|---|
| Claude Code, plan | 1 | 54/54 | $33.17 | 101.0 | 11 | 1 pause, 99 min |
| Claude Code, plan | 2 | 54/54 | $26.94 | 85.2 | 9 | — |
| Claude Code, plan | 3 | 54/54 | $29.63 | 93.0 | 11 | — |
| Claude Code, plan | 4 | 54/54 | $23.31 | 71.5 | 6 | — |
| Claude Code, plan | 5 | 54/54 | $31.26 | 88.3 | 10 | 2 pauses, 520 min |
| Claude Code, goal | 1 | 54/54 | $27.31 | 77.7 | 4 | — |
| Claude Code, goal | 2 | 53/54 | $22.52 | 74.7 | 3 | — |
| Claude Code, goal | 3 | 53/54 | $22.67 | 77.6 | 3 | — |
| Claude Code, goal | 4 | 53/54 | $21.32 | 67.2 | 3 | 1 pause, 269 min |
| Claude Code, goal | 5 | 53/54 | $22.02 | 75.2 | 3 | 1 pause, 244 min |
| OpenCode, plan | 1 | 54/54 | $0.78 | 404.7 | 8 | 1 nudge |
| OpenCode, plan | 2 | 54/54 | $0.57 | 421.5 | 21 | 1 nudge |
| OpenCode, plan | 3 | 54/54 | $0.73 | 377.8 | 17 | — |
| OpenCode, plan | 4 | 53/54 | $0.52 | 401.1 | 21 | — |
| OpenCode, plan | 5 | 54/54 | $0.73 | 387.6 | 15 | — |
| OpenCode, goal | 1 | 54/54 | $0.37 | 354.1 | 15 | 1 nudge |
| OpenCode, goal | 2 | 54/54 | $0.65 | 332.3 | 4 | — |
| OpenCode, goal | 3 | 48/54 | $0.38 | 270.5 | 5 | — |
| OpenCode, goal | 4 | 50/54 | $0.40 | 289.9 | 11 | — |
| OpenCode, goal | 5 | 53/54 | $0.35 | 437.6 | 5 | — |
Note - interruptions occur through either the Claude Code subscription 5 hour limit being reached or occasionally the OpenCode local model being briefly paused to make way for another process. In both cases active minutes are adjusted to take into account active working time.
Where the marks went
The 4 Claude Code failures were all misnaming the view_page tool triggering a failed verification test - which is an expected failure mode for effectively asking AI to wing a build, but a minor fail none the less.
2 of the Opencode goal runs with failure were major defects that note only broke the build but also the chat function of the app.
The only Opencode plan mode failure was shipping 1 non-functioning test, so a minor failure but not breaking.
The final Opencode goal failure was a simple process failure in updating of the backlog list.
| Sitting | Score | What happened | Verdict |
|---|---|---|---|
| Claude Code, goal 2, 3, 4, 5 | 53/54 each | The property job’s tool list does not name view_page. Each of the four gave the job its own tool that is built on view_page’s code and takes the photos itself. | Defensible design. The board asks for view_page to be “wired into the property job”; a wrapper does that. The test looks for the name because that is what a script can check. |
| OpenCode, plan 4 | 53/54 | Its own test suite failed: one test it wrote about refusing blocked image names inside the allowed folder. | Small defect: it shipped a red test. |
| OpenCode, goal 3 | 48/54 | Reopening a saved chat that holds a screenshot crashes (a TypeError in its image result type), which costs four tests. Gallery mode returned 2 photos when asked for 6. The property job carries neither the scoring rubric nor the tracker sheet’s name. | Real defects. The crash would hit any saved conversation that used the tool. |
| OpenCode, goal 4 | 50/54 | Screenshot mode returns the page’s text and no image at all, so three tests find nothing to look at. The property job does not name the tracker sheet. | Real defects. The model never sees the page in that mode. |
| OpenCode, goal 5 | 53/54 | Archived BK-001 as done with its boxes unticked and nothing new in its notes. | Process defect: the board’s rule 6 asks for ticked boxes or a written reason. |
4 - Technical Notes
The following post-test probes were performed to further stress test the quality of the runs outside of the core spec.
These illustrate the varying additional requirements needed at the app level to ensure security and quality outside of the immediate specification request.
Security probes
H1 - Almost every run exposed itself to a backdoor redirect through the image view tool.
H5 - Editing the saved chat so a path pointed at another file on disk allowing sending of new unintended files to the database.
Functional probes
H2 - Only OC goal 3 implemented a situation where changes in the photo in storage location would break the app on re-open.
H3 - OC run's didn't implement additional validation to check if a google task list being deleted still contained open tasks.
5 - Future Tests
The following two novel topics are becoming more important to be across and will be covered in future blog articles:
Graph AI engineering is becoming very talked about. We'll answer when it works and when it doesn't using data.
Hacking and data exfiltration are increasing in frequency due to AI enablement. Soon we will cover basic approaches to improve protection of your app during the development process of AI coding.
Sign up for the next novel AI experiment
Once a month you'll join us and get deep into the weeds of the latest AI engineering setups.
Get the next article
Comments ()