Tomato

Tomato is a DOM-first agent rather than a fixed recipe runner. A request starts in the assistant chat, becomes a short plan, and runs as small browser actions that must prove they worked.

The aim is practical autonomy without invisible control. Tomato reads the page structure first, keeps the user informed through task cards and confirmations, and asks for help when the browser no longer provides enough evidence.

• Plan, act, verify. Tomato makes a short plan, performs one action at a time, predicts the expected page change, and checks the result before continuing.
• Work across tabs. It can follow a task between tabs while showing which tab it controls and where attention moved.
• Ask instead of guessing. Open-ended questions return to the chat when a task needs missing input or a user decision.
• Confirm sensitive actions. A confirmation appears over any app, including fullscreen, and names the action, site, control, and text involved.
• Use site skills. Maps, Docs, Keep, Gmail, X, and LinkedIn behaviors are stored as data rather than hard-coded recipes.
• Replay successful work carefully. Saved runs can repeat useful paths, but every replayed step still gets checked against the live page.
• Respect private browsing. Private windows use their own cookie context and forget it when the last private window closes.

Tomato is a DOM-first Mac agent. An observation builder turns the current DOM snapshot into the model's working context. Qwen3.5 runs locally, using a 2B model on Macs below 16 GB and a 9B model from 16 GB upward. The session holds 32K tokens with an 8-bit cache.

The browser connection borrows the useful shape of Chrome DevTools automation without exposing a debugging port that another local program could use to control a signed-in browser. Separate model contexts keep page understanding and action checking apart: the action checker sees the request and proposed step, never untrusted page content.

Each request follows the same bounded loop: observe the page, plan a small next move, validate that move against the request, act with real input, and compare the result with the predicted change. Loop detection catches repeated failures, while a hard ceiling limits every task to 60 steps or 30 minutes. Users can lower those limits but cannot raise them.

Formatted paste, tab changes, and page reads all use this loop. The task card remains the visible record of what Tomato is doing and why it stopped, continued, or asked a question.

DOM evidence remains the first choice because it is fast, precise, and cheap. When the structure is incomplete, Tomato retries the DOM path before escalating.

On Apple silicon, Fara 1.5 can inspect a screenshot and propose a visual target. Tomato captures the tab at its CSS size, maps Fara's 1000 by 1000 coordinates back through DOM.getNodeForLocation, and returns control to the normal verified loop. On Macs below 32 GB, the main model unloads while Fara works, which adds about 25 seconds.

If local vision cannot finish the task and the user has supplied a Claude or ChatGPT key, Tomato can send a redacted screenshot to that model. The step log makes the handoff explicit. If no model path has enough evidence, Tomato asks the user rather than fabricating a target.

Tomato keeps a locked safety core that settings cannot disable:

• prompt-injection screening
• phishing-list blocking
• hard execution boundaries
• secret isolation from models
• the Stop control
• independent action checking
• the step and time ceiling

Horizon's 268 MB prompt-injection-guard-small ONNX model screens page content at a 0.95 threshold. Deterministic rules run beside it on supported Macs and run alone on Intel hardware and macOS 13. A separate llama.cpp sequence compares each proposed step with the user's request. It holds actions on unnamed sites and sensitive actions the user did not request.

Ten user switches tune behavior outside that locked core. Sensitive-action confirmation starts enabled. Other controls cover per-site permission, a separate agent profile, watch mode, and confirmation before cloud use.

• Swift and XPC keep the Mac interface responsive and isolate model-process failures. XPC error handlers run away from the main actor, so a dropped model connection does not close the app.
• Qwen3.5 through llama.cpp provides local planning and a separate action-check context without sending ordinary browser work to a hosted service.
• ONNX injection detection gives page content a dedicated classifier instead of asking the acting model to judge its own input.
• DOM snapshots and DevTools primitives provide structured page state and real browser input without opening a general-purpose debugging port.
• Fara 1.5 handles canvas and other visual surfaces only when structured evidence is insufficient.
• Playwright site specs exercise complete local flows on controlled site copies.

Evidence captured on 7 October 2026: the local suite recorded 2,171 of 2,172 Swift tests passing, with one pixel test isolated, plus 373 page tests and 29 local-site Playwright specs. Injection checks resisted all 7 tested pages. With every guard enabled, ordinary-task runs completed 17 of 20 tasks.

Agent benchmarks completed 14, 15, and 16 of 20 local tasks across three runs. Median step times measured 4.9 and 5.5 seconds against 8-second and 25-second targets. A confirmation-heavy set completed 7 of 8 tasks, while local site-skill flows passed on controlled copies.

Reliability fixes also cover the surrounding app: passkey sign in no longer crashes Tomato, private cookies are discarded with their private context, failed builds stop immediately, and model disconnects no longer close the interface.

The architecture favors evidence over confidence. DOM-first operation gives the agent semantic targets. One action per cycle makes failures traceable. Expected-change checks stop a plausible click from becoming assumed success. The separate action checker cannot be persuaded by page text because it never receives the page.

Fallbacks widen perception without weakening the contract. Visual and cloud models may suggest a target, but execution returns to the same bounded loop. Locked protections cover risks users should not have to configure, while explicit switches keep ordinary preferences under user control.