Measured evidence · two distinct tracks

Does the AI actually finish the job?

Part 1 measures whether a small local model produces a file that passes objective checks. Part 2 separately measures orchestration overhead in a synthetic multi-tool path. Neither is a general AI leaderboard or a human productivity study.

AgentBrowse suite · 28/28 Local-AI reliability Workflow orchestration
See the verified result → Set up Easy Mode free
New to agents? Here’s the 30-second version
What is an “agent”?
An AI that can call tools (resize an image, merge PDFs, make a QR code) instead of only chatting. Think of it as a junior assistant with buttons it can press.
What is “Easy Mode”?
A free helper that runs on your computer. It gives small local AIs a short checklist so they finish jobs more reliably — privacy stays local unless you opt into hosted tools.
What do the big numbers mean?
0/20 = without guidance the AI never produced a correct compressed file in our test. 20/20 = with Easy Mode it succeeded every time. −78.6% = it also used far fewer “thinking” tokens (cheaper / faster for the model).
What is Part 2 about?
A synthetic engineering test of orchestration overhead: page-load latency plus equivalent local processing versus authenticated AMT API calls. It is not a stopwatch study of people clicking through websites.

I just want browser tools

You can ignore model names. Scroll to Part 2 or open the Toolbox and try a free tool.

I run a local AI (LM Studio, etc.)

Part 1 is for you. Then set up Easy Mode so small models finish tasks.

0 / 20 without Easy ModeAI never finished the file correctly
20 / 20 with Easy ModeAI finished every trial correctly
−78.6% less model “thinking” workSame job, fewer tokens used

Read the tracks separately. Track 1 is a model-execution experiment. Track 2 is a synthetic software-path benchmark. The percentages in Track 2 describe elapsed machine time only; they do not measure employee time, usability, or business ROI.

Part 1 · Does the AI finish the job?Also called “verified execution” / Easy Mode benchmark · for local models

In plain English: we gave the same small local AI the same “compress this image” task 20 times. Without guidance it never produced a correct file. With Easy Mode it succeeded every time. Success means we checked the actual file — not just that the AI said “done.”

Published · test suite v1

Image compress test (guided vs unguided)

Make a fixed test image smaller than 80,000 bytes without changing its size (320×320 pixels). The AI only “wins” if the output file passes those checks.

Full test details →

Coming later

More tasks under the same rules

Next up may include exact resize, PDF merge, QR packs — still scoring real output files, not chat confidence.

Test details · image compressTechnical ID: IMG-COMPRESS-01 · same setup both sides

What we fixedWhat that means
The jobCompress a fixed test image below 80,000 bytes without changing its pixel size (must stay 320×320).
The test fileSame image file every time for fair comparison.
The AI modelQwen3.5 9B Q4_K_M, served locally through an OpenAI-compatible LM Studio endpoint.
What stayed the sameSame model, settings, file, goal, and 20 tries each side — only the “guidance layer” changed.
Without Easy Mode (“raw”)The AI sees a big menu of tools and has to figure everything out alone.
With Easy Mode (“guided”)A short checklist: remember the goal, try again safely, and check the file before claiming success.
How we score a winThe finished file must exist, stay 320×320, and be under 80 KB. The AI saying “done” is not enough.
How unguided usually failedIt inspected or described the image instead of compressing — then claimed success while the file was still too big.

Results side by side

How the AI workedJobs finished correctlyThinking cost (tokens)What helped it
Without Easy Mode
Big tool menu, no checklist
0 / 20 21,738 average Tools only — no extra guidance
With Easy Mode
Short checklist + file checks
20 / 20 4,653 average (−78.6%) Goal memory, safe retries, output verification
Keep expectations honest. This is one model, one image job, one controlled test. It shows that guidance helped a lot here. Other models, hardware, or tasks can differ — results are not a universal guarantee.
Reproducibility status: the model family, quantization, task, dimensions, byte threshold, aggregate trial count, and acceptance checks are published. The original v1 record does not include publishable per-trial logs, exact inference settings, fixture hash, or bridge commit. Those omissions limit independent reproduction; a future suite version should archive them before results are published.
Animated story — same AI, same job (replay anytime)

Without Easy Mode

You: Compress this image under 80KB, keep 320×320

With Easy Mode

You: Compress this image under 80KB, keep 320×320

How we score Part 1 (simple rules)

Try the same idea at home

1. Install free Easy Mode

Download the local bridge for Windows, macOS, or Linux. Point it at LM Studio, Ollama, or another local AI server on your machine (or a trusted home network address).

Installation guide →

2. Load a tool-capable small model

Models that support “tool calling” work best (for example many Qwen or Llama instruct builds). Closer match to our test = closer numbers.

3. Ask for a hard, clear job

Example: “Compress this image under 80,000 bytes without changing dimensions.” Require a real file, not just a yes.

4. Judge the file yourself

Check width, height, and file size. If the AI says “done” but the file fails, count it as a fail — same as we do.

# Optional: power users — start the free local bridge
export AMT_LLM_UPSTREAM_URL=http://127.0.0.1:1234/v1   # or your LM Studio address
export AMT_EASY_MODE=true
npm start
# Then in your chat UI: “Compress /path/to/image.png under 80000 bytes without changing dimensions.”

Part 2 · Synthetic orchestration overheadMachine-path benchmark · not human workflow ROI

Question tested: how much elapsed machine time does a unified API path remove compared with a constructed multi-tool path? The baseline combines measured page-load latency from real tool sites with equivalent local CLI processing. It does not upload files, fill forms, or complete work through those third-party interfaces.

88–98%less elapsed machine timesynthetic baseline vs AMT API
4 originsrepresented in each baselinevs one AMT API origin
N = 5successful runs per pathmedian reported
Picture the speed gap — social pack example
Synthetic
6.7s
AMT API
0.4s

Bars match our measured medians for the social pack (6.7s vs 0.4s). Longer bar = more time.

Interpretation limit. This isolates one kind of engineering overhead; it does not establish how long a person would take or which browser experience is easier. A future human study would need the same operator, real uploads/forms/downloads, failed attempts, and complete artifact handling on both sides.

How we timed Part 2

Timed · social pack

Social content pack

What you get: one photo turned into Instagram, X/Twitter, LinkedIn, and Stories sizes, plus a QR code, short tracking link, and a shareable notes paste.

PathMedian timeStepsOrigins
Synthetic multi-tool6.7s194 origins
AMT REST API0.4s121 origin

94.7% less elapsed machine time in this synthetic test. This is an API-path result, not a claim about minutes of human labor saved.

We checked: exact pixel sizes, QR created, short link + paste created. Does not include writing captions or posting to social apps.

Watch the pack build — one photo → ready assets
$ amt pack social --source hero.jpg
IG square1080×1080
𝕏X / Twitter1200×675
LinkedIn1200×627
Stories1080×1920
▣QR PNGbranded
Short linkUTM ready
Team pasteone share
✓Done0.4s median

Playbook →  ·  Resize →  ·  QR →

Timed · product photos

Product image pack

What you get: one product photo → small grid size (800×800), large hero (1600×1600), modern WebP version, listing file under 250 KB, plus a short size note.

PathMedian timeStepsOrigins
Synthetic multi-tool3.5s104 origins
AMT REST API0.4s91 origin

88.8% less elapsed machine time in this synthetic test. No human-time extrapolation is made.

We checked dimensions, WebP output, and the size budget. No photo retouching or AI art generation in this test.

Product pack ready
listing.jpg1600×1600 · WebP + grid 800² <250 KB
analyze → resize → convert → compress→0.4s median

Playbook →  ·  Resize →  ·  Compress →

Timed · documents

Document intake pack

What you get: three PDFs combined into one, text pulled out, a rough “how big is this for an AI?” token count, and a short intake note.

PathMedian timeStepsOrigins
Synthetic multi-tool3.2s114 origins
AMT REST API0.1s61 origin

97.8% less elapsed machine time in this synthetic test. No human-time extrapolation is made.

We checked the merge and that marker text from each PDF showed up. This test uses PDFs that already contain text (not scanned photos of paper).

Watch the pipeline — three PDFs → one intake pack
A.pdf + B.pdf + C.pdf → merge → extract → tokens → paste ✓

Playbook →  ·  Merge →  ·  PDF to text →

Power users / reproduce Part 2: node scripts/run-workflow-roi-bench.mjs with your API key · full protocol in docs/workflow-roi-benchmark-specs.md.

What should I do next?

Pick the path that matches you — no need to understand every acronym.

I want free browser tools

Open the Toolbox and try compress, resize, PDF, or QR — no agent required.

Open Toolbox →

I run a local AI on my PC

Install free Easy Mode so small models finish jobs more often.

Easy Mode setup →

I want my agent on the web

Five-minute path: free API call → account → key → first authenticated request.

First win →

FAQ

Is this ranking ChatGPT vs Claude vs Gemini?

No. We are not ranking chat AIs. Part 1 tests whether a small local model finishes a file job with vs without Easy Mode. Part 2 times multi-site busywork vs one toolkit.

What does “0/20” and “20/20” mean?

We ran the same compress job 20 times. Without Easy Mode: zero correct files. With Easy Mode: twenty correct files. We checked the actual images, not the chat messages.

What is Easy Mode in one sentence?

A free local helper that gives small on-device AIs a short checklist so they finish tools more reliably — without sending your files to a cloud model unless you choose to.

Does Part 2 measure people using websites?

No. It measures a synthetic baseline of page-load latency plus local CLI work against an authenticated AMT API path. It is useful for understanding orchestration overhead, not for claiming employee time saved.

Did a big cloud model secretly do the Easy Mode work?

No. The same local Qwen model ran both sides. Easy Mode adds structure around it; it does not swap in a larger online model for the published score.

Can I quote these numbers?

Yes — link this page, mention the date of the test, and include the caveats (one model / one task for Part 1; automated multi-site lower bound for Part 2). Prefer the full story over only “20/20.”

Related

Easy Mode setup

Install the free bridge and connect LM Studio, Open WebUI, or Hermes.

Open guide →

Design article

Why small models fail at tools and what the seven controls change.

Read article →

First win

Free curl → browser tool → account → API key in about five minutes.

Start first win →

One toolkit vs glue

Why builders consolidate image, PDF, and utility vendors.

Open compare →

Ready to try something real?

New users: open the free Toolbox. Local AI users: install Easy Mode. Builders: grab a free API key in five minutes.