Model Data Points

A data point is one small experiment with AI models: what was tested, on which models, why, and what happened. They are not systematic benchmarks; read each one with its caveats.

28 Oct 2026OCR

OCR comparison: Unlimited-OCR 2, Docling and MinerU on academic PDFs

Unlimited-OCR 2 · openDocling · openMinerU · open

Docling took 3.5 minutes for all 10 papers, Unlimited-OCR 2 on SGLang 31 minutes and MinerU 62 minutes, with no accuracy advantage for either slower tool on born-digital papers.

28 Oct 2026API behaviour

Muse CLI: one unattended source-review run

Muse Wonder Full · closed

The source-only review completed with exit code 0 and no approval requests.

27 Oct 2026Computer use

Computer use: Claude Haiku 5.5, GPT-6.1 Terra and GPT-6 Luna

Claude Haiku 5.5 · closedGPT-6.1 Terra · closedGPT-6 Luna · closed

On the web-page check Haiku scored 0.95 in 23.4 s, Luna 0.9 in 43.6 s and Terra 0.9 in 46.7 s; on the desktop-app check all three scored 1.0 in 35.1 s, 47.9 s and 40.0 s.

25 Oct 2026Local models

Eight small local models on transcript-cataloguing tasks

Gemma 4 31B · openGemma 4 12B · openGemma 4 E4B · openQwen 3.5 4B · openQwen 3.5 9B · openLFM 2.5 8B · openMinistral 3 8B · openGranite 4.2 8B · open

Gemma 4 31B and 12B passed 5 of 6 checks and LFM 2.5 8B passed 0 of 6; median generation speed ranged from 10.3 to 153.7 tokens per second.

25 Oct 2026Local models

A 17-hour local run: Gemma 4 31B over 508 documents

Gemma 4 31B · open

508 documents took about 17 h 19 min on a DGX Spark at a median of about 32 tokens per second, using 8,571,832 input tokens and 1,589,326 generated tokens.

24 Oct 2026Local models

Draft-token setting and speed on Gemma 4 31B

Gemma 4 31B · open

With the draft-token setting at 0, 3 and 5, median speed was 10.6, 32.6 and 41.3 tokens per second; each setting passed 4 of 6 checks.

16 Oct 2026Coding

Builder pilot: Opus 5.5 rebuilds five app tasks first built by Sonnet 5.5

Claude Opus 5.5 · closedClaude Sonnet 5.5 · closedGPT-6.1 Sol · closedMuse Wonder Full 1.0 · closedClaude Fable 5.1 · closed

At list prices Opus cost $9.02 against $3.78 for Sonnet (2.4 times); blind review scores were close (3.30 and 3.40 against 3.27 and 3.23). One build per cell.

3 Aug 2026API behaviour

Reasoning tokens used up an output cap on GPT-5.5 Pro

GPT-5.5 Pro · closed

With max_output_tokens at 6,000, all 6,000 tokens went to reasoning and no text came back; an uncapped run took 542 s and 22,828 output tokens.

8 Mar 2026Image generation

Local image generation on an M4 Max: FLUX.2 Klein 4B and Qwen-Image

FLUX.2 Klein 4B · openQwen-Image 2512 · open

At 1024 x 1024, FLUX Klein took about 18 s; Qwen-Image at 20 steps took about 8 minutes.

Models tested

Green is open weights, amber is closed. Each model lists its entries.

Claude Fable 5.1 closedBuilder pilot: Opus 5.5 rebuilds five app tasks…
Claude Haiku 5.5 closedComputer use: Claude Haiku 5.5, GPT-6.1 Terra a…
Claude Opus 5.5 closedBuilder pilot: Opus 5.5 rebuilds five app tasks…
Claude Sonnet 5.5 closedBuilder pilot: Opus 5.5 rebuilds five app tasks…
Docling openOCR comparison: Unlimited-OCR 2, Docling and Mi…
FLUX.2 Klein 4B openLocal image generation on an M4 Max: FLUX.2 Kle…
Gemma 4 12B openEight small local models on transcript-catalogu…
Gemma 4 31B openEight small local models on transcript-catalogu…
A 17-hour local run: Gemma 4 31B over 508 docum…
Draft-token setting and speed on Gemma 4 31B
Gemma 4 E4B openEight small local models on transcript-catalogu…
GPT-5.5 Pro closedReasoning tokens used up an output cap on GPT-5…
GPT-6 Luna closedComputer use: Claude Haiku 5.5, GPT-6.1 Terra a…
GPT-6.1 Sol closedBuilder pilot: Opus 5.5 rebuilds five app tasks…
GPT-6.1 Terra closedComputer use: Claude Haiku 5.5, GPT-6.1 Terra a…
Granite 4.2 8B openEight small local models on transcript-catalogu…
LFM 2.5 8B openEight small local models on transcript-catalogu…
MinerU openOCR comparison: Unlimited-OCR 2, Docling and Mi…
Ministral 3 8B openEight small local models on transcript-catalogu…
Muse Wonder Full closedMuse CLI: one unattended source-review run
Muse Wonder Full 1.0 closedBuilder pilot: Opus 5.5 rebuilds five app tasks…
Qwen 3.5 4B openEight small local models on transcript-catalogu…
Qwen 3.5 9B openEight small local models on transcript-catalogu…
Qwen-Image 2512 openLocal image generation on an M4 Max: FLUX.2 Kle…
Unlimited-OCR 2 openOCR comparison: Unlimited-OCR 2, Docling and Mi…