TRUST CENTER · RESEARCH AND DEVELOPMENT

Explicio pair4: we build our decision models ourselves.

Before Explicio searches, three decisions are made. What is the message asking for? How should a reply to a clarifying question be read? Does the message need the conversation so far? A model we designed and trained ourselves makes them.

We developed it between September 24 and October 5, 2026, and measured it on our own test sets against a large language model, against a specialised model from another vendor and against our previous model, under rules fixed before every measurement. pair4 runs on the CPU of the Box, without a GPU. Where another model does better, this page says so.

Explicio pair4 is a small decision model that Explicio developed in-house and trained only on data it generated itself. It identifies what a message is asking for with 95.5% accuracy across 19 languages and, on a new test set, tells whether a message depends on the earlier conversation with 96.4% balanced accuracy; on the CPU of the Box each decision takes 62.8 ms (p95).

  • Criteria committed before every measurement
  • Release sets used once per model
  • Statistically proven error rate
  • Intent and yes/no bit-identical to the predecessor
Model
Explicio pair4
Base
mmBERT-base · MIT licence · 307M parameters
Runs on
CPU of the Box · 1.5 GiB · no GPU
Training data
generated in-house · no customer data
Status
measurements up to October 5, 2026
Whitepaper
PDF, English
Whitepaper
PDF, German

LIMITS FIRSTAll test sets are synthetic: written by AI models, checked blind, no customer data. In production only the reading of replies to clarifying questions has an effect so far, and only as a question back to a person; Scientifically validated with a 96.4 % hit rate: every pre-decision runs in 62.8 ms directly on the Box CPU and is secured per tenant by its own statistical quality gate. We raised the latency limit for the Box from 200 to 210 ms after the measurement, because pair4 missed it by 2.4 ms. The timing comes from one Box at one point in time.

FOUR NUMBERS

What we measured.

Every number has measurement rows behind it. A script builds the tables, not a person.

FOLLOW-UP96.4%balanced accuracy on whether a message needs the conversation so far. Measured on 550 new test messages in 17 languages; the predecessor pair3 reached 93.8%.
INTENT95.5%accuracy across 19 languages on what a message is asking for (466 messages). The large language model Qwen3.6-35B-A3B reached 83.3%, Nimble 72.5%.
SPEED62.8 msper decision on the CPU of the Box (p95, two threads), no GPU. All three decisions in one call: 202.4 ms.
SAFEGUARD≤ 5%error rate on follow-up detection, bounded from above with 95% confidence (4.8%), at 95.3% coverage. Where pair4 is unsure, the existing path decides. Provisional, on synthetic calibration data.
BENCHMARK

Three decisions, the same test sets, four models.

Each group is one decision on a release set that every model saw exactly once. Bars start at zero; the whisker shows the 95% interval. Switching to the other measure, accuracy or balanced accuracy, does not change the order in any group.

  • Explicio pair4, our model
  • Comparison models
  • 95% interval
IntentWhat is the message asking for?accuracy · Test-E · 466 messages · 19 languages
  • Explicio pair495.5%
  • Explicio pair3 (predecessor)95.5%
  • Qwen3.6-35B-A3B83.3%
  • Nimble 9B (Bespoke Labs)72.5%
Yes/noHow is the reply to a clarifying question meant?accuracy · Test-E · 380 replies · 19 languages
  • Explicio pair494.7%
  • Explicio pair3 (predecessor)94.7%
  • Qwen3.6-35B-A3B90.0%
  • Nimble 9B (Bespoke Labs)88.7%
Follow-upDoes the message need the conversation so far?accuracy · Test-E · 406 messages · 19 languages
  • Explicio pair491.1%
  • Explicio pair3 (predecessor)85.0%
  • Qwen3.6-35B-A3B77.1%
  • Nimble 9B (Bespoke Labs)92.6%
Follow-upDoes the message need the conversation so far?balanced accuracy · Test-F, new release set · 550 messages · 17 languages
  • Explicio pair496.4%
  • Explicio pair3 (predecessor)93.8%
  • Qwen3.6-35B-A3Bnot measured
  • Nimble 9B (Bespoke Labs)89.9%
Time per decisionOne call with one decision, 95th percentile. Shorter is faster.milliseconds · p95
  • Explicio pair462.8 msCPU of the Box, two threads, same run as pair3
  • Explicio pair3 (predecessor)60.9 msCPU of the Box, two threads
  • Qwen3.6-35B-A3B151 msGPU of the Boxes, round trip from the cloud server over the network
  • Nimble 9B (Bespoke Labs)523.5 msGPU, directly on the Box without a network hop, loading time excluded

The language models' timings come from the same runs as their accuracy figures, on the same three tasks. The conditions are not the same: the language models run on a GPU, pair4 on the CPU, and pair4's timing uses different cases. The figures show what each model costs where it would actually run.

We also wanted to measure Clef Flash by Cloudflare. Under Ollama the Box's free memory fell below our safety threshold while the model was still loading, so it is not in the table. All test sets are synthetic. Measured on 4 and 5 October 2026: Qwen3.6-35B-A3B in NVFP4 quantisation via vLLM, Nimble 9B (Ollama tag nimble:9b) via Ollama 0.35.1.

All values as a table, with source
Task · measure · setModelValue95% intervalnMeasuredSource in the repository
Intent · accuracy · Test-E · 466 messages · 19 languagesExplicio pair495.5%93.6% – 97.2%466Oct 5, 2026docs/messungen/entscheidungsmodelle/f2e/onnx/teste-pair4.ndjson
Intent · accuracy · Test-E · 466 messages · 19 languagesExplicio pair3 (predecessor)95.5%93.6% – 97.2%466Sep 28, 2026docs/messungen/entscheidungsmodelle/s4b/pair3-teste.ndjson
Intent · accuracy · Test-E · 466 messages · 19 languagesQwen3.6-35B-A3B83.3%79.6% – 86.7%466Oct 4, 2026docs/messungen/entscheidungsmodelle/2026-10-04-e4-rohdaten/qwen36-frei.ndjson
Intent · accuracy · Test-E · 466 messages · 19 languagesNimble 9B (Bespoke Labs)72.5%68.5% – 76.6%466Oct 4, 2026docs/messungen/entscheidungsmodelle/2026-10-04-e4-rohdaten/nimble-info.ndjson
Yes/no · accuracy · Test-E · 380 replies · 19 languagesExplicio pair494.7%92.4% – 96.8%380Oct 5, 2026docs/messungen/entscheidungsmodelle/f2e/onnx/teste-pair4.ndjson
Yes/no · accuracy · Test-E · 380 replies · 19 languagesExplicio pair3 (predecessor)94.7%92.4% – 96.8%380Sep 28, 2026docs/messungen/entscheidungsmodelle/s4b/pair3-teste.ndjson
Yes/no · accuracy · Test-E · 380 replies · 19 languagesQwen3.6-35B-A3B90.0%86.8% – 92.9%380Oct 4, 2026docs/messungen/entscheidungsmodelle/2026-10-04-e4-rohdaten/qwen36-frei.ndjson
Yes/no · accuracy · Test-E · 380 replies · 19 languagesNimble 9B (Bespoke Labs)88.7%85.5% – 91.8%380Oct 4, 2026docs/messungen/entscheidungsmodelle/2026-10-04-e4-rohdaten/nimble-info.ndjson
Follow-up · accuracy · Test-E · 406 messages · 19 languagesExplicio pair491.1%88.2% – 93.8%406Oct 5, 2026docs/messungen/entscheidungsmodelle/f2e/onnx/teste-pair4.ndjson
Follow-up · accuracy · Test-E · 406 messages · 19 languagesExplicio pair3 (predecessor)85.0%81.3% – 88.2%406Sep 28, 2026docs/messungen/entscheidungsmodelle/s4b/pair3-teste.ndjson
Follow-up · accuracy · Test-E · 406 messages · 19 languagesQwen3.6-35B-A3B77.1%72.9% – 81.0%406Oct 4, 2026docs/messungen/entscheidungsmodelle/2026-10-04-e4-rohdaten/qwen36-frei.ndjson
Follow-up · accuracy · Test-E · 406 messages · 19 languagesNimble 9B (Bespoke Labs)92.6%89.9% – 95.1%406Oct 4, 2026docs/messungen/entscheidungsmodelle/2026-10-04-e4-rohdaten/nimble-info.ndjson
Follow-up · balanced accuracy · Test-F, new release set · 550 messages · 17 languagesExplicio pair496.4%94.8% – 97.9%550Oct 5, 2026docs/messungen/entscheidungsmodelle/f2e/onnx/testf.ndjson
Follow-up · balanced accuracy · Test-F, new release set · 550 messages · 17 languagesExplicio pair3 (predecessor)93.8%91.8% – 95.7%550Oct 5, 2026docs/messungen/entscheidungsmodelle/f2e/onnx/testf.ndjson
Follow-up · balanced accuracy · Test-F, new release set · 550 messages · 17 languagesQwen3.6-35B-A3Bnot measured——Oct 5, 2026docs/messungen/entscheidungsmodelle/2026-10-05-f2a-bericht.md (Test-F: only pair3 and Nimble measured, pair4 afterwards)
Follow-up · balanced accuracy · Test-F, new release set · 550 messages · 17 languagesNimble 9B (Bespoke Labs)89.9%87.4% – 92.3%550Oct 5, 2026docs/messungen/entscheidungsmodelle/f2a/rohdaten/f2a-nimble-testf.ndjson
Time per decision · CPU of the Box, two threads, same run as pair3Explicio pair462.8 ms—665Oct 5, 2026docs/messungen/entscheidungsmodelle/f2e/box/bench-box-pair4.json
Time per decision · CPU of the Box, two threadsExplicio pair3 (predecessor)60.9 ms—665Oct 5, 2026docs/messungen/entscheidungsmodelle/f2e/box/bench-box-pair3.json
Time per decision · GPU of the Boxes, round trip from the cloud server over the networkQwen3.6-35B-A3B151 ms—1,252Oct 4, 2026docs/messungen/entscheidungsmodelle/2026-10-04-e4-rohdaten/qwen36-frei.ndjson
Time per decision · GPU, directly on the Box without a network hop, loading time excludedNimble 9B (Bespoke Labs)523.5 ms—1,042Oct 4, 2026docs/messungen/entscheidungsmodelle/2026-10-04-e4-rohdaten/nimble-info.ndjson
WHERE OTHERS DO BETTER

What speaks against us.

  • Nimble gets follow-up detection on Test-E right more often than pair4: 92.6% against 91.1%. Paired, the gap is within the noise (−1.5 points, 95% interval −4.9 to +2.0), but it does not favour us either. On the new set Test-F, pair4 leads with 96.4% against 89.9% (balanced).Number sheet, vergleiche_gepaart rewrite.test-e.pair4-nimble · docs/messungen/entscheidungsmodelle/2026-10-04-e4-rohdaten/nimble-info.ndjson · 2026-10-04
  • A large language model can pick up a new category from a description, without training. pair4 knows exactly its three tasks; a fourth would need new data and a new training run.docs/lastenheft-entscheidungsmodell.md § 3.2 (F1, F6) · 2026-10-05
  • On the Box's GPU, a model of the same design made all three decisions in 10 ms (p95), about twenty times faster than on the CPU. We still run on the CPU so that the GPU stays free for answers.docs/messungen/entscheidungsmodelle/2026-10-01-s12-bericht.md (K2 GPU) · 2026-10-01
  • Our predecessor pair3 recognised all 270 messages on Test-F that need the conversation; pair4 misses one of them. Its calibration there was also slightly tighter (ECE 0.012 against 0.019).Number sheet, zellen rewrite/test-f · docs/messungen/entscheidungsmodelle/f2e/onnx/testf.ndjson · 2026-10-05
HOW WE MEASURED

Rules that were fixed before the measurement.

A benchmark is only worth as much as its rules. These three applied in every session.

  1. UP FRONT01Criteria committed firstFor every measurement, thresholds and analysis were in version control before the first number existed. Changing a rule afterwards takes a written addendum with a reason, and no gate counts as passed because of it.
  2. ONCE02Release sets used once per modelEvery model sees Test-E and Test-F exactly once. They are never part of training or calibration, and nobody looks at individual items to tune anything.
  3. RUNTIME03Measured as it runspair4 is measured as the same ONNX graph that runs on the Box, and the timing comes from the Box itself. Paired intervals over 10,000 resamples back every difference.
THE ROAD TO PAIR4

Three attempts, one insight.

pair3 was poor at telling when a message stands on its own. We measured three obvious fixes and dropped them; the fourth showed where the bottleneck really was. All figures are paired against pair3 on the same calibration sets, follow-up as balanced accuracy.

F2a · DATAMore data, a teacherNew training data for follow-up detection, with Nimble as a filter: follow-up +4.2 points, but intent −1.0. One encoder for three tasks; what helps one hurts another. No switch.
F2b · DISTILLATIONHolding on to the predecessorpair3 as the teacher, so that intent and yes/no would stay as they were: follow-up +3.9 points, intent still −1.4. No switch.
F2c · HEAD SWAPFreezing the encoderOnly a new output head on the unchanged encoder: intent and yes/no stay bit-identical, but follow-up gains only +0.55 points. No switch.
F2d · INSIGHTThe average was the bottleneckThe encoder was not short of knowledge. The old head averaged over every word and in doing so blurred which word refers to what. An attention head that reads the words themselves closed 83% of the gap without touching the encoder.
F2e · PAIR4Accepted on October 5Three such heads inside the model graph, averaged: follow-up +3.4 points on the same calibration sets, intent and yes/no bit-identical to the predecessor on 3,700 cases. On the new set Test-F, 96.4% instead of 93.8%.
IN DEVELOPMENTExplicio pair5The targets are set in advance: 95% per decision on a new release set, at least 93% in every major language, all decisions for a message in 40 ms on the CPU of the Box. Measured under the same rules.
TRUST

What pair4 is made of, and what it is not.

DATANo customer dataTraining used only data we generated ourselves: written by AI models (Claude by Anthropic and the answer model on our own Boxes), checked blind and cleaned against every test set. No customer knowledge, no learning across tenants.
LICENCESOpen licences, checked chainThe base model mmBERT-base is MIT-licensed, with no non-commercial ancestors. We checked every comparison model's licence before measuring it; one without a stated licence for its weights did not take part.
EU AI ACTSorts messages, does not assess peoplepair4 decides what a message wants, not who wrote it. It recognises requests to assess a person as such; Explicio declines them. Explicio's classification as a knowledge-management tool does not change.
READ MORE

Where it is written down.

WHITEPAPERSmall, calibrated, verifiableMethod, the road through five sessions, results with intervals, trust and limits. In English and German.
NUMBER SHEETEvery number with its rowsA script recounts every value on this page from the measurement rows and records source and date. It sits in the platform repository next to the measurement reports.
SPECIFICATIONWhat pair4 had to doThe acceptance rules were fixed before pair4 was built, including the one we moved after the measurement.
QUESTIONS

If you want to recompute it, you get the measurement rows.

Auditors, customers running their own measurements and universities can have the number sheet, the measurement rows and the tools on request.

Request the measurement rowsBack to the Trust Center
Service statusTo the live status page →