Verification record · Build with Gemini XPRIZE
Veritas by Optume Translations
Official WMT25 data corroborates 87 documents and 332 segments, with no legal-domain text. The claimed performance percentage remains unverified.
Claim
Veritas by Optume Translations reports 92.68% “legal-invariant preservation” on the WMT25 English→Egyptian Arabic benchmark, against 82.93% for the nearest of ten frontier systems. It specifies 87 documents and 332 segments. The submission also reports CAD 10,045.44 invoiced and paid, with CAD 1,537.03 in expenses.
What we tested
On 16 September 2026, sound.fan cloned the official wmt-conference/wmt25-general-mt repository and counted en-ar_EG documents and segments in the score files for Claude-4, Gemini-2.5-Pro and GPT-4.1. We tallied the domain labels, compared the named metric with WMT25’s published evaluation methods, and fetched Veritas’s public pricing endpoint.
Finding
Corroborated: the official files contain exactly 87 documents and 332 segments for each of the three systems checked. Their domain labels are literary (85 segments), news (94), social (91) and speech (62). There are no legal-domain segments.
The dataset citation is exact. Its domain does not establish performance on contracts or court documents, and the headline performance percentage was not independently reproduced.
What the evidence establishes
The test-set identity and counts moved from Claimed to Corroborated against an independent source. “Invariant preservation” is the participant’s own measure, not an official WMT25 metric. The task’s published evaluation uses human ESA and AutoRank, with chrF++ for two low-resource targets. Figures, dates and cross-references can survive or change in any domain, so the proposed measure may still be meaningful; the evidence has a narrower scope than legal translation performance.
Observed, later live surface: on 16 September the pricing endpoint listed $0.0005 per word, the Base network (eip155:8453) and a treasury wallet, consistent with the submitted API price. Observing a payment configuration does not establish payments, customers or production use.
What remains unverified
Not verified: the 92.68% result, the ten-system comparison and significance claims. The metric definition, outputs and system list were not public. The four-stage Gemini pipeline, 1,110-line deterministic verifier and reported 14 August model promotion also lacked public run records.
The paid invoice and expenses remain participant-reported. The claim that the API was live and pre-revenue was not established through a ledger lookup; the OTTIAQ permit claim was not checked against its member directory. Those checks were not completed and are not findings of this record.
What evidence would change that
The smallest useful benchmark package is the invariant extractor, per-segment outputs and named comparison systems, including whether they are 2025 WMT25 submissions or 2026 models. A run on a public legal corpus would address legal-domain scope. A redacted invoice with a matching deposit would address the paid-customer claim; a treasury-wallet ledger check and an OTTIAQ directory entry would address their separate claims.