‘MIZAN’

I first heard that sentence from a YouTube video of a talk Tristan Harris and Aza Raskin gave in 2023. It was projected behind them on a screen—attributed to Harari. It compressed an entire civilizational shift into a single image, and I turned it over for months.

But it was a trap.

If AI is the bomb, the conclusion seems obvious: build your own. So the race begins. But a bomb costs what a bomb costs. Sam Altman has estimated GPT-4’s training run at more than $100 million, with successive generations demanding far more. Even the wealthiest regional initiatives enter a competition whose rules, infrastructure, and incentives were designed elsewhere—in a language that is not ours, against benchmarks that were not built for us. The metaphor seduces us into competing on the one axis where we are least likely to win.

So put the bomb down. Ask what the metaphor concealed.

Imagine a merchant in a Cairo souk holding a mizan: a two-pan scale, suspended in equilibrium, patient and exact. For centuries, that instrument was the decisive one. Not the sword. Not the siege engine. The scale. Because whoever controlled the measure controlled the meaning of fair. The decisive question was never who possessed the weapon. It was who possessed the scale.

From the Edge of the Motion

I am writing this from Beirut, which is to say from the edge of the motion. The motion is real—MBZUAI and Falcon in Abu Dhabi, HUMAIN and ALLaM in Riyadh, Fanar in Doha, the deepest talent pool and the least compute in Cairo. My own university, AUB, trains computer scientists and then watches many of them leave, for the Gulf labs or further west. From here, the question does not feel abstract.

The real test is simple. Ask a frontier model an ordinary question in Modern Standard Arabic and the answer arrives polished and confident. Shift into Lebanese or Egyptian colloquial—the hybrid speech of a WhatsApp thread between Beirut and Amman—and the fluency thins. The grammar holds, but something disappears: the country. The answer lies not in the model but in what we ask it to optimize for.

The Layer That Lasts

Think of the AI stack as a hierarchy of assets with different half-lives. Model weights depreciate rapidly. Falcon 180B was state-of-the-art when it launched in 2023; within months, stronger systems had arrived. Fine-tuning techniques survive somewhat longer. Curated datasets last years. Benchmarks last decades.

The SAT dates to 1926. MMLU, introduced in 2020, remains one of the most cited measures of language-model capability2. Long after specific models become obsolete, the tests remain—shaping what researchers optimize, what investors reward, and what institutions purchase. Yet most Arab AI investment sits at the most rapidly depreciating layer of the stack: the model itself.

The clearest evidence came from the people closest to the frontier. In May 2025, TII—the institution behind Falcon—announced that adapting multilingual models outperformed building Arabic models from scratch3. This was not a failure. It was an honest recognition of comparative advantage—a signal that even the region’s best-resourced players are navigating away from pure model competition. Velocity is not direction. The region is sprinting. The question is toward what.

Who Defines the Target

Frontier laboratories are score-chasers by design: OpenAI, Anthropic, Google, and their peers optimize against the metrics that matter to the field. Open any model card and the same names recur: MMLU, GPQA, HumanEval, MATH, SWE-bench. Whoever builds the benchmark defines the target. And whoever defines the target shapes the future behavior of every serious competitor.

The irony is that Arabic benchmarks already exist. ArabicMMLU contains more than 14,000 native Arabic questions drawn from real examinations across the Arab world4. ArabLegalEval evaluates legal reasoning against Arabic legal frameworks6. Other projects measure dialect competence and cultural alignment. Yet these remain largely academic artifacts—fragmented, measuring isolated dimensions, lacking the prestige and institutional adoption that turn a benchmark into a global standard. ArabicMMLU is excellent; it is also absent from the release notes that define the frontier. So frontier labs have little incentive to optimize for it.

Consider a lawyer in Casablanca who asks whether a Moroccan court can revise a contract that changed circumstances have made ruinous. Article 230 of Morocco’s Dahir des Obligations et des Contrats is unambiguous: a validly formed contract holds as law between the parties, revocable only by mutual consent or by law—and Moroccan courts, unlike France’s post-2016 Code civil, do not admit revision for unforeseen hardship (imprévision). A model trained overwhelmingly on French-language legal text answers in the register of French law—fluent, well-cited, and wrong, importing a doctrine Moroccan courts reject. Nothing in the benchmark regime governing most frontier models penalizes that error, so it persists across every version and deployment. The problem is not vocabulary. The models know Arabic words. They do not consistently know what Arabic is for5.

Move the Finish Line

Which suggests a different strategy. The Arab world does not need to outcompete OpenAI. It needs to hand that giant a target it cannot afford to ignore. You do not win the race by running faster. You move the finish line.

Call the project Mizan. Its scale need not be grand. A core team of eight to ten researchers would suffice, drawn from institutions that already exist—MBZUAI, QCRI, KAUST, AUB, AUC, Mohammed VI Polytechnic—and from the Arab diaspora inside leading laboratories. No new building, no sovereign compute cluster, no nine-figure budget. The goal is a unified Arabic evaluation suite that consolidates the strongest existing benchmarks and extends them: Modern Standard Arabic reasoning, dialect competence, legal and medical understanding, Islamic textual comprehension, regional factuality, and code-switching, with public and protected evaluation sets and a transparent leaderboard tracking frontier models.

Building the benchmark is the easy part. The hard work lies beneath it, in four problems a serious suite must solve, not merely name.

Contamination. Once a test is public, models train on it, and high scores measure memorization rather than competence. Mizan answers with a date-stratified, rolling design: a small public development set, and a private test set refreshed quarterly from material created after the leading models’ training cutoffs—newly published bar-exam and medical-board questions, recent rulings, fresh dialect transcripts—each stamped with its creation date. A model that scores far higher on pre-cutoff than on post-cutoff items of equal difficulty is flagged as contaminated; canary strings seeded in the private set confirm leakage if a model ever reproduces them.

Dialect adjudication. There is no single correct Lebanese or Egyptian, so Mizan does not adjudicate between varieties—it scores within them. Each dialect item is judged by a panel of native speakers of that variety, and acceptability is treated as a distribution, not a verdict. Items where the panel itself disagrees—inter-annotator agreement below a set threshold—go to a separate “contested” set, excluded from the headline score, so the leaderboard rewards the competence the region agrees on and surfaces, rather than buries, what it does not.

Saturation. As models cluster near the ceiling on a slice, that slice stops discriminating. Mizan borrows from psychometrics: from the leaderboard’s own response matrix, item-response theory estimates each question’s difficulty and discriminating power, retires the items everyone passes into the public set, and commissions harder replacements—so the test stays informative as the field improves, the way the SAT is restandardized rather than frozen.

Open-ended scoring. BLEU and ROUGE reward surface overlap with a reference answer and miss whether a response is actually correct7. Mizan replaces them with rubric decomposition: each open-ended question carries a checklist of atomic, independently verifiable criteria, and an answer is scored on how many it satisfies. An ensemble of judge models from different families grades each criterion—an Arabic-native model among them—and those automated grades are calibrated against human raters on a sample, so that judge-human agreement is measured and reported, not assumed.

Return to Casablanca. Mizan’s legal slice scores that answer not on fluency but against a rubric: does it name the governing instrument as the DOC rather than the French Code civil; does it state the binding-force rule of Article 230 correctly; does it refuse to import imprévision; does it cite the right article? The fluent French-law answer satisfies the first criterion and fails the rest—an aggregate a conventional, overlap-based metric would round up to “good.” The rubric does not. That gap, repeated across thousands of items and published on a leaderboard, is precisely the signal a frontier lab cannot see today.

A Science for Judging Language

For centuries, Arabic civilization built sciences for judging language, not merely for producing it. In Asrar al-Balagha, al-Jurjani sought to explain why one arrangement of words yields meaning while another collapses into noise8; critics weighed poets against explicit criteria, and the doctrine of i’jaz presupposed rigorous comparison between texts. When we ask whether a model is good, we are asking a question that has a history. And the history is ours.

Structural Influence, Not Symbolic

A benchmark is a statement about what counts. The chain from that statement to a changed model is short and concrete. Suppose one Arab state—say the UAE or Morocco—makes a minimum Mizan score a procurement condition for any model deployed in its courts, public hospitals, or schools, verified by an independent body. Government technology contracts in these markets run to the hundreds of millions, and frontier labs already tune and re-evaluate their systems to win sovereign deals. Adding a Mizan target to an internal eval suite is a rounding error against a nine-figure contract. The first lab that optimizes for it publishes the score; because procurement shortlists are public, competitors must match it to stay eligible, and the leaderboard becomes a ratchet. Crucially, a lab ships one set of weights worldwide—it will not maintain a weaker model for one region—so gains made to clear a regional floor propagate into the model everyone else uses. A procurement rule in Rabat quietly edits the Arabic a student receives in Jakarta.

A realistic program runs eight to ten researchers plus annotation panels across five or six dialect regions, with frontier APIs evaluated quarterly—inference for a full benchmark run costs thousands of dollars per model, not millions. Roughly two to three million dollars a year, a public leaderboard within twelve months, and a first ministry pilot inside two years: under one-tenth the cost of a single frontier training run, for influence that outlasts it.

That is structural influence—not symbolic, but the kind that persists across model generations and geopolitical cycles, across the half-life of any single training run. And this is not a call to abandon model-building—models have their place, and some will succeed—but the deeper opportunity lies at a layer the region has so far largely ceded.

The Question Beneath the Race

Six centuries ago, Ibn Khaldun warned that the vanquished imitate the victorious1. We are repeating the pattern, this time in tokens instead of robes—optimizing for metrics we did not design, racing toward benchmarks that do not know we exist.

The challenge is not to reject the future. It is to enter it on terms that reflect our strengths. The Arab world has spent years asking how to build the next model. Perhaps it should ask a different question: who decides what a good model is? I believe the answer to that question matters more than any single training run, any single Falcon release, any single hundred-million-dollar compute cluster.

A model speaks. A benchmark decides what is worth saying.

The Mizan will decide what counts.