ai-modelsdutchbenchmarkseuroevallocal-ai

Best AI Model for Dutch: Scores, Prices and Local Knowledge

November 13, 202613 min readPixel Management

This article is also available in Dutch

On Dutch sentiment analysis and summarising, the most expensive language models barely beat the cheap ones, yet they can know twice as much about the Netherlands itself.

The best AI model for Dutch on 6 November 2026 is GPT-6 Astra, with a score of 1.17 on the independent EuroEval leaderboard, although Claude 5.5, Gemini 4 and Mistral Large 4 aren't on that board. For most companies Astra isn't the best buy: GPT-6.1 Sol sits 0.05 behind it and costs a fifth as much. We didn't run these tests ourselves. All scores come from EuroEval, read on 6 November 2026, and prices come from the makers' own pricing pages on the same day, in US dollars as listed.

Which is the best AI model for Dutch, by situation?

The best AI model for Dutch depends on the job, because no single model wins everywhere. The picks below are our reading of the EuroEval results and the price lists. The scores, and the limits of that data, follow in the rest of this piece.

What you needOur pickEuroEval score (lower is better)List price per million tokens, in/out
Best overall score, price no objectGPT-6 Astra1.17 ± 0.03$10 / $50
Best balance for hosted useGPT-6.1 Sol1.22 ± 0.03$2 / $10
Strong and cheaperGemini 3.7 Flash1.29 ± 0.04$0.75 / $3.75 through 31 December
High volume, simple tasksGPT-6 Luna1.82 ± 0.12$0.10 / $0.50
Data stays on your own hardwareGemma 4 31B, Qwen3.6 27B or Mistral Small 3.21.60 to 1.82No token price, but hardware and upkeep
Dutch facts without your own documentsGemini 3.7 Flash or GPT-6.1 SolLocal knowledge 87.12% and 86.48%See above
Answers from your own documentsThe cheapest model that passes your own testReading comprehension is closeSee below

The last row is the one that matters to most companies.

What EuroEval measures, and what it doesn't

EuroEval is a European benchmark project that tests language models per language. Its Dutch leaderboard has 522 rows, 183 of them with a rank score, and uses nine Dutch datasets across eight tasks: sentiment, named entities, linguistic acceptability, reading comprehension, summarising, knowledge, common sense and simplification.

The score is an average. According to the methodology, the best model on a dataset gets a 1, and every other model gets 1 plus the number of standard deviations it sits behind the leader. A score of 1.00 would therefore mean best at everything. The margin after each score is a 95% confidence interval.

Four limits are worth knowing before you put one of these scores in front of your board:

  • The newest flagships are missing. On 6 November, Claude 5.5, Gemini 4, Gemini 3.8 Flash and Mistral Large 4 weren't on the board, and the newest Claude there is Sonnet 4.6 from February 2026. Evaluation requests for Claude Opus 5 and Mistral Large 4 are open.
  • Every top row is a validation run. The val label shows it, and EuroEval's FAQ says those scores "aren't strictly comparable" to test-split scores. The highest test-split row is Mistral Small 3.2, at 1.82 ± 0.17.
  • The setup differs per row. GLM-5.3-Flash scores 1.32 with examples in the prompt (few-shot) and 1.58 without (zero-shot). That gap of 0.26 is larger than the distance between GPT-6 Astra and GLM itself. Hosted rows are mostly zero-shot, open models mostly few-shot.
  • Agent work doesn't count. Instruction following and tool use belong to the Chat board under the methodology, and for Dutch that board still reads "soon". This score comes from the Generative board, so it says nothing about an agent working inside your systems.

Where do AI models tie in Dutch, and where don't they?

AI models sit close together in Dutch on sentiment and summarising, and far apart on factual knowledge. You can see that by comparing the 19 configurations in the top five rank groups, per dataset, which is our own cut of the EuroEval figures. Compare within a row, not between rows, because the metric changes per task.

TaskDatasetMetricLowestHighestGap
SummarisingWikiLingua-nlCHRF3++31.9635.944.0
SentimentDBRDMCC89.5094.615.1
Reading comprehensionSQuAD-nlF172.3084.5512.3
Knowledge from examsINCLUDE-nlMCC74.6394.6720.0
SimplificationDuidelijke TaalMETEOR38.5459.7921.3
Named entitiesCoNLL-nlF165.5387.8022.3
Linguistic acceptabilityDutch CoLAMCC40.2364.8524.6
Common senseWinogrande-nlMCC55.4192.9537.5
Local knowledgeMultiLoKo-nlMCC22.1484.2362.1

The five MCC rows can be compared with each other. MCC runs from -100 to 100, and 0 is guessing. Sentiment on Dutch book reviews differs by 5 points, local knowledge by 62. So a model that sorts customer feedback or summarises an email barely separates itself from its pricier neighbour.

"Language ability is a commodity" doesn't hold for everything, though. Common sense differs by 38 points, and recognising names and organisations, the heart of document extraction, runs from 66 to 88. Simplification is different. There one outlier drives the gap: Gemini 3 Flash with thinking scores 38.54, and everything else in the group is at 47.91 or higher.

Does the model know the Netherlands? Local knowledge is the real gap

Local knowledge about the Netherlands is the task on which AI models diverge most, from 41% to 88% correct within the top group and 37% for EuroLLM-22B. EuroEval tests it with MultiLoKo-nl, a dataset of 234 multiple-choice questions on topics relevant to Dutch speakers. The examples are of the "how many goals did Youri Mulder score in his last season for FC Twente?" kind. The model gets no text with the question and has to know the answer.

This dataset has no separate validation split, as its configuration shows. Validation and test rows are scored on the same questions here. That makes this the fairest column on the board.

The GPT-6 class and Gemini Flash sit around 86–88%, while open models that fit on one workstation stall at 41–46%, close to Claude Sonnet 4.6.

Local knowledge of the Netherlands: share answered correctly

Higher is better. 234 multiple-choice questions, four options each.

  • Open weights
  • Closed
  1. Gemini 3 Flash preview (thinking on)88.07%
  2. Gemini 3.7 Flash87.12%
  3. GPT-6.1 Sol86.48%
  4. GPT-6 Astra86.14%
  5. Gemini 3.6 Flash85.84%
  6. GPT-5.6 Sol75.49%
  7. GPT-6 Sol66.91%
  8. GLM-5.3-Flash63.56%
  9. Claude Sonnet 4.5 (thinking on)50.13%
  10. Claude Sonnet 4.646.31%
  11. Gemma 4 31B46.27%
  12. Qwen3.6 27B43.13%
  13. Mistral Small 3.241.07%
  14. EuroLLM-22B37.38%

Blind guessing: 25%

Source: EuroEval, MultiLoKo-nl, as of 6 November 2026. Small question set. The wrong options were generated with GPT-4.1, and EuroEval can't rule out test questions having entered training data. GLM and the open models ran few-shot, the closed models zero-shot.

Three things stand out, and each is a reason not to read this column as gospel.

Settings matter enormously. The same Gemini 3 Flash preview scores 88.07% with thinking and 27.94% without, barely above blind guessing. EuroEval doesn't explain why. You're not buying a model, you're buying a model with a setting, and you pay for that thinking: Google's pricing page labels the output price as including thinking tokens, and EuroEval gives reasoning models a budget of 8,192 tokens. Fully used at $3 per million, that's almost 2.5 US cents per question before the answer begins.

One point release can move a lot. GPT-6 Sol scores 66.91%, GPT-6.1 Sol 86.48%. We don't know why. One possible explanation is contamination: the questions come from the dev split of MultiLoKo, a benchmark from April 2025, and EuroEval itself says it can't stop public test data ending up in training corpora. That's a hypothesis, not a finding.

Finally, the set is small and partly time-bound. With 234 questions a difference of a few points means little, and the wrong options came from GPT-4.1, a model on the same board. INCLUDE-nl, a second knowledge test with exam questions on 57 topics, shows a 20-point MCC gap within the same group rather than 62. So the gap sits mainly in specifically Dutch facts, not in knowledge in general.

Why your own documents close the local-knowledge gap

Your own documents close the local-knowledge gap because the model no longer has to know anything by heart. EuroEval's knowledge task gives no context. In reading comprehension the model gets the text, and there the lead of the expensive models disappears. The three local candidates even sit above GPT-6 Astra on SQuAD-nl.

ModelLocal knowledge, no text (% correct)Reading comprehension, with text (F1)Setup
GPT-6 Astra86.1474.83zero-shot, val
GPT-6.1 Sol86.4873.82zero-shot, val
Gemini 3.7 Flash87.1276.55zero-shot, val
Gemma 4 31B46.2778.67few-shot, val
Qwen3.6 27B43.1381.01few-shot, val
Mistral Small 3.241.0781.05few-shot, test

Don't over-read this. Few-shot helps, and those are exactly the rows on top, so the table doesn't say open beats Astra. SQuAD-nl is also a machine translation of an English dataset, with a hand-corrected test set. What does hold up: given the right text, even a small open model reaches a score on a par with the top.

That moves the question from "which model knows the most" to "does my system find the right passage", because with RAG the system first pulls that passage from your own documents and hands it to the model. EuroEval doesn't test that search step, because the dataset supplies the correct text ready-made. In a real project that's where the work sits: badly split PDFs, outdated versions and tables that arrive as flat text.

For company facts, the model was never the source anyway. No language model knows your price list, your collective labour agreement or your product specs. If those are the questions you care about, 88% on Youri Mulder won't help you.

For work based on your own documents, built-in local knowledge is hardly a selection criterion. Test how well the model reads a supplied text, and spend your budget on the quality of the search.

Dutch at what price?

An AI model's list price says little about its Dutch: GPT-6.1 Sol costs a fifth of GPT-6 Astra and sits 0.05 behind, and Gemini 3.7 Flash costs over thirteen times less than Astra and sits 0.12 behind. That comparison joins two sources that weren't built for each other, with prices read on 6 November 2026 from OpenAI, Google Cloud, Google AI Studio and Anthropic. The blended price is our assumption: three parts input to one part output.

Dutch against list price

Higher on the chart is better. Price: three parts input, one part output, log scale.

  • No cheaper model scores better
  • A cheaper model scores better

Source: EuroEval (Dutch) and pricing pages of OpenAI, Google and Anthropic, as of 6 November 2026. All rows are validation runs. Gemini 3.6 and 3.7 Flash prices double on 1 January 2027. Claude points are older models: Claude 5.5 isn't on the board. Margins aren't drawn.

View the numbers
Blended list price per million tokensEuroEval score
GPT-6 Astra$201.17
GPT-6.1 Sol$41.22
Gemini 3.7 Flash$1.51.29
Gemini 3.6 Flash$1.51.39
GPT-5.6 Sol$81.45
GPT-6 Sol$41.48
Gemini 3 Flash preview (thinking on)$1.1251.55
Claude Sonnet 4.5 (thinking on)$61.57
GPT-5.6 Terra$4.51.65
Claude Sonnet 4.6$61.8
GPT-5.6 Luna$0.451.8
GPT-6 Luna$0.21.82
Gemini 3.5 Flash-Lite$0.851.88
Claude Haiku 4.5$22.08

Here's a worked example, indicative only, with these assumptions: 100 million tokens a month, three quarters input, short context, no cache, no thinking tokens, standard tier.

ModelInput, 75 millionOutput, 25 millionPer month
GPT-6 Astra$750$1,250$2,000
GPT-6.1 Sol$150$250$400
Gemini 3.7 Flash, through 31 December$56.25$93.75$150
Gemini 3.7 Flash, from 1 January 2027$112.50$187.50$300
GPT-6 Luna$7.50$12.50$20

Read this table as a floor. Four things skew the comparison:

  • A model that thinks a lot costs more per answer than its token price suggests. That's why we compare on the whole job in cost per task, not per token.
  • Prices don't last. Google charges an introductory rate for Gemini 3.6, 3.7 and 3.8 Flash through 31 December 2026 and doubles it after that, so put the date in any business case.
  • The Claude points are old. Sonnet 4.5 and 4.6 cost $3/$15, while Sonnet 5.5 now costs $2/$10 and hasn't been tested. The chart says nothing about the current Claude line.
  • DeepSeek-flash isn't plotted, because that alias now points to a newer build than EuroEval tested. GLM-5.3-Flash is out because we couldn't confirm a price on the maker's own page.

Does an English benchmark predict Dutch?

An English benchmark predicts the Dutch order well, but less well at the top. We calculated the rank correlation (Spearman) between the Dutch and English EuroEval scores for the 183 models ranked on both boards: 0.955. Restrict it to the 40 rows in the top seven Dutch rank groups and it's 0.84.

So English leaderboards are fine for a first shortlist, because a model that's mediocre in English doesn't suddenly turn good in Dutch. At the top the order does shift. DeepSeek-flash without thinking is in rank 1 in English at 1.15 and in rank 4 in Dutch at 1.50. GPT-6 Astra goes the other way: number 1 in Dutch, rank 2 in English.

The line "English benchmarks say nothing about Dutch" is therefore wrong. This one is right: if you're choosing among the top five models for Dutch work, an English ranking won't settle it.

Which AI models can you run locally for Dutch?

For local use in Dutch, Gemma 4 31B, Qwen3.6 27B and Mistral Small 3.2 are the logical starting points: all three open under Apache 2.0, all three between 1.60 and 1.82, and small enough for one workstation. Their margins overlap. The leaderboard won't choose for you.

ModelLicenceEuroEval scoreLocal knowledgeDownload, 4-bit build (Ollama)Maker
Qwen3.6 27BApache 2.01.60 ± 0.21 (val)43.13%17 GBAlibaba, China
Gemma 4 31BApache 2.01.64 ± 0.19 (val)46.27%20 GBGoogle, US
Mistral Small 3.2Apache 2.01.82 ± 0.17 (test)41.07%15 GBMistral, France

Mind the difference between the tested build and the one you download. EuroEval tested Qwen3.6 in FP8, the 4-bit download is lighter, and that 4-bit quality isn't measured on this board. For Mistral Small 3.2 the model card states about 55 GB of GPU memory at full precision. So don't treat the download size as the hardware requirement, and allow extra memory for context and concurrent users. Which hardware fits what is covered in open-weight models vs frontier.

The strongest open model on the board is GLM-5.3-Flash, at 1.32 ± 0.15 and 63.56% on local knowledge. At over 321 billion parameters that won't fit an ordinary workstation GPU: plan on a server, or a Mac with a lot of memory.

Our order for a first test: start with Mistral Small 3.2 if your policy calls for a European maker, since it's also the only one of the three with a test-split score. Start with Gemma 4 31B or Qwen3.6 if you want the highest ceiling. If a small language model doesn't suit a task, route that task to a larger model rather than making everything expensive. We build that kind of setup as part of local AI.

GPT-NL, GEITje and Fietje: is a Dutch-built model better?

A Dutch-built model isn't the better choice right now, because the board has no commercially usable Dutch model with a full score. GPT-NL, the Dutch national model project, isn't on EuroEval. The GPT-NL site says "Najaar 2026: GPT-NL v.1" (autumn 2026) and states that the model is for now available only to Launching Customers. The project's Hugging Face page showed no public models on 6 November. We found no published scores, and while there's more background in GPT-NL and European AI, the project's own site is the place to check current status.

GEITje-7B-ultra and Fietje fall out on licence. GEITje-7B-ultra is CC-BY-NC-4.0, and the Fietje model card says it may not be used for commercial purposes. Note that EuroEval shows a tick in the Commercial column for fietje-2b-instruct, so always check a licence at the source. Neither has a rank score, because neither has been run on every dataset.

European models are on the board, just not at the top. EuroLLM-22B-Instruct scores 2.07 ± 0.19 on the test split and Apertus v1.5 70B scores 2.04 ± 0.20 on validation. Dutch or European origin can be a good reason to choose a model, for instance for transparency about training data. Language quality isn't that reason today.

Being factually right isn't the same as admitting doubt

The "Grip on LLMs" study by the City of Amsterdam and the University of Amsterdam shows that a model that's often right doesn't necessarily admit when it doesn't know. The paper of 10 August 2026 tested 31 models for government use in Dutch. GPT-5 scored highest on factuality (0.76) and lowest on honesty (0.14), and GPT-4o reached 0.43 on honesty. The authors conclude that no single model excels across all dimensions.

Honesty here means acknowledging uncertainty or the limits of what the model can do. The study ran through the city's Azure environment, without Claude, on models of the GPT-5 generation, so the figures say nothing about the GPT-6 line. The principle still helps. For an internal knowledge assistant, "I don't know" is often the best answer. EuroEval shows a separate column for hallucinations, but it doesn't count toward the rank score.

How do you test an AI model on Dutch for your own company?

A test of your own in Dutch takes a few days and decides more than any leaderboard, because no benchmark knows your jargon, documents and cost of errors. The full method is in evaluating AI benchmarks. For Dutch we add five items:

  1. Use real Dutch text with your own trade language, abbreviations, and Flemish or informal spellings.
  2. Test names and amounts from your own records separately. CoNLL-nl showed a 22-point gap among top models.
  3. Ask twenty factual questions about your sector with no document attached, and count how often the model invents an answer instead of flagging doubt.
  4. Run each model with and without thinking, and with and without examples in the prompt. On EuroEval, one choice moved GLM-5.3-Flash from 1.32 to 1.58, the other moved Gemini 3 Flash from 88% to 28% on local knowledge.
  5. Measure cost per completed task, not per token.

There's a case where you can skip all this: you're classifying or summarising short Dutch text and a person checks the output. Take the cheapest model that passes a handful of tests. You don't need to buy anything more expensive for that.

Want that selection run against your own processes? How to buy that kind of work is covered in our guide to hiring AI consulting. We do that selection as part of AI consulting.

Service

Learn more about local AI?

View service →

Curious how much time you could save?

Request a free efficiency audit. We'll analyze your processes and show you where the gains are, no strings attached.