Monday, July 13, 2026probability mass ≠ 1.0
Machine-runLog-linearReceipted
THE REGRESSION DESKThe Stochastic Parrot
Regression // 586 // 2026-10-30 // Epoch AI, keyless

Did Chinchilla
start the shift,
or just name it?

174 from-scratch, Language-domain models, 2018-2026, Epoch AI's own Notable AI Models dataset. Tokens-per-parameter ratio has risen +0.270 log₁₀ units/yr, 95% CI [+0.200, +0.340] — excludes zero. But the slope-change exactly at Chinchilla's own 2022-03-29 publication date is +0.079, CI [-0.188, +0.345] — contains zero, under all six specifications tried.

Two-panel chart. Left: scatter of 174 language models' training tokens per parameter, log scale, against publication year 2018-2026, with separate OLS trend lines before and after a red dashed line marking Chinchilla's 2022-03-29 publication date and a green dotted reference line at the paper's own 20-tokens-per-parameter optimum; the two trend lines sit on nearly the same slope with no visible kink at the red line. Right: a forest plot of the break-test slope-change coefficient under six specifications -- classical SE, HC3-robust SE, confidence-filtered, outlier-dropped, and two alternative candidate break dates (ChatGPT release, Llama 1 release) -- every one straddling zero.
Left: the ratio climbs steadily across the whole window; no visible kink at the paper. Right: six specifications test for a kink at the cutoff, and all six find none.
Full-sample trend (n=174)
+0.270 log₁₀/yr
95% CI [+0.200, +0.340], p=1.5e-12. Excludes zero -- the rise is real.
Slope-change AT the cutoff
+0.079
95% CI [-0.188, +0.345], p=0.562. Contains zero -- no detectable kink.

Backlog #96: “Chinchilla, checked — does more data beat more parameters?” DeepMind's 2022 paper (Hoffmann et al., “Training Compute-Optimal Large Language Models,” arXiv:2203.15556, submitted 2022-03-29) argued that the field's biggest models — GPT-3, Gopher, PaLM — were trained on too few tokens for their parameter count, and that a compute-optimal model should scale tokens and parameters together, landing near a 20-tokens-per-parameter ratio. The popular summary is “Chinchilla changed how everyone trains models.” This run tests whether the paper actually marks a break in the data, using Epoch AI's own public Notable AI Models dataset (keyless CSV): 174 from-scratch, Language-domain models, 2018-2026, with both a disclosed parameter count and a disclosed training-token count, dropping fine-tuned derivatives (whose reported training set is the fine-tuning data, not the pretraining corpus — a different, non-comparable ratio against the full inherited parameter count).

The ratio really has climbed, continuously, since 2018: log₁₀(tokens per parameter) rises +0.270 per year, 95% CI [+0.200, +0.340], p=1.5e-12, R²=0.253 — excludes zero. In plain terms, the typical model's tokens-per-parameter ratio has risen roughly 1.86× a year for eight straight years.

But the question this run exists to answer is narrower: did Chinchilla's own publication date mark a kink in that line? A model with a year term, a post-Chinchilla dummy, and their interaction tests exactly that — the interaction coefficient is the slope-change at the cutoff. It is +0.079, 95% CI [-0.188, +0.345], p=0.562 — contains zero. HC3-robust SE agrees: CI [-0.158, +0.315]. Fit separately, the pre-Chinchilla slope (+0.292, CI [+0.096, +0.488], n=100) and the post-Chinchilla slope (+0.370, CI [+0.204, +0.537], n=74) both individually exclude zero and sit on nearly the same line — the rise was already underway, and the paper did not visibly speed it up.

The null holds under every alternative specification tried. Restricting to Epoch's own “Confident”-confidence rows only (n=119): interaction +0.106, CI [-0.235, +0.448]. Dropping the single most extreme ratio in the sample (AFM-on-device, 2,779 tokens/param; n=173): +0.071, CI [-0.194, +0.336]. Substituting two other dates people commonly credit with “the shift” — ChatGPT's public release (+0.038, CI [-0.274, +0.350]) and Llama 1's release (+0.021, CI [-0.306, +0.349]) — returns the same null. No candidate date in this run finds a kink.

Named, for scale. GPT-3 175B (2020) trained at 1.4 tokens/param; Gopher 280B (2021), DeepMind's own prior flagship and the paper's lead example of undertraining, at 1.1. Chinchilla itself is in the dataset at exactly its own prescription, 70B params on 1.4T tokens, 20.0 tokens/param. By 2024, Llama 3-70B ran 214 tokens/param and Qwen2.5-32B ran 554 — both far past Chinchilla's own optimum, because at today's hardware and serving costs, a smaller model trained longer is cheaper to run forever after, even past the point that minimizes training compute alone. The share of models at or above Chinchilla's 20:1 ratio rose from 27% before the paper's date to 66% after it, and the median ratio rose from 1.8 to 28.6 — both real shifts in level across those two multi-year windows (bootstrap gap in mean log-ratio +0.966, CI [+0.628, +1.286], excludes zero) — but a level difference between two windows is not the same claim as a kink exactly at the boundary between them, and the break test above is the test built to tell those two claims apart.

Read plainly. “More data beats more parameters” is directionally true of the eight-year trend: the field really has moved toward training smaller models on much more data. But the data does not support crediting Chinchilla's own publication date with starting that movement, or even visibly accelerating it — the rate of rise before the paper and after it are statistically indistinguishable under six separate specifications. The more defensible read is that Chinchilla gave the field's own already-underway practice a theoretical name and a number, rather than causing a break in the practice itself.

The math

log₁₀(training tokens ÷ parameters) ~ β₀ + β₁·year [+ β₂·post + β₃·post·year] · one row per model

Trend fits

Specificationnslope, log₁₀ tokens/param per yr95% CIFitVerdict
Full sample, 2018-2026n=174+0.270[+0.200, +0.340]R²=0.253excludes zero, p=1.5e-12
Before Chinchilla onlyn=100+0.292[+0.096, +0.488]R²=0.082excludes zero, p=0.004
After Chinchilla onlyn=74+0.370[+0.204, +0.537]R²=0.215excludes zero, p=3.2e-05

Break test: does the slope change at the candidate cutoff? (the coefficient on post·year — this is the test the run's headline rests on)

Break-test specification (candidate date the slope might kink at)nslope-change coefficient95% CIVerdict
Classical SEn=174+0.079[-0.188, +0.345]contains zero, p=0.562
HC3-robust SEn=174+0.079[-0.158, +0.315]contains zero, p=0.516
Confidence-filteredn=119+0.106[-0.235, +0.448]contains zero, p=0.539
Outlier dropped (AFM-on-device)n=173+0.071[-0.194, +0.336]contains zero, p=0.599
ChatGPT release (2022-11-30)n=174+0.038[-0.274, +0.350]contains zero, p=0.811
Llama 1 release (2023-02-24)n=174+0.021[-0.306, +0.349]contains zero, p=0.899

Method. Epoch AI publishes a flat, keyless CSV export of its Notable AI Models database at epoch.ai/data/notable_ai_models.csv — every model the project judges notable by citation count, training compute, or historical significance, each row carrying parameter count, training dataset size, publication date, and a per-field confidence rating (Confident / Likely / Speculative) reflecting how directly each number is sourced. This run keeps rows with Domain == “Language”, both Parameters and Training dataset size (total) disclosed, publication date 2018-01-01 or later, and an empty Base model field — the last filter drops fine-tuned or instruction-tuned derivatives (InstructGPT, Flan-PaLM, CoEdiT, and similar), whose “training dataset” in the source records only the fine-tuning set, not the pretraining corpus, which would understate their effective ratio against the full parameter count they inherit from a base model. 174 rows survive every filter. “Before” and “after” Chinchilla are split at the paper's own arXiv submission timestamp, 2022-03-29.

Limits, stated plainly. Epoch's dataset selects for notability, not for a random or complete sample of all models ever trained — small, unremarkable models are underrepresented, especially in later years, which could bias the measured ratio in either direction depending on which models get written up. Parameter and token counts for proprietary models (GPT-4-class, for instance) are frequently estimates built from public statements or third-party analysis, not disclosed figures, and carry Epoch's own “Likely” or “Speculative” confidence tag rather than “Confident”; the confidence-filtered row in the break-test table shows the result is not solely an artifact of the less-certain rows, but does not make every individual number in the raw table certain. This run tests only whether the RATE of change in the ratio shifts at a handful of candidate dates — it cannot rule out a kink at some other date not tested, and it cannot speak to whether any individual lab explicitly cited the Chinchilla paper in choosing its own training recipe.

The data (all 174 models fitted)

chinchilla_scaling_586.csv · fit output (JSON).

ModelOrganizationPublishedParametersTraining tokensTokens/paramConfidence
QRNNSalesforce Research2018-02-01135,000,000103,000,0000.76Likely
ELMoUniversity of Washington,Allen Institute for AI2018-02-0194,000,0002,000,000,00021.28Speculative
4 layer QRNN (h=2500)Salesforce Research2018-03-22151,000,000103,000,0000.68Likely
Dropout-LSTM+Noise(Bernoulli) (WT2)Columbia University,New York University (NYU),Princeton University2018-05-0351,000,0002,000,0000.04Likely
aLSTM(depth-2)+RecurrentPolicy (WT2)University of Manchester,Alan Turing Institute2018-05-2232,000,0002,000,0000.06Confident
GPT-1OpenAI2018-06-01117,000,0001,333,333,33311.40Likely
AWD-LSTM-MoS+PDR + dynamic evaluation (WT2)IBM2018-08-1435,000,0002,000,0000.06Confident
(ensemble): AWD-LSTM-DOC (fin) × 5 (WT2)NTT Communication Science Laboratories,Tohoku University2018-08-30185,000,0002,000,0000.01Confident
Transformer + Simple Recurrent UnitASAPP,Cornell University,Google,Princeton University2018-09-1790,000,000112,500,0001.25Confident
AWD-LSTM-MoS + dynamic evaluation (WT2, 2018)Peking University,Microsoft Research Asia2018-09-1835,000,0002,000,0000.06Confident
LSTM+NeuralCacheKU Leuven,ESAT - PSI,Apple2018-09-242,100,0002,000,0000.95Likely
Transformer (Adaptive Input Embeddings) WT103Facebook AI Research2018-09-28247,000,000100,000,0000.40Confident
BERT-LargeGoogle2018-10-11340,000,0002,649,900,0007.79Confident
TrellisNetCarnegie Mellon University (CMU),Bosch Center for Artificial Intelligence,Intel Labs2018-10-15180,000,000103,000,0000.57Confident
Mesh-TensorFlow Transformer 4.9B (language)Google Brain2018-11-054,900,000,0005,000,000,0001.02Confident
Mesh-TensorFlow Transformer 2.9B (translation)Google Brain2018-11-052,900,000,0001,550,000,0000.53Likely
Multi-cell LSTMUniversity of Hyderabad2018-11-157,200,000929,0000.13Likely
GPipe (Transformer)Google2018-11-166,000,000,0001,450,000,000,000241.67Likely
Transformer ELMoAllen Institute for AI,University of Washington2019-01-0156,000,0002,000,000,00035.71Confident
Transformer-XL (257M)Carnegie Mellon University (CMU),Google Brain2019-01-09257,000,000103,000,0000.40Speculative
MT-DNNMicrosoft2019-01-31330,000,0001,000,0000.00Confident
GPT-2 (1.5B)OpenAI2019-02-141,500,000,00010,666,666,6677.11Speculative
SciBERTAllen Institute for AI2019-03-26110,000,0003,170,000,00028.82Confident
WeNet (Penn Treebank)Amazon2019-04-0823,000,000929,0000.04Likely
True-Regularization+Finetune+Dynamic-EvalMobvoi,Williams College2019-04-087,000,000929,0000.13Confident
Transformer-XL + RMS dynamic evalUniversity of Edinburgh2019-04-17257,000,000103,000,0000.40Confident
BERT-Large-CAS (PTB+WT2+WT103)Amazon2019-04-20395,000,0001,300,000,0003.29Likely
AWD-LSTM-DRILL + dynamic evaluation† (WT2)IDIAP2019-05-1434,000,0002,000,0000.06Likely
XLNetCarnegie Mellon University (CMU),Google Brain2019-06-01340,000,00032,890,000,00096.74Confident
AWD-LSTM + MoS + Partial ShuffledUniversity of Texas at Austin2019-06-1035,000,0002,000,0000.06Confident
RoBERTa LargeFacebook,University of Washington2019-07-01355,000,00042,666,666,666120.19Confident
Mogrifier (d2, MoS2, MC) + dynamic evalDeepMind,University of Oxford2019-09-0435,000,0002,000,0000.06Confident
Megatron-LM (8.3B)Nvidia2019-09-178,300,000,00046,400,000,0005.59Likely
Megatron-LM (1.2B)Nvidia2019-09-171,200,000,000157,000,000,000130.83Confident
Megatron-BERTNvidia2019-09-173,900,000,0006,960,000,0001.78Confident
Adaptive Inputs + LayerDropFacebook AI Research,LORIA2019-09-25423,000,000103,000,0000.24Confident
ALBERTToyota Technological Institute at Chicago,Google Research2019-09-2618,000,0003,300,000,000183.33Confident
DistilBERTHugging Face2019-10-0266,000,000495,000,0007.50Confident
T5-11BGoogle2019-10-2311,000,000,00034,000,000,0003.09Confident
T5-3BGoogle2019-10-232,800,000,0005,100,000,0001.82Confident
BART-largeFacebook AI2019-10-29406,291,45642,666,666,666105.01Likely
Base LM + kNN LM + Continuous CacheStanford University,Facebook AI Research2019-11-01247,000,000103,000,0000.42Likely
XLM-RoBERTaFacebook AI2019-11-05550,000,000167,000,000,000303.64Confident
Sandwich TransformerAllen Institute for AI,Facebook AI Research2019-11-10209,000,000700,000,0003.35Speculative
CamemBERTFacebook,INRIA,Sorbonne University2019-11-10335,000,00028,615,771,78085.42Confident
Transformer-XL DeFINE (141M)University of Washington,Allen Institute for AI2019-11-27141,000,000103,000,0000.73Speculative
MMLSTM (WT-2)Beijing University of Posts and Telecommunications,University of West London2019-12-0532,300,0002,000,0000.06Likely
MMLSTM (PTB)Beijing University of Posts and Telecommunications,University of West London2019-12-0521,300,000929,0000.04Likely
MeenaGoogle Brain2020-01-282,600,000,00053,333,333,33320.51Confident
TaLK ConvolutionCarleton University2020-02-08240,000,000103,000,0000.43Likely
ALBERT-xxlargeToyota Technological Institute at Chicago,Google2020-02-09235,000,0003,300,000,00014.04Speculative
Turing-NLGMicrosoft2020-02-1317,000,000,00046,400,000,0002.73Likely
FFN SwiGLUGoogle2020-02-14220,000,00050,600,083,456230.00Confident
Feedback TransformerLORIA,University of Lorraine,Facebook AI Research2020-02-21126,000,000103,000,0000.82Speculative
TCAN (WT2)Nanjing University,Ant Group2020-02-2833,000,0002,000,0000.06Confident
TransformerXL + spectrum controlUniversity of California Los Angeles (UCLA),JD.com2020-03-11151,000,000103,000,0000.68Speculative
Routing Transformer (WT-103)Google Research2020-03-1279,500,000103,000,0001.30Confident
Tensor-Transformer(1core)+PN (WT103)University of California (UC) Berkeley2020-03-1785,300,000103,000,0001.21Confident
ELECTRAStanford University,Google,Google Brain2020-03-23335,000,00033,000,000,00098.51Likely
Retrieval-Augmented GeneratorFacebook,New York University (NYU),University College London (UCL)2020-05-22626,000,0003,074,5600.00Confident
GPT-3 175B (davinci)OpenAI2020-05-28174,600,000,000238,000,000,0001.36Confident
GShard (dense)Google2020-06-302,300,000,000346,666,666,667150.72Confident
DeLighTUniversity of Washington,Allen Institute for AI,Facebook AI Research2020-08-0399,000,000103,000,0001.04Likely
ERNIE-GEN (large)Baidu2020-08-06340,000,000114,666,666,667337.25Speculative
mT5-XXLGoogle,Google Research2020-10-2013,000,000,0001,000,000,000,00076.92Confident
German ELECTRA Largedeepset,Bayerische Staatsbibliothek Muenchen2020-10-21335,000,00036,383,733,333108.61Confident
CPM-LargeTsinghua University,Beijing Academy of Artificial Intelligence / BAAI2020-12-012,600,000,00016,700,000,0006.42Confident
CT-MoS (WT2)Google,National Tsing Hua University2020-12-2545,000,0002,000,0000.04Confident
ERNIE-Doc (247M)Baidu2020-12-31247,000,000103,000,0000.42Confident
SwitchGoogle2021-01-111,571,000,000,00086,400,000,0000.05Confident
SRU++ LargeASAPP2021-02-24234,000,000103,000,0000.44Confident
Generative BSTFacebook AI Research2021-03-059,431,810,04856,800,000,0006.02Confident
PLUGAlibaba2021-04-1927,000,000,00060,000,000,0002.22Confident
ByT5-XXLGoogle,Google Research2021-05-2812,900,000,0001,048,576,000,00081.28Likely
EMDRMila - Quebec AI (originally Montreal Institute for Learning Algorithms),McGill University,DeepMind2021-06-09440,000,000171,600,000,000390.00Confident
DeBERTaMicrosoft2021-06-101,500,000,00020,800,000,00013.87Confident
Adaptive Input Transformer + RDMicrosoft Research Asia,Soochow University2021-06-28247,000,000103,000,0000.42Likely
ERNIE 3.0Baidu2021-07-0510,000,000,000375,000,000,00037.50Confident
Jurassic-1-JumboAI21 Labs2021-08-11178,000,000,000300,000,000,0001.69Confident
XLMR-XXLFacebook AI Research2021-08-1710,700,000,000167,000,000,00015.61Confident
PermuteFormerPeking University2021-09-06149,697,024103,000,0000.69Speculative
HyperCLOVA 204BNAVER2021-09-10204,000,000,000560,000,000,0002.75Speculative
PLATO-XLBaidu2021-09-2011,000,000,000150,000,000,00013.64Confident
Megatron-Turing NLG 530BMicrosoft,Nvidia2021-10-11530,000,000,000270,000,000,0000.51Confident
Yuan 1.0Inspur2021-10-12245,730,000,000180,000,000,0000.73Confident
S4Stanford University2021-10-31249,000,000103,000,0000.41Likely
Gopher (280B)DeepMind2021-12-08280,000,000,000300,000,000,0001.07Confident
GLaMGoogle2021-12-131,200,000,000,000600,000,000,0000.50Confident
LongT5Google Research2021-12-153,000,000,000524,288,000,000174.76Confident
XGLM-7.5BMeta AI,Facebook AI Research2021-12-207,500,000,000500,000,000,00066.67Confident
ERNIE 3.0 TitanBaidu,Peng Cheng Laboratory2021-12-23260,000,000,000668,000,000,0002.57Confident
data2vec (language)Meta AI2022-01-20705,134,592131,072,000,000185.88Likely
AlphaCodeDeepMind2022-02-0241,100,000,000967,000,000,00023.53Confident
RETRO-7BDeepMind2022-02-077,500,000,000419,430,400,00055.92Confident
GPT-NeoX-20BEleutherAI2022-02-0920,000,000,000341,173,367,96517.06Confident
LaMDAGoogle2022-02-10137,000,000,0002,080,000,000,00015.18Confident
ST-MoEGoogle,Google Brain,Google Research2022-02-17269,000,000,0001,500,000,000,0005.58Likely
PolyCoderCarnegie Mellon University (CMU)2022-02-262,700,000,00039,300,000,00014.56Likely
DeepNetMicrosoft Research2022-03-013,200,000,000272,629,760,00085.20Confident
Segatron-XL large, M=384 + HCPMicrosoft Research,University of Waterloo2022-03-21257,000,000103,000,0000.40Confident
ChinchillaDeepMind2022-03-2970,000,000,0001,400,000,000,00020.00Confident
PaLM (540B)Google Research2022-04-04540,350,000,000780,000,000,0001.44Confident
Sparse all-MLPMeta AI2022-04-149,410,000,000100,000,000,00010.63Confident
OPT-175BMeta AI2022-05-02175,000,000,000180,000,000,0001.03Confident
UL2Google Research,Google Brain2022-05-1020,000,000,0001,000,000,000,00050.00Confident
GPT-2 Medium (FlashAttention)Stanford University,University at Buffalo2022-05-27355,000,00010,130,000,00028.54Confident
DITTOTsinghua University,Apple,Westlake University,Chinese University of Hong Kong (CUHK)2022-06-06750,000,000103,000,0000.14Confident
CodeT5-largeSalesforce2022-07-05770,000,00010,500,000,00013.64Likely
NLLBMeta AI2022-07-0654,500,000,000300,000,000,0005.50Confident
BLOOM-176BHugging Face,BigScience2022-07-11176,247,271,424379,000,000,0002.15Confident
AlexaTM 20BAmazon2022-08-0219,750,000,0001,319,000,000,00066.78Confident
GLM-130BTsinghua University2022-08-04130,000,000,000152,000,000,0001.17Confident
Mogrifier RLSTM (WT2)DeepMind2022-11-0335,000,0002,666,6670.08Confident
Fusion in EncoderSamsung2022-11-18330,000,000960,0000.00Likely
ALM 1.0Beijing Academy of Artificial Intelligence / BAAI2022-11-28335,000,00022,696,572,40067.75Speculative
Vega v2Wuhan University,JD Explore Academy,Shanghai AI Lab,Nanyang Technological University,Washington University in St Louis,Chongqing University of Posts and Telecommunications,University of Sydney2022-12-046,000,000,0006,439,999,9991.07Confident
Hybrid H3-2.7BStanford University,University at Buffalo2022-12-282,700,000,000400,000,000,000148.15Likely
LLaMA-65BMeta AI2023-02-2465,200,000,0001,400,000,000,00021.47Confident
Falcon-40BTechnology Innovation Institute2023-03-1540,000,000,0001,000,000,000,00025.00Confident
PanGu-ΣHuawei Noah's Ark Lab2023-03-201,085,000,000,000329,000,000,0000.30Confident
BloombergGPTBloomberg,Johns Hopkins University2023-03-3050,558,868,480569,000,000,00011.25Confident
Incoder-6.7BFacebook AI Research,University of Washington,University of California (UC) Berkeley,Carnegie Mellon University (CMU),Toyota Technological Institute at Chicago2023-04-096,700,000,00052,000,000,0007.76Confident
StarCoderHugging Face,ServiceNow,Northeastern University,Mila - Quebec AI (originally Montreal Institute for Learning Algorithms),Carnegie Mellon University (CMU),Johns Hopkins University,Leipzig University,ScaDS.AI,Queen Mary University of London,Roblox,Sea AI Lab,Technion - Israel Institute of Technology,Monash University,CSIRO,Data61,McGill University,Saama,University of British Columbia (UBC),Massachusetts Institute of Technology (MIT),Technical University of Munich,IBM,University of Vermont,UnfoldML,SAP,University of Notre Dame,Columbia University,New York University (NYU),University of Allahabad,Discover Dollar,Toloka,Telefonica,Stanford University,Weizmann Institute of Science,Alan Turing Institute,Wellesley College,EleutherAI,Forschungszentrum Julich2023-05-0915,500,000,000203,750,000,00013.15Confident
PaLM 2Google2023-05-10340,000,000,0003,600,000,000,00010.59Likely
CodeT5+Salesforce2023-05-2016,000,000,00051,500,000,0003.22Confident
InternLMShanghai AI Lab,SenseTime2023-07-06104,000,000,0001,600,000,000,00015.38Confident
Llama 2-70BMeta AI2023-07-1870,000,000,0002,000,000,000,00028.57Confident
Llama 2-7BMeta AI2023-07-187,000,000,0002,000,000,000,000285.71Confident
JaisCerebras Systems,Mohamed bin Zayed University of Artificial Intelligence (MBZUAI),Inception G422023-08-2913,000,000,000395,000,000,00030.38Confident
Falcon-180BTechnology Innovation Institute2023-09-06180,000,000,0003,500,000,000,00019.44Confident
CODEFUSION (Python)Microsoft,Microsoft Research2023-10-2675,000,0004,390,4000.06Confident
Skywork-13BKunlun Inc.2023-10-3013,000,000,0003,200,000,000,000246.15Confident
Yi-34B01.AI2023-11-0234,000,000,0003,100,000,000,00091.18Confident
Grok-1xAI2023-11-04314,000,000,0006,200,000,000,00019.75Likely
RoFormerZhuiyi Technology2023-11-08110,000,0003,276,800,00029.79Confident
Nemotron-3-8BNvidia2023-11-158,000,000,0003,800,000,000,000475.00Confident
Qwen-72BAlibaba2023-11-3072,000,000,0003,000,000,000,00041.67Confident
Qwen1.5-72BAlibaba2024-02-0472,000,000,0003,000,000,000,00041.67Confident
Aramco Metabrain AISaudi Aramco2024-03-04250,000,000,0007,000,000,000,00028.00Likely
DBRXDatabricks2024-03-27132,000,000,00012,000,000,000,00090.91Confident
Llama 3-70BMeta AI2024-04-1870,000,000,00015,000,000,000,000214.29Confident
Yi-Large01.AI2024-05-13100,000,000,0003,000,000,000,00030.00Speculative
Qwen2-72BAlibaba2024-06-0772,710,000,0007,000,000,000,00096.27Confident
Nemotron-4 340BNvidia2024-06-14340,000,000,0009,000,000,000,00026.47Confident
Llama 3.1-405BMeta AI2024-07-23405,000,000,00015,600,000,000,00038.52Confident
AFM-on-deviceApple2024-07-292,730,000,0007,588,000,000,0002779.49Confident
Qwen2.5-32BAlibaba2024-09-1732,500,000,00018,000,000,000,000553.85Confident
Qwen2.5-72BAlibaba2024-09-1972,700,000,00018,000,000,000,000247.59Confident
Telechat2-115BChina Telecom2024-09-20115,000,000,00010,000,000,000,00086.96Confident
Doubao-proByteDance2024-10-28500,000,000,0008,350,000,000,00016.70Likely
Hunyuan-LargeTencent2024-11-06389,000,000,0007,000,000,000,00017.99Confident
Llama 3.3 70BMeta AI2024-12-0670,000,000,00015,000,000,000,000214.29Confident
EXAONE 3.5 32BLG AI Research2024-12-0932,000,000,0006,500,000,000,000203.12Confident
DeepSeek-V3DeepSeek2024-12-24671,000,000,00014,800,000,000,00022.06Confident
Hunyuan-TurboSTencent2025-03-11560,000,000,00016,000,000,000,00028.57Confident
DeepSeek-V3 (Mar 2025)DeepSeek2025-03-24671,000,000,00014,800,000,000,00022.06Confident
Pangu UltraHuawei2025-04-10135,000,000,00013,200,000,000,00097.78Confident
Qwen3-235B-A22BAlibaba2025-04-29235,000,000,00036,000,000,000,000153.19Likely
Kimi K2Moonshot2025-07-111,000,000,000,00015,500,000,000,00015.50Confident
EXAONE 4.0 (32B)LG AI Research2025-07-1532,000,000,00014,000,000,000,000437.50Confident
Qwen3-Coder-480B-A35BAlibaba2025-07-22480,000,000,0007,500,000,000,00015.62Confident
Qwen3-235B-A22B-Thinking (Jul 2025)Alibaba2025-07-25235,000,000,00036,000,000,000,000153.19Likely
Qwen3-235B-A22B (Jul 2025)Alibaba2025-07-25235,000,000,00036,000,000,000,000153.19Likely
GLM-4.5Z.ai (Zhipu AI),Tsinghua University2025-07-28355,000,000,00023,100,000,000,00065.07Confident
LongCat-FlashMeituan Inc2025-09-01560,000,000,00023,000,000,000,00041.07Likely
Qwen3-MaxAlibaba2025-09-051,000,000,000,00036,000,000,000,00036.00Speculative
GLM-4.6Z.ai (Zhipu AI),Tsinghua University2025-09-30357,000,000,00023,000,000,000,00064.43Likely
Ling-1TAnt Group2025-10-101,000,000,000,00020,000,000,000,00020.00Confident
Olmo 3Allen Institute for AI2025-11-2032,000,000,0005,500,000,000,000171.88Confident
Solar Open 100BUpstage2025-12-31102,000,000,00019,700,000,000,000193.14Confident
K-EXAONELG AI Research2025-12-31236,000,000,00011,000,000,000,00046.61Confident
MiMo-V2.5-ProXiaomi Corp2026-04-231,020,000,000,00027,000,000,000,00026.47Likely
Solar Open2 250BUpstage2026-06-28250,300,000,00011,000,000,000,00043.95Likely
LongCat-2.0Meituan Inc2026-06-291,600,000,000,00035,000,000,000,00021.88Likely
Sources. Epoch AI, Notable AI Models dataset, keyless CSV export · Hoffmann et al., “Training Compute-Optimal Large Language Models,” arXiv:2203.15556 (2022), the Chinchilla paper itself, cited for its own publication date and compute-optimal ratio, not as a data source.

← The Regression Desk