Research
Benchmarks
How the Lumen family performs across public and internal evaluations. We report the setup alongside the score — numbers are only as trustworthy as the methodology behind them.
| Benchmark | Lumen 3 | Lumen 2 | Lumen Nano |
|---|---|---|---|
| Graduate reasoning (GPQA) | 84.1 | 71.6 | 58.3 |
| Competition math (AIME) | 79.4 | 62.0 | 41.7 |
| Code generation (SWE-bench) | 66.8 | 49.2 | 27.9 |
| Long-context recall (1M tokens) | 99.2 | 94.7 | 88.1 |
| Tool use (agentic tasks) | 81.5 | 68.3 | 52.6 |
| Calibrated refusals (internal) | 93.7 | 85.1 | 80.4 |
Scores are pass@1 unless noted, averaged over five runs at temperature 0. Long-context recall is measured with a needle-in-a-haystack protocol at full 1M-token context. Full methodology and per-task breakdowns ship with each model's technical report.