Model card
DeepSeek-V3.
DeepSeekopen-sourceLLM
DeepSeek's V3 model.
§ 01 · Benchmarks
Every benchmark DeepSeek-V3 has a recorded score for.
| # | Benchmark | Area · Task | Metric | Value | Rank | Date | Source |
|---|---|---|---|---|---|---|---|
| 01 | MBPP | Computer Code · Code Generation | pass@1 | 89.3% | #9 | — | source ↗ |
| 02 | MATH | Reasoning · Mathematical Reasoning | accuracy | 90.2% | #17 | — | source ↗ |
| 03 | GSM8K | Reasoning · Mathematical Reasoning | accuracy | 95.8% | #18 | — | source ↗ |
| 04 | LiveCodeBench | Computer Code · Code Generation | pass@1 | 49.2% | #19 | 2024-03-12 | source ↗ |
| 05 | MMLU | Reasoning · Commonsense Reasoning | accuracy | 88.5% | #22 | — | source ↗ |
| 06 | HumanEval | Computer Code · Code Generation | pass@1 | 82.6% | #32 | — | source ↗ |
| 07 | SWE-Bench Verified | Computer Code · Code Generation | resolve-rate | 42.0% | #36 | — | source ↗ |
| 08 | PLCC | Natural Language Processing · Polish Cultural Competency | vocabulary | 63.0% | #59 | — | source ↗ |
| 09 | PLCC | Natural Language Processing · Polish Cultural Competency | culture-and-tradition | 73.0% | #60 | — | source ↗ |
| 10 | PLCC | Natural Language Processing · Polish Cultural Competency | art-and-entertainment | 61.0% | #61 | — | source ↗ |
| 11 | PLCC | Natural Language Processing · Polish Cultural Competency | geography | 79.0% | #61 | — | source ↗ |
| 12 | PLCC | Natural Language Processing · Polish Cultural Competency | average | 69.2% | #67 | — | source ↗ |
| 13 | PLCC | Natural Language Processing · Polish Cultural Competency | history | 77.0% | #69 | — | source ↗ |
| 14 | SWE-bench Verified | Agentic AI · SWE-bench | resolve-rate | 42.0% | #70 | — | source ↗ |
| 15 | PLCC | Natural Language Processing · Polish Cultural Competency | grammar | 62.0% | #77 | — | source ↗ |
Rank column shows this model’s position vs all other models scored on the same benchmark + metric (competitors after the slash). #1 in red means current SOTA. Sorted by rank, then newest result.
§ 02 · Strengths by area
Where DeepSeek-V3 actually performs.
§ 03 · Papers
1 paper with results for DeepSeek-V3.
- 2024-03-12· Computer Code· 1 result
LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
§ 04 · Related models
Other DeepSeek models scored on Codesota.
§ 05 · Sources & freshness
Where these numbers come from.
sdadas/PLCC
7
results
arxiv
2
results
openai-simple-evals
2
results
deepseek-paper
1
result
official-leaderboard
1
result
swebench-leaderboard
1
result
editorial
1
result
11 of 15 rows marked verified.