Codesota · Papers · Multimodal2024-10-25
Paper

SWE-bench Verified

arXiv ↗
§ 01 · Benchmark results

4 results reproduced from this paper.

#ModelVendorBenchmarkMetricValueSOTADateSource
01GPT-4oOpenAIMMBenchaccuracy83.4%2024-10-25source ↗
02GPT-4oOpenAIMMMUaccuracy69.1%2024-10-25source ↗
03GPT-4oOpenAITextVQAaccuracy77.4%2024-10-25source ↗
04GPT-4oOpenAIVQA v2.0accuracy78.5%2024-10-25source ↗
§ 02 · Models

1 model from this paper.

evaluates
GPT-4o
OpenAI
§ 04 · Related papers

Other Multimodal papers tracked on Codesota.

  1. 2025-02-19 · 3 results
    Qwen2.5-VL Technical Report
  2. 2025-01-22 · 2 results
    InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models
  3. 2025-01-15 · 1 result
    Gemini 2.0 Flash Technical Report
  4. 2024-10-22 · 1 result
    Claude 3.5 Sonnet Model Card
  5. 2024-09-18 · 4 results
    Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution
  6. 2024-04-25 · 4 results
    InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
Read next

Three places to go from here.

Index
All papers
Every ML paper with a sourced benchmark result, sorted by publication date.
Replacement
Papers with Code is dead — alternatives
What replaced PWC for each use case: LLMs, OCR, speech, vision, robotics.
Top hub
Multimodal
Every benchmark in Multimodal.