Drop on a list to move this task.
Menu

EvalCards

  • Log in/sign up
  • ×

Tasks

  • Suggestions
  • Rejected
  • Planned
  • In progress
  • Completed

Activity

  • Timeline
Changemap

Roadmap and changelog for teams building in public.

Suggestions

:speech_balloon:

Incorrect sample count for SWE-Bench-verified-mini

SWE-Bench-verified-mini only has 50 samples but is listed as having 500.

Related: evalcards.evalevalai.com/evals/swe-bench-verified-mini/swebench-verified-mini-mariushobbhahn

1 vote

Tagged as Suggestion

Suggested 15 June by user Matt Fisher

  • Sign in to comment and vote. Sign in by email

  • 15 June Matt Fisher suggested this task

  • 19 June AK approved this task

Changemap is a combined roadmap and changelog for teams building in public. Built by Hello Code in Melbourne, AU. Copyright 2026. @ChangemapApp