MathEL provides automatic benchmarks with best practices for evaluating AI performance on mathematical competitions including IMO (1959-2025), Putnam (1938-2025), and IMC (1994-2024).

Overview

• IMO Average Score by Model: Average scores across all 398 IMO problems
• IMO Perfect Scores (7/7): Percentage of problems receiving full marks
• IMO Failure Rate (0/7): Percentage of problems with zero score
• IMO Score Distribution: Distribution of scores (0-7) for each model

Problems

• IMO Problems by Category: Distribution of problems across mathematical domains
• Problems by Competition: Number of problems in each competition benchmark
• IMO Problem Subcategories: All 32 subcategories grouped by mathematical category

Performance

• IMO Performance by Category: Average scores by mathematical category
• IMO Performance by Problem Position: Performance on Day 1 vs Day 2 problems
• Gemini-3-Pro Performance: Performance across competitions
• IMO Performance Over Time: Average scores by decade
• AI Performance by Category (IMO): Heatmap of model performance
• IMO AI Performance by Difficulty: AI performance on easy, medium, and hard problems
• IMO Performance by Year: Year-by-year performance comparison

Solutions

• IMO Solution Length Comparison: Average solution length by model
• IMO Solution Techniques: Average number of techniques per solution
• IMO Solution Technique Diversity: Technique usage patterns by category
• IMO Solution Technique Usage: Human vs AI technique comparison
• IMO Solution Technique Preferences: Diverging chart of Human vs AI preferences

Compute

• Average Token Usage per IMO Problem: Token consumption by model
• Average Compute Time per IMO Problem: Processing time by model

Leakage

• Overall Leakage Distribution: Proportion of problems with data contamination
• Leakage by Competition: Data contamination rates by competition
• IMO Leakage by Year: Leakage detection trends from 1959 to 2025
• Leakage World Map: Contamination by language region
• IMO Leakage by Language (55 Languages): Stacked bars by language
• Average IMO Scores by Solver and Grader: Cross-model grading validation
• IMO Leakage by Predictor and Analyzer: Cross-model leakage detection