AIMS Collaboratory

DiagramIR: An Automatic Pipeline for Educational Math Diagram Evaluation

Academic Paper

Academic article introducing DiagramIR, an automatic pipeline for evaluating educational math diagrams.

Resource links lead to an external website.

PURPOSE / ABSTRACT

Large Language Models (LLMs) are increasingly being adopted as tools for learning; however, most tools remain text-only, limiting their usefulness for domains where visualizations are essential, such as mathematics.

Recent work shows that LLMs are capable of generating code that compiles to educational figures, but a major bottleneck remains: scalable evaluation of these diagrams. The authors address this by proposing DiagramIR: an automatic and scalable evaluation pipeline for geometric figures.

The method relies on intermediate representations (IRs) of LaTeX TikZ code. The authors compare the pipeline to other evaluation baselines such as LLM-as-a-Judge, showing that their approach has higher agreement with human raters.

This evaluation approach also enables smaller models like GPT-4.1-Mini to perform comparably to larger models such as GPT-5 at a 10x lower inference cost, which is important for deploying accessible and scalable education technologies.

CITATION

Kumar, V., Mishra, S., Hao, R. L., Malik, R., Broman, D., & Demszky, D. (2025). DiagramIR: An automatic pipeline for educational math diagram evaluation. ArXiv, abs/2511.08283.

PROJECT TEAM / LEARNING PARTNER

Stanford-Language CoPilots; Teaching Lab

WHO THIS IS FOR

Researchers, Developers

GRADES & AREAS RESEARCHED

AI; Platform/Program; All Grades

AIMS Collaboratory

Advancing mathematics education through research, collaboration, and shared public goods.