Vision-Language • Document AI
Handwritten mathematics → compilable LaTeX PDFs
Multimodal LoRA fine-tunes of GLM-OCR and Baidu OCR models to transform unconstrained handwritten mathematics into complete, standalone compilable LaTeX PDFs.
- 0% → 82.4% compile rate (88.9% on v3.1)
- 0.382 mean CER — 25.9% relative cut vs base
- 13,973 validated pairs from 34,080 raw scans
- GGUF models verified on Hugging Face
Fine-tunes GLM-OCR (0.9B) and Baidu OCR via LoRA on a single RTX 3060 12GB to transcribe
unconstrained handwritten university-level mathematics into complete, pdflatex-compilable
LaTeX documents. Takes the compile rate on the 700-page held-out test benchmark from 0.0% (frozen base)
to 82.4% (v4.1) / 88.9% (v3.1) while lifting Math-F1 to 0.827.
Curated a 34,080-scan pipeline yielding 13,973 validated image–LaTeX pairs, with every training target
confirmed to compile before training. Solved autoregressive preamble failures by fine-tuning on
document bodies and emitting compilable standalone LaTeX.
Evaluated on a 700-page held-out test split across 15 logged metrics. Pre-quantized GGUF models
built and verified on GPU for private, offline local inference via llama.cpp.
Models on Hugging Face ↗
Code ↗