A second training stage - applied after supervised fine-tuning (SFT), on the same documents, using the same model - reduced text degeneration in every family tested. No exceptions. Average reduction: 59.4%. Best case: 87.6%.

Figure 1: DPO reduced degeneration relative to SFT in every family tested - average reduction of 59.4%, peak of 87.6% (Nanonets-OCR2–3B: 1.61% to 0.20%). The direction is invariant; only the magnitude varies.

That second stage was Direct Preference Optimization (DPO). Almost all published DPO applications target chat alignment - models trained on human judgments about helpfulness or harmlessness (example: Rafailov et al., 2023). OCR carries none of that subjectivity: the task is objective, and there is no conversational context. There is, however, a clear preference signal. A correct transcription is chosen; a degeneration loop is rejected. DharmaOCR used that binary to construct a DPO training set, testing the technique not for alignment, but as a direct mitigation tool for a specific failure mode.

The training signal came from the model itself - specifically from the outputs it produced when it failed. How a failure mode becomes a training signal is a structural question about the failure, not the model.