REASONING LLM PIPELINE

An end-to-end research pipeline that converts Large Language Model (LLM) reasoning traces into a Reasoning Language Model (RLM) dataset for verifier model training. The project extracts reasoning steps from public datasets, generates corrupted reasoning examples, constructs a balanced verification dataset, and trains a sequence classification model to distinguish between correct and incorrect reasoning steps. The processed dataset can then be exported for downstream reasoning and verification research.

Technologies: Python

Industry: Academics

Challenge

Training reasoning verification models requires datasets containing both correct and incorrect reasoning steps. Since public reasoning datasets primarily contain valid solutions, the challenge was to automatically generate realistic negative examples while preserving the reasoning context.

The project addresses this by systematically corrupting reasoning steps and constructing balanced datasets suitable for verifier model training.

Complexity & Innovation

The pipeline transforms raw chain-of-thought reasoning into a structured verification dataset through automated reasoning extraction, numerical corruption, and logical error generation. Instead of collecting manually labeled data, it programmatically creates balanced positive and negative samples before training a DistilBERT sequence classifier to evaluate reasoning plausibility.
The modular notebook-based workflow also supports downstream research on Reasoning Language Models and verifier-guided reasoning.

Process

Load a reasoning dataset and extract individual reasoning steps.
Generate corrupted numerical and logical reasoning examples.
Build a labeled verification dataset.
Train a DistilBERT verifier model.
Evaluate the trained model and export the processed dataset.

Feature Inventory

Automated Reasoning Dataset Generation: The pipeline extracts individual reasoning steps from public reasoning datasets and automatically constructs labeled training samples containing the original question, previous reasoning context, current reasoning step, and verification label. This creates a structured dataset suitable for reasoning verification tasks.
Reasoning Corruption Engine: To generate negative training examples, the system introduces both numerical calculation errors and logical reasoning mistakes into otherwise correct solutions. Every original reasoning step produces one positive sample and multiple negative samples, resulting in a balanced dataset without requiring manual annotation.
Verifier Model Training: The project fine-tunes a DistilBERT sequence classification model to distinguish between correct and corrupted reasoning steps. After tokenization and dataset splitting, the model is trained and evaluated using the Hugging Face Trainer, producing a verifier checkpoint that can be reused for downstream reasoning research.

Conclusion

Provides a complete research pipeline for transforming LLM-generated reasoning into datasets suitable for reasoning verification. By combining automated reasoning extraction, synthetic error generation, and verifier model training, the project supports experimentation with Reasoning Language Models and model-based verification of chain-of-thought reasoning.