Agent-Trace Fine-Tuning of Small Language Models under Constrained Compute

Abstract

Naively fine-tuning small language models on agent traces regresses general coding ability by roughly 19 points, and the regression is invisible unless you measure for it. A recipe of completion masking, mixed-domain replay and measured decontamination recovers about half of the lost capability. We name and eliminate session leakage, a failure mode in which transcript artifacts from the training corpus surface in model outputs. Six of our own headline results turned out to be artifacts of the apparatus or our process, each caught by re-measurement against a same-instrument baseline; the paper reports all of them. We argue that at this scale, evaluation discipline matters more than compute.

Read

Models this paper describes

• ibm-granite/granite-4.1-3b fine-tuned on Claude Fable 5 agent traces (GGUF)
• Qwen/Qwen3-4B fine-tuned on Claude Fable 5 agent traces (GGUF)
• HuggingFaceTB/SmolLM3-3B fine-tuned on Claude Fable 5 agent traces (GGUF)

Cite

@misc{aglawe2026agenttrace,
  author    = {Aglawe, Ankit},
  title     = {Agent-Trace Fine-Tuning of Small Language Models
               under Constrained Compute},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.21676407},
  url       = {https://doi.org/10.5281/zenodo.21676407},
}

All papers