Label-Free Steering: Compressing Test-Time Reinforcement Learning into Bias-Only Subspaces
Hybrid Seminar Session
In-person attendance is highly encouraged. Remote participants can join via Zoom (Meeting ID: 667 2398 3300). Contact: HAIF@utu.fi
Abstract
Test-time reinforcement learning (TTRL) lets models improve their reasoning without labelled data, but usually updates a large share of the model’s parameters. This study asks whether adaptation still works when both the reward and the trainable parameters are severely restricted. Label-free bias-only TTRL rewards the model with majority-vote pseudo-labels and tunes only ~100K bias parameters, keeping the pretrained backbone frozen. With Qwen2.5-7B it reaches 76.67% accuracy on MATH-500 while training 76,000× fewer parameters than full-parameter TTRL, also improves vision-language and audio reasoning, and transfers to 4,500 unseen MATH problems.
Project page: Label-Free Steering
About the Speaker
Naveen Vakada is a Doctoral Researcher at the Department of Computing, University of Turku. Naveen’s research develops capable, efficient, and adaptable multimodal AI systems. The work focuses on understanding and controlling the internal representations of multimodal language models through activation steering and efficient test-time adaptation, improving reasoning, personalisation, and reliability without full-model training. The goal is practical methods for human-centred applications, including health and well-being.