Agentic data science with the Data Agent Kit
cloud.google.com·6h ago
AI Research
arXiv CS.AI·Sep 12·5 min read
Off-Target Effects of Response-Style Alignment in a Korean 27B Language Model
TL;DR
A Korean 27B language model was post-trained to align with specific response styles, revealing unexpected behaviors in answering ambiguous questions and disclosing information. The findings indicate that response style significantly influences the model's answer propensity.
✦ Why It Matters
Consider adjusting response styles in language models to improve their handling of ambiguous queries and disclosures.
Key Takeaways
- Post-train models with specific response styles for better alignment.
- Measure unexpected behaviors to understand model responses.
- Control for variables to isolate the impact of training targets.
- Explore the influence of response styles on answer propensity.
Full Summary
A Korean language model, Qwen3.8-27B, was post-trained to adopt specific response styles, including verbosity and markdown usage. Researchers measured two unexpected behaviors: the model's tendency to abstain from answering ambiguous social questions and its unprompted disclosure in financial guidance.
By controlling for various factors, they found that the model's answer propensity was influenced by the training target rather than the prompts alone. Three response style seeds led to a positive increase in answer rates, while three neutral seeds resulted in a decrease, with a notable difference of 2.34 percentage points.
The study also highlighted that changes in answer propensity dominated the model's behavior, suggesting that response style alignment can have significant off-target effects.