Authors
Tao Meng, Sidi Lu, Nanyun Peng, and Kai-Wei Chang
Overview
NeurAlly-Decomposed Oracle, or NADO, turns a sequence-level boolean control function into token-level guidance for an autoregressive language model. The guiding model is trained on samples from the base model and does not require an additional labeled dataset.
A posterior-regularization formulation provides a closed-form way to combine the decomposed oracle with the base distribution while preserving the base model’s generation quality.
Results
The framework is evaluated on lexically constrained text generation and formality-controlled machine translation. Across both settings, it moves generation toward the requested control condition while maintaining output quality.
Paper
Status
NeurIPS 2022 Highlighted Paper (oral-equivalent).
Citation
Tao Meng, Sidi Lu, Nanyun Peng, and Kai-Wei Chang. “Controllable Text Generation with Neurally-Decomposed Oracle.” Advances in Neural Information Processing Systems 35, 2022.