Authors
Sidi Lu, Wenbo Zhao, Chenyang Tao, Arpit Gupta, Shanchan Wu, Tagyoung Chung, and Nanyun Peng
Overview
DiNADO revisits the parameterization and training of NeurAlly-Decomposed Oracles. Vanilla NADO can suffer from vanishing gradients under low-probability control signals, depend heavily on consistency regularization, and have limited capacity when implemented with a small number of added Transformer layers.
The method disentangles the step-wise global norm of the predicted oracle values from their relative token-level structure. This improves training stability and allows the control mechanism to be combined naturally with higher-capacity adaptation methods such as LoRA.
Results
Experiments on formality-controlled machine translation and CommonGen lexically constrained generation demonstrate improved capacity, stability, and flexibility over the original NADO formulation.
Paper
Status
ICML 2024 Poster.
Citation
Sidi Lu, Wenbo Zhao, Chenyang Tao, Arpit Gupta, Shanchan Wu, Tagyoung Chung, and Nanyun Peng. “DiNADO: Norm-Disentangled Neurally-Decomposed Oracles for Controlling Language Models.” Proceedings of the 41st International Conference on Machine Learning, PMLR 235:33243–33253, 2024.