Authors
Sidi Lu, Tao Meng, and Nanyun Peng
Overview
InsNet is an insertion-based text generator designed to make non-left-to-right generation practical to train and flexible to decode. Its insertion-oriented positional encoding lets the model reuse one context encoding across an entire insertion sequence instead of re-encoding after every operation.
The model can switch between parallel decoding for highly parallelizable tasks and sequential decoding when output constraints require more precise control.
Results
Experiments cover two unsupervised lexically constrained generation datasets and three machine-translation datasets. InsNet improves training speed, inference efficiency, and generation quality over earlier insertion-based approaches.
Paper
Status
NeurIPS 2022 Poster.
Citation
Sidi Lu, Tao Meng, and Nanyun Peng. “InsNet: An Efficient, Flexible, and Performant Insertion-based Text Generation Model.” Advances in Neural Information Processing Systems 35, 2022.