Authors
Sidi Lu, Yaoming Zhu, Weinan Zhang, Jun Wang, and Yong Yu
Overview
This survey organizes the development of neural text generation from recurrent language models trained with maximum likelihood to approaches based on reinforcement learning, reparameterization, and generative adversarial networks.
It compares how these model families address recurring problems such as exposure bias, unstable gradients, and limited output diversity.
Results
Alongside the survey, the paper benchmarks representative neural generation models on two datasets. The empirical comparison connects their observed behavior with the optimization and diversity issues discussed in the technical review.
Paper
Status
Preprint, 2018.
Citation
Sidi Lu, Yaoming Zhu, Weinan Zhang, Jun Wang, and Yong Yu. “Neural Text Generation: Past, Present and Beyond.” arXiv:1803.07133, 2018.