Sidi Lu / Research
Research
My long-term research interest lies in measure-theoretic questions about language, especially how measures over language shape model intelligence, generalization, and training efficiency in reinforcement learning. My work on non-autoregressive modeling and post-training follows this same line of inquiry.
Additional Tencent AI Lab collaborations
- Save the Good Prefix: Precise Error Penalization via Process-Supervised RL to Enhance LLM Reasoning Findings of ACL 2026
- Too Correct to Learn: Reinforcement Learning on Saturated Reasoning Data ACL 2026 · Short Papers
-
Turbo Connection: Reasoning as Information Flow from Higher to Lower Layers
Turbo Connection routes higher-layer states at one token to lower layers at the next, extending the effective computational path available for multi-step reasoning.
Read entry ↗ -
Locas: Your Models are Principled Initializers of Locally-Supported Parametric Memories
Locas introduces locally-supported parametric memories that can be attached to a language model for continual learning, then offloaded or merged into its feed-forward parameters.
Read entry ↗ -
G2RL: Can LLMs Guide Their Own Exploration?
G2RL guides reinforcement-learning exploration with the model's own first-order update geometry, rewarding trajectories that contribute useful new gradient directions.
Read entry ↗ -
InsNeXt: Training Scalable Insertion-based Language Models from Scratch
InsNeXt modernizes insertion-based language modeling for scalable pretraining, reaching 573M parameters with a 4,096-token context window.
Read entry ↗ -
DiNADO: Norm-Disentangled Neurally-Decomposed Oracles for Controlling Language Models
DiNADO stabilizes and expands NADO by disentangling the step-wise global norm of oracle values, enabling higher-capacity control through methods such as LoRA.
Read entry ↗ -
Open-Domain Text Evaluation via Contrastive Distribution Methods
CDM evaluates open-domain generation by contrasting the predictive distributions of stronger and weaker language models, either to synthesize training data or to score text directly.
Read entry ↗ -
Controllable Text Generation with Neurally-Decomposed Oracle
NADO decomposes a sequence-level control oracle into token-level guidance, steering a frozen autoregressive model without requiring additional labeled data.
Read entry ↗ -
InsNet: An Efficient, Flexible, and Performant Insertion-based Text Generation Model
InsNet introduces insertion-oriented positional encoding that shares context computation across insertion steps and supports both parallel and sequential decoding.
Read entry ↗ -
Neurally-Guided Structure Inference
NG-SI combines neural guidance with hierarchical search, retaining the generality of search-based structure inference while making it substantially more efficient.
Read entry ↗ -
CoT: Cooperative Training for Generative Modeling of Discrete Data
CoT replaces adversarial min–max training with a cooperative objective that directly estimates and optimizes Jensen–Shannon divergence for discrete sequence generation.
Read entry ↗ -
Neural Text Generation: Past, Present and Beyond
A systematic survey and empirical comparison of early neural text-generation approaches based on maximum likelihood, reinforcement learning, reparameterization, and adversarial training.
Read entry ↗ -
Texygen: A Benchmarking Platform for Text Generation Models
Texygen provides a shared open-source platform for comparing text-generation models across complementary measures of quality, diversity, and consistency.
Read entry ↗ -
Long Text Generation via Adversarial Training with Leaked Information
LeakGAN exposes discriminator features to a hierarchical Manager–Worker generator, providing intermediate guidance for adversarial long-text generation.
Read entry ↗