publications

2026

1. 

   On-Policy Residual Learning for Diffusion Language Models

   Yifan* Yu, Yuqing* Jian, Jisen* Li, and 9 more authors

   arXiv preprint arXiv:2610.20687, 2026

   Abs arXiv Code Website

   Residual context is key to unlocking the speed-quality Pareto frontier of diffusion language models (DLMs) by carrying forward predictive distributions across denoising steps. Yet turning this seemingly simple mechanism into an effective learning paradigm is far from simple: there is no clear recipe for how residual context should be learned. We introduce ORL, a framework for on-policy residual learning in DLMs. Algorithmically, we identify that residual learning is inherently on-policy. ORL learns from model-induced residual trajectories on the fly and introduces an entropy-contraction objective that sharpens predictions as the model revisits its own predictions. System-wise, exact residual supervision requires capturing, retaining, and reprocessing many trajectory-dependent states. ORL co-designs the algorithm with asynchronous residual capture and transport, specialized block-diffusion attention, and packed compute-balanced training, making it practical at scale. On a 4B DLM converted from Qwen3-4B, ORL improves tokens per forward (TPF) by up to 6.2x and, with our dlmrt engine, end-to-end throughput by approximately 6.9x at comparable or better quality. In single-request serving on one H100, this mode achieves a stronger speed-quality Pareto frontier than autoregressive models with EAGLE3 and DFlash speculative decoding under both strict and relaxed verification, running 1.74x faster than DFlash at matched accuracy. ORL also transfers to existing DLMs: on LLaDA-2.1-mini, 36 H100-hours and 4.2M self-generated tokens, about 10x less data than prior LLaDA recipes, yield up to 1.5x higher TPF and up to 3x higher throughput at comparable quality, and on DiffusionGemma, ORL raises TPF by 1.1-1.2x. With dlmrt and NVFP4 quantization, ORL models decode a single request at up to 5,046 tokens/s on one B200, 7,708 on eight B200s, and 2,788 on a consumer RTX 5090, with comparable quality.

2. 

   QATFactory: A Versatile, Deployment-Aligned Framework for Quantization-aware Training and Distillation of LLMs

   Weili* Xu, Jisen* Li, Yuqing* Jian, and 8 more authors

   2026

   Abs HTML Code

   QATFactory is an open-source framework for deployment-aligned quantization-aware distillation and reinforcement learning. It simulates deployment-time quantization while performing matrix multiplications in BF16, so models can adapt to quantization noise without training hardware that natively supports the target format. The framework supports NVFP4, MXFP4, and llama.cpp’s Q4_K, dense and mixture-of-experts models, and both full-parameter and LoRA training, and exports checkpoints directly to vLLM and llama.cpp. Across models from 8B to 230B parameters, quantization-aware distillation consistently improves deployed-model quality over strong post-training quantization baselines.

3. 

   Osprey: Target-agnostic Pre-training Makes Stronger Drafters in Speculative Decoding

   Fengxiang* Bie, Yuqing* Jian, Yifan Yu, and 7 more authors

   In Conference on Empirical Methods in Natural Language Processing, 2026

   Main Conference Abs HTML Code

   Accepted to the EMNLP 2026 Main Conference (2,719 of 17,669 submissions; 15.4 percent acceptance rate)

   Speculative decoding is critical for accelerating LLM inference, but drafters are typically trained for a narrow distribution and a single target model, causing acceptance rates to collapse under workload shifts. Osprey bootstraps drafters from off-the-shelf pretrained small language models, treating broad pretraining as a reusable, target-agnostic asset and reducing per-target work to lightweight adaptation. It prunes to a shallow backbone, restores language-modeling capability through target-agnostic next-token pretraining, and adapts to each target through vocabulary alignment, zero-initialized QKV expansion, and output-distribution distillation. A single pretrained Osprey backbone transfers across targets and improves mean acceptance length by 16.1% for Qwen3-8B, 21.2% for Llama-3.3-70B-Instruct, and 22.7% for MiniMax-M2.5.

4. 

   Introspective Diffusion Language Models

   Yifan* Yu, Yuqing* Jian, Junxiong Wang, and 12 more authors

   In Conference on Language Modeling, 2026

   Oral Presentation Abs HTML

   About 0.8 percent of the submissions are selected as oral presentation

   Diffusion language models (DLMs) offer a compelling promise: parallel token generation could break the sequential bottleneck of autoregressive (AR) decoding. Yet in practice, DLMs consistently lag behind AR models in quality. We argue that this gap stems from a fundamental failure of introspective consistency: AR models agree with what they generate, whereas DLMs often do not. We introduce the Introspective Diffusion Language Model (I-DLM), which uses introspective strided decoding (ISD) to verify previously generated tokens while advancing new ones in the same forward pass. Empirically, I-DLM-8B is the first DLM to match the quality of its same-scale AR counterpart, outperforming LLaDA-2.1-mini (16B) by +26 on AIME-24 and +15 on LiveCodeBench-v6 with half the parameters, while delivering 2.9-4.1x throughput at high concurrency. With gated LoRA, ISD enables bit-for-bit lossless acceleration.

5. 

   Multi-Turn Agentic Scientific Literature Search via Workflow Induction

   Jisen Li, Bingxuan Li, Nanyi Jiang, and 10 more authors

   2026

   HTML

6. 

   Squeeze Evolve: Unified Multi-Model Orchestration for Verifier-Free Evolution

   Monishwaran* Maheswaran, Leon* Lakhani, Zhongzhu Zhou, and 16 more authors

   In Conference on Language Modeling• Now part of NVIDIA Dynamo , 2026

   arXiv Code NVIDIA Dynamo Website

7. 

   Foundry: Host-Owned Trust and Memory for Long-Horizon Agent Swarms

   Monishwaran* Maheswaran, Leon* Lakhani, Shu Liu, and 7 more authors

   In 3rd AI for Math Workshop at ICML, 2026

   PDF OpenReview

2022

1. 

   MICW: A Multi-Instrument Music Generation Model Based on the Improved Compound Word

   Yikai Liao, Wang Yue, Yuqing Jian, and 3 more authors

   In 2022 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), Jul 2022

   DOI