Yeyun Gong

Principal Research Manager

论文与出版物

查看方式：

- Beyond Static Dialogues: Benchmarking Realistic, Heterogeneous, and Evolving Long-Term Memory
  
  Han Zhang, Zihao Tang, Xinyan Velocity Yu, Xiao Liu, Yeyun Gong, Haizhen Huang, Yan Lu, Weiwei Deng, Feng Sun, Qi Zhang, Hanfang Yang
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- Pull Requests as a Training Signal for Repo-Level Code Editing
  
  Qinglin Zhu, Tianyu Chen, Shuai Lu, Lei Ji, Runcong Zhao, Murong Ma, Xiangxiang Dai, Yulan He, Lin Gui, Peng Cheng, Yeyun Gong
  
  ICML 2026 | May 2026
  
  Publication
- From Patches to Trajectories: Privileged Process Supervision for Software-Engineering Agents
  
  Murong Ma, Tianyu Chen, Yun Lin, Shuai Lu, Qinglin Zhu, Yeyun Gong, Zhiyong Huang, Peng Cheng, Yan Lu, Jin Song Dong
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- Memory Grafting: Scaling Language Model Pre-training via Offline Conditional Memory
  
  Runxi Cheng, Yuchen Guan, Yongxian Wei, Qianpu Sun, Qixiu Li, Sinan Du, Feng Xiong, Chun Yuan, Yan Lu, Yeyun Gong
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- m3BERT: A Modern, Multi-lingual, Matryoshka Bidirectional Encoder
  
  Yaoxiang Wang, Simiao Zuo, Qingguo Hu, Yucheng Ding, Yeyun Gong, Jian Jiao, Jinsong Su
  
  KDD 2026 | May 2026
  
  Publication
- Beyond Sunk Costs: Boosting LLM Pre-training Efficiency via Orthogonal Growth of Mixture-of-Experts
  
  Ruizhe Wang, Yucheng Ding, Xiao Liu, Yaoxiang Wang, Peng Cheng, Baining Guo, Zhengjun Zha, Yeyun Gong
  
  ICML 2026 | May 2026
  
  Publication
- Reducing the Costs of Proof Synthesis on Rust Systems by Scaling Up a Seed Training Set
  
  Nongyu Di, Tianyu Chen, Shan Lu, Shuai Lu, Yeyun Gong, Peng Cheng, Jay Lorch, Yuan Yao, Xiaoxing Ma
  
  February 2026
  
  February 2026
  
  Publication 项目
- AutoVerus: Automated Proof Generation for Rust Code
  
  Chenyuan Yang, Xuheng Li, Md Rakib Hossain Misu, Jianan Yao, Weidong Cui, Yeyun Gong, Chris Hawblitzel, Shuvendu Lahiri, Jay Lorch, Shuai Lu, Fan Yang, Ziqiao Zhou, Shan Lu
  
  Object-Oriented Programming, Systems, Languages & Applications (OOPSLA) | October 2025
  
  Distinguished Artifact Award
  
  Publication Github 项目
- LayerNorm Induces Recency Bias in Transformer Decoders
  
  Junu Kim, Xiao Liu, Zheng-Wen Lin, Lei Ji, Yeyun Gong, Edward Choi
  
  September 2025
  
  arXiv | September 2025
  
  Publication
- Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVR
  
  Xiao Liang, Zhong-zhi Li, Yeyun Gong, Yelong Shen, Yingchun Wu, Zhijiang Guo, Weizhu Chen
  
  ICLR 2026 | August 2025
  
  DOI Publication
- DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collections
  
  Haebin Shin, Lei Ji, Xiao Liu, Zhiwei Yu, Qi Chen, Yeyun Gong
  
  August 2025
  
  arXiv | August 2025
  
  DOI Publication
- Velocitune: A Velocity-based Dynamic Domain Reweighting Method for Continual Pre-training
  
  Zheheng Luo, Xin Zhang, Xiao Liu, Haoling Li, Yeyun Gong, Qi Chen, Peng Cheng
  
  ACL | August 2025
  
  DOI Publication Publication
- Routing Mamba: Scaling State Space Models with Mixture-of-Experts Projection
  
  Zheng Zhan, Liliang Ren, Shuohang Wang, Liyuan Liu, Yang Liu, Yeyun Gong, Yanzhi Wang, Yelong Shen
  
  NeurIPS 2025 | June 2025
  
  DOI Publication Publication
- PeRL: Permutation-Enhanced Reinforcement Learning for Interleaved Vision-Language Reasoning
  
  Yizhen Zhang, Yang Ding, Shuoshuo Zhang, Xinchen Zhang, Haoling Li, Zhong-zhi Li, Peijie Wang, Jie Wu, Lei Ji, Yelong Shen, Yujiu Yang, Yeyun Gong
  
  NeurIPS 2025 | June 2025
  
  DOI Publication Publication
- SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoning
  
  Xiao Liang, Zhong-zhi Li, Yeyun Gong, Yang Wang, Hengyuan Zhang, Yelong Shen, Yingchun Wu, Weizhu Chen
  
  NeurIPS 2025 | June 2025
  
  DOI Publication Publication
- Enhancing Large Language Model Performance with Gradient-Based Parameter Selection
  
  Haoling Li, Xin Zhang, Xiao Liu, Yeyun Gong, Yifan Wang, Qi Chen, Peng Cheng
  
  AAAI Conference on Artificial Intelligence, 2025 | April 2025
  
  Oral Presentation
  
  DOI Publication
- Overcoming Vocabulary Mismatch: Vocabulary-agnostic Teacher Guided Language Modeling
  
  Haebin Shin, Lei Ji, Xiao Liu, Yeyun Gong
  
  ICML 2025 | March 2025
  
  DOI Publication Publication
- Optimizing Large Language Model Training Using FP4 Quantization
  
  Ruizhe Wang, Yeyun Gong, Xiao Liu, Guoshuai Zhao, Ziyue Yang, Baining Guo, Zhengjun Zha, Peng Cheng
  
  ICML 2025 | January 2025
  
  DOI Publication Publication
- Integrative Decoding: Improve Factuality via Implicit Self-consistency
  
  Yi Cheng, Xiao Liang, Yeyun Gong, Wen Xiao, Song Wang, Yuji Zhang, Wenjun Hou, Kaishuai Xu, Wenge Liu, Wenjie Li, Jian Jiao, Qi Chen, Peng Cheng, Wayne Xiong
  
  ICLR 2025 | January 2025
  
  DOI Publication Publication
- Automated Proof Generation for Rust Code via Self-Evolution
  
  Tianyu Chen, Shuai Lu, Shan Lu, Yeyun Gong, Chenyuan Yang, Xuheng Li, Md Rakib Hossain Misu, Hao Yu, Nan Duan, Peng Cheng, Fan Yang, Shuvendu Lahiri, Tao Xie, Lidong Zhou
  
  ICLR 2025 | October 2024
  
  Publication 项目
- Alchemy: Amplifying Theorem-Proving Capability through Symbolic Mutation
  
  Shaonan Wu, Shuai Lu, Yeyun Gong, Nan Duan, Ping Wei
  
  October 2024
  
  October 2024
  
  Publication
- Not All Tokens Are What You Need for Pretraining
  
  Yeyun Gong, Xiao Liu, Yelong Shen, Ruochen Xu, Jian Jiao, Nan Duan, Weizhu Chen
  
  Neural Information Processing Systems | October 2024
  
  Best Paper Runner Up
  
  Publication
- MS MARCO Web Search: A Large-scale Information-rich Web Dataset with Millions of Real Click Labels
  
  Qi Chen, Xiubo Geng, Corby Rosset, Carolyn Buractaon, Jingwen Lu, Tao Shen, Kun Zhou, Chenyan Xiong, Yeyun Gong, Paul Bennett, Nick Craswell, Xing Xie, Fan Yang, Bryan Tower, Zheng Liu, Mingqin Li, Chuanjie Liu, Jason (Zengzhong) Li, Rangan Majumder, Jennifer Neville, Harsha Simhadri, Manik Varma, Yujing Wang, Linjun Yang, Mao Yang, Ce Zhang
  
  The Web Conference | May 2024
  
  Publication PDF 访问
- Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoning
  
  Yiming Huang, Xiao Liu, Yeyun Gong, Zhibin Gou, Yelong Shen, Nan Duan, Weizhu Chen
  
  March 2024
  
  arXiv | March 2024
  
  Publication
- AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation
  
  Tong Wu, Zhihao Fan, Xiao Liu, Yeyun Gong, Yelong Shen, Jian Jiao, Hai-Tao Zheng, Juntao Li, zhongyu wei, Jian Guo, Nan Duan, Weizhu Chen
  
  NeurIPS 2023 | December 2023
  
  Publication
- ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving
  
  Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Minlie Huang, Nan Duan, Weizhu Chen
  
  ICLR 2024 | September 2023
  
  DOI PDF
- Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph
  
  Jiashuo Sun, Chengjin Xu, Lumingyuan Tang, Sai Wang, Chen Lin, Yeyun Gong, Lionel M. Ni, H. Shum, Jian Guo
  
  ICLR 2024 | July 2023
  
  Publication
- CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
  
  Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, Weizhu Chen
  
  ICLR 2024 | May 2023
  
  DOI Publication
- CodeRetriever: A Large Scale Contrastive Pre-Training Method for Code Search.
  
  Xiaonan Li, Yeyun Gong, Yelong Shen, Xipeng Qiu, Hang Zhang, Bolun Yao, Weizhen Qi, Daxin Jiang (姜大昕), Weizhu Chen, Nan Duan
  
  EMNLP 2022 | October 2022
  
  Publication
- DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation
  
  Wei Chen, Yeyun Gong, Song Wang, Bolun Yao, Weizhen Qi, Zhongyu Wei, Xiaowu Hu, Bartuer Zhou, Yi Mao, Weizhu Chen, Biao Cheng, Nan Duan
  
  ACL 2022 | May 2022
  
  Publication
- Distill-VQ: Learning Retrieval Oriented Vector Quantization by Distilling Knowledge from Dense Embeddings
  
  Shitao Xiao, Zheng Liu, Weihao Han, Jianjin Zhang, Defu Lian, Yeyun Gong, Qi Chen, Fan Yang, Hao Sun, Yingxia Shao, Xing Xie
  
  SIGIR 2022 | April 2022
  
  Publication
- Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversations
  
  Wei Chen, Yeyun Gong, Can Xu, Huang Hu, Bolun Yao, Zhongyu Wei, Zhihao Fan, Xiaowu Hu, Bartuer Zhou, Biao Cheng, Daxin Jiang (姜大昕), Nan Duan
  
  ACL 2022 | April 2022
  
  Publication
- Adversarial Retriever-Ranker for dense text retrieval
  
  Hang Zhang, Yeyun Gong, Yelong Shen, Jiancheng Lv, Nan Duan, Weizhu Chen
  
  ICLR 2022 | April 2022
  
  Publication Publication
- KFCNet: Knowledge Filtering and Contrastive Learning Network for Generative Commonsense Reasoning
  
  Haonan Li, Yeyun Gong, Jian Jiao, Ruofei Zhang, Timothy Baldwin, Nan Duan
  
  Empirical Methods in Natural Language Processing | September 2021
  
  Publication
- FastSeq: Make Sequence Generation Faster
  
  Yu Yan, Fei Hu, Jiusheng Chen, Nikhil Bhendawade, Ting Ye, Yeyun Gong, Nan Duan, Desheng Cui, Bingyu Chi, Ruofei Zhang
  
  Meeting of the Association for Computational Linguistics | August 2021
  
  Publication Github
- ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation
  
  Weizhen Qi, Yeyun Gong, Yu Yan, Can Xu, Bolun Yao, Bartuer Zhou, Biao Cheng, Daxin Jiang (姜大昕), Jiusheng Chen, Ruofei Zhang, Houqiang Li, Nan Duan
  
  Meeting of the Association for Computational Linguistics | August 2021
  
  Publication Github
- EL-Attention: Memory Efficient Lossless Attention for Generation
  
  Yu Yan, Jiusheng Chen, Weizhen Qi, Nikhil Bhendawade, Yeyun Gong, Nan Duan, Ruofei Zhang
  
  2021 International Conference on Machine Learning | July 2021
  
  Publication
- Poolingformer: Long Document Modeling with Pooling Attention
  
  Hang Zhang, Yeyun Gong, Yelong Shen, Weisheng Li, Jiancheng Lv, Nan Duan, Wei Chen
  
  2021 International Conference on Machine Learning | July 2021
  
  Publication Publication
- BANG: Bridging Autoregressive and Non-autoregressive Generation with Large Scale Pretraining
  
  Weizhen Qi, Yeyun Gong, Jian Jiao, Yu Yan, Dayiheng Liu, Wei Chen, Kewen Tang, Houqiang Li, Jiusheng Chen, Ruofei Zhang, Ming Zhou, Nan Duan
  
  2021 International Conference on Machine Learning | December 2020
  
  Publication
- Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation
  
  Dayiheng Liu, Yeyun Gong, Yu Yan, Jie Fu, Bo Shao, Daxin Jiang (姜大昕), Jiancheng Lv,, Nan Duan
  
  Empirical Methods in Natural Language Processing | November 2020
  
  Publication
- Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space
  
  Dayiheng Liu, Yeyun Gong, Jie Fu, Yu Yan, Jiusheng Chen, Jiancheng Lv, Nan Duan, Ming Zhou
  
  Empirical Methods in Natural Language Processing | November 2020
  
  Publication
- ProphetNet-Ads: A Looking Ahead Strategy for Generative Retrieval Models in Sponsored Search Engine
  
  Weizhen Qi, Yeyun Gong, Yu Yan, Jian Jiao, Bo Shao, Ruofei Zhang, Houqiang Li, Nan Duan, Ming Zhou
  
  The 9th CCF International Conference on Natural Language Processing and Chinese Computing(NLPCC2020) | October 2020
  
  Publication Github
- ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training
  
  Weizhen Qi, Yu Yan, Yeyun Gong, Dayiheng Liu, Nan Duan, Jiusheng Chen, Ruofei Zhang, Ming Zhou
  
  EMNLP 2020 | October 2020
  
  Publication Github
- RikiNet: Reading Wikipedia Pages for Natural Question Answering
  
  Dayiheng Liu, Yeyun Gong, Jie Fu, Yu Yan, Jiusheng Chen, Daxin Jiang (姜大昕), Jiancheng Lv, Nan Duan
  
  Meeting of the Association for Computational Linguistics | July 2020
  
  Publication

- Beyond Static Dialogues: Benchmarking Realistic, Heterogeneous, and Evolving Long-Term Memory
  
  Han Zhang, Zihao Tang, Xinyan Velocity Yu, Xiao Liu, Yeyun Gong, Haizhen Huang, Yan Lu, Weiwei Deng, Feng Sun, Qi Zhang, Hanfang Yang
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- Pull Requests as a Training Signal for Repo-Level Code Editing
  
  Qinglin Zhu, Tianyu Chen, Shuai Lu, Lei Ji, Runcong Zhao, Murong Ma, Xiangxiang Dai, Yulan He, Lin Gui, Peng Cheng, Yeyun Gong
  
  ICML 2026 | May 2026
  
  Publication
- From Patches to Trajectories: Privileged Process Supervision for Software-Engineering Agents
  
  Murong Ma, Tianyu Chen, Yun Lin, Shuai Lu, Qinglin Zhu, Yeyun Gong, Zhiyong Huang, Peng Cheng, Yan Lu, Jin Song Dong
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- Memory Grafting: Scaling Language Model Pre-training via Offline Conditional Memory
  
  Runxi Cheng, Yuchen Guan, Yongxian Wei, Qianpu Sun, Qixiu Li, Sinan Du, Feng Xiong, Chun Yuan, Yan Lu, Yeyun Gong
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- m3BERT: A Modern, Multi-lingual, Matryoshka Bidirectional Encoder
  
  Yaoxiang Wang, Simiao Zuo, Qingguo Hu, Yucheng Ding, Yeyun Gong, Jian Jiao, Jinsong Su
  
  KDD 2026 | May 2026
  
  Publication
- Beyond Sunk Costs: Boosting LLM Pre-training Efficiency via Orthogonal Growth of Mixture-of-Experts
  
  Ruizhe Wang, Yucheng Ding, Xiao Liu, Yaoxiang Wang, Peng Cheng, Baining Guo, Zhengjun Zha, Yeyun Gong
  
  ICML 2026 | May 2026
  
  Publication
- Reducing the Costs of Proof Synthesis on Rust Systems by Scaling Up a Seed Training Set
  
  Nongyu Di, Tianyu Chen, Shan Lu, Shuai Lu, Yeyun Gong, Peng Cheng, Jay Lorch, Yuan Yao, Xiaoxing Ma
  
  February 2026
  
  February 2026
  
  Publication 项目
- AutoVerus: Automated Proof Generation for Rust Code
  
  Chenyuan Yang, Xuheng Li, Md Rakib Hossain Misu, Jianan Yao, Weidong Cui, Yeyun Gong, Chris Hawblitzel, Shuvendu Lahiri, Jay Lorch, Shuai Lu, Fan Yang, Ziqiao Zhou, Shan Lu
  
  Object-Oriented Programming, Systems, Languages & Applications (OOPSLA) | October 2025
  
  Distinguished Artifact Award
  
  Publication Github 项目
- LayerNorm Induces Recency Bias in Transformer Decoders
  
  Junu Kim, Xiao Liu, Zheng-Wen Lin, Lei Ji, Yeyun Gong, Edward Choi
  
  September 2025
  
  arXiv | September 2025
  
  Publication
- Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVR
  
  Xiao Liang, Zhong-zhi Li, Yeyun Gong, Yelong Shen, Yingchun Wu, Zhijiang Guo, Weizhu Chen
  
  ICLR 2026 | August 2025
  
  DOI Publication
- DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collections
  
  Haebin Shin, Lei Ji, Xiao Liu, Zhiwei Yu, Qi Chen, Yeyun Gong
  
  August 2025
  
  arXiv | August 2025
  
  DOI Publication
- Velocitune: A Velocity-based Dynamic Domain Reweighting Method for Continual Pre-training
  
  Zheheng Luo, Xin Zhang, Xiao Liu, Haoling Li, Yeyun Gong, Qi Chen, Peng Cheng
  
  ACL | August 2025
  
  DOI Publication Publication
- Routing Mamba: Scaling State Space Models with Mixture-of-Experts Projection
  
  Zheng Zhan, Liliang Ren, Shuohang Wang, Liyuan Liu, Yang Liu, Yeyun Gong, Yanzhi Wang, Yelong Shen
  
  NeurIPS 2025 | June 2025
  
  DOI Publication Publication
- PeRL: Permutation-Enhanced Reinforcement Learning for Interleaved Vision-Language Reasoning
  
  Yizhen Zhang, Yang Ding, Shuoshuo Zhang, Xinchen Zhang, Haoling Li, Zhong-zhi Li, Peijie Wang, Jie Wu, Lei Ji, Yelong Shen, Yujiu Yang, Yeyun Gong
  
  NeurIPS 2025 | June 2025
  
  DOI Publication Publication
- SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoning
  
  Xiao Liang, Zhong-zhi Li, Yeyun Gong, Yang Wang, Hengyuan Zhang, Yelong Shen, Yingchun Wu, Weizhu Chen
  
  NeurIPS 2025 | June 2025
  
  DOI Publication Publication
- Enhancing Large Language Model Performance with Gradient-Based Parameter Selection
  
  Haoling Li, Xin Zhang, Xiao Liu, Yeyun Gong, Yifan Wang, Qi Chen, Peng Cheng
  
  AAAI Conference on Artificial Intelligence, 2025 | April 2025
  
  Oral Presentation
  
  DOI Publication
- Overcoming Vocabulary Mismatch: Vocabulary-agnostic Teacher Guided Language Modeling
  
  Haebin Shin, Lei Ji, Xiao Liu, Yeyun Gong
  
  ICML 2025 | March 2025
  
  DOI Publication Publication
- Optimizing Large Language Model Training Using FP4 Quantization
  
  Ruizhe Wang, Yeyun Gong, Xiao Liu, Guoshuai Zhao, Ziyue Yang, Baining Guo, Zhengjun Zha, Peng Cheng
  
  ICML 2025 | January 2025
  
  DOI Publication Publication
- Integrative Decoding: Improve Factuality via Implicit Self-consistency
  
  Yi Cheng, Xiao Liang, Yeyun Gong, Wen Xiao, Song Wang, Yuji Zhang, Wenjun Hou, Kaishuai Xu, Wenge Liu, Wenjie Li, Jian Jiao, Qi Chen, Peng Cheng, Wayne Xiong
  
  ICLR 2025 | January 2025
  
  DOI Publication Publication
- Automated Proof Generation for Rust Code via Self-Evolution
  
  Tianyu Chen, Shuai Lu, Shan Lu, Yeyun Gong, Chenyuan Yang, Xuheng Li, Md Rakib Hossain Misu, Hao Yu, Nan Duan, Peng Cheng, Fan Yang, Shuvendu Lahiri, Tao Xie, Lidong Zhou
  
  ICLR 2025 | October 2024
  
  Publication 项目
- Alchemy: Amplifying Theorem-Proving Capability through Symbolic Mutation
  
  Shaonan Wu, Shuai Lu, Yeyun Gong, Nan Duan, Ping Wei
  
  October 2024
  
  October 2024
  
  Publication
- Not All Tokens Are What You Need for Pretraining
  
  Yeyun Gong, Xiao Liu, Yelong Shen, Ruochen Xu, Jian Jiao, Nan Duan, Weizhu Chen
  
  Neural Information Processing Systems | October 2024
  
  Best Paper Runner Up
  
  Publication
- Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoning
  
  Yiming Huang, Xiao Liu, Yeyun Gong, Zhibin Gou, Yelong Shen, Nan Duan, Weizhu Chen
  
  March 2024
  
  arXiv | March 2024
  
  Publication
- AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation
  
  Tong Wu, Zhihao Fan, Xiao Liu, Yeyun Gong, Yelong Shen, Jian Jiao, Hai-Tao Zheng, Juntao Li, zhongyu wei, Jian Guo, Nan Duan, Weizhu Chen
  
  NeurIPS 2023 | December 2023
  
  Publication
- ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving
  
  Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Minlie Huang, Nan Duan, Weizhu Chen
  
  ICLR 2024 | September 2023
  
  DOI PDF
- Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph
  
  Jiashuo Sun, Chengjin Xu, Lumingyuan Tang, Sai Wang, Chen Lin, Yeyun Gong, Lionel M. Ni, H. Shum, Jian Guo
  
  ICLR 2024 | July 2023
  
  Publication
- CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
  
  Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, Weizhu Chen
  
  ICLR 2024 | May 2023
  
  DOI Publication
- CodeRetriever: A Large Scale Contrastive Pre-Training Method for Code Search.
  
  Xiaonan Li, Yeyun Gong, Yelong Shen, Xipeng Qiu, Hang Zhang, Bolun Yao, Weizhen Qi, Daxin Jiang (姜大昕), Weizhu Chen, Nan Duan
  
  EMNLP 2022 | October 2022
  
  Publication
- Distill-VQ: Learning Retrieval Oriented Vector Quantization by Distilling Knowledge from Dense Embeddings
  
  Shitao Xiao, Zheng Liu, Weihao Han, Jianjin Zhang, Defu Lian, Yeyun Gong, Qi Chen, Fan Yang, Hao Sun, Yingxia Shao, Xing Xie
  
  SIGIR 2022 | April 2022
  
  Publication
- Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversations
  
  Wei Chen, Yeyun Gong, Can Xu, Huang Hu, Bolun Yao, Zhongyu Wei, Zhihao Fan, Xiaowu Hu, Bartuer Zhou, Biao Cheng, Daxin Jiang (姜大昕), Nan Duan
  
  ACL 2022 | April 2022
  
  Publication
- Adversarial Retriever-Ranker for dense text retrieval
  
  Hang Zhang, Yeyun Gong, Yelong Shen, Jiancheng Lv, Nan Duan, Weizhu Chen
  
  ICLR 2022 | April 2022
  
  Publication Publication
- KFCNet: Knowledge Filtering and Contrastive Learning Network for Generative Commonsense Reasoning
  
  Haonan Li, Yeyun Gong, Jian Jiao, Ruofei Zhang, Timothy Baldwin, Nan Duan
  
  Empirical Methods in Natural Language Processing | September 2021
  
  Publication
- ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation
  
  Weizhen Qi, Yeyun Gong, Yu Yan, Can Xu, Bolun Yao, Bartuer Zhou, Biao Cheng, Daxin Jiang (姜大昕), Jiusheng Chen, Ruofei Zhang, Houqiang Li, Nan Duan
  
  Meeting of the Association for Computational Linguistics | August 2021
  
  Publication Github
- FastSeq: Make Sequence Generation Faster
  
  Yu Yan, Fei Hu, Jiusheng Chen, Nikhil Bhendawade, Ting Ye, Yeyun Gong, Nan Duan, Desheng Cui, Bingyu Chi, Ruofei Zhang
  
  Meeting of the Association for Computational Linguistics | August 2021
  
  Publication Github
- Poolingformer: Long Document Modeling with Pooling Attention
  
  Hang Zhang, Yeyun Gong, Yelong Shen, Weisheng Li, Jiancheng Lv, Nan Duan, Wei Chen
  
  2021 International Conference on Machine Learning | July 2021
  
  Publication Publication
- EL-Attention: Memory Efficient Lossless Attention for Generation
  
  Yu Yan, Jiusheng Chen, Weizhen Qi, Nikhil Bhendawade, Yeyun Gong, Nan Duan, Ruofei Zhang
  
  2021 International Conference on Machine Learning | July 2021
  
  Publication
- BANG: Bridging Autoregressive and Non-autoregressive Generation with Large Scale Pretraining
  
  Weizhen Qi, Yeyun Gong, Jian Jiao, Yu Yan, Dayiheng Liu, Wei Chen, Kewen Tang, Houqiang Li, Jiusheng Chen, Ruofei Zhang, Ming Zhou, Nan Duan
  
  2021 International Conference on Machine Learning | December 2020
  
  Publication
- Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation
  
  Dayiheng Liu, Yeyun Gong, Yu Yan, Jie Fu, Bo Shao, Daxin Jiang (姜大昕), Jiancheng Lv,, Nan Duan
  
  Empirical Methods in Natural Language Processing | November 2020
  
  Publication
- Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space
  
  Dayiheng Liu, Yeyun Gong, Jie Fu, Yu Yan, Jiusheng Chen, Jiancheng Lv, Nan Duan, Ming Zhou
  
  Empirical Methods in Natural Language Processing | November 2020
  
  Publication
- ProphetNet-Ads: A Looking Ahead Strategy for Generative Retrieval Models in Sponsored Search Engine
  
  Weizhen Qi, Yeyun Gong, Yu Yan, Jian Jiao, Bo Shao, Ruofei Zhang, Houqiang Li, Nan Duan, Ming Zhou
  
  The 9th CCF International Conference on Natural Language Processing and Chinese Computing(NLPCC2020) | October 2020
  
  Publication Github
- RikiNet: Reading Wikipedia Pages for Natural Question Answering
  
  Dayiheng Liu, Yeyun Gong, Jie Fu, Yu Yan, Jiusheng Chen, Daxin Jiang (姜大昕), Jiancheng Lv, Nan Duan
  
  Meeting of the Association for Computational Linguistics | July 2020
  
  Publication
- Beyond Static Dialogues: Benchmarking Realistic, Heterogeneous, and Evolving Long-Term Memory
  
  Han Zhang, Zihao Tang, Xinyan Velocity Yu, Xiao Liu, Yeyun Gong, Haizhen Huang, Yan Lu, Weiwei Deng, Feng Sun, Qi Zhang, Hanfang Yang
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- MS MARCO Web Search: A Large-scale Information-rich Web Dataset with Millions of Real Click Labels
  
  Qi Chen, Xiubo Geng, Corby Rosset, Carolyn Buractaon, Jingwen Lu, Tao Shen, Kun Zhou, Chenyan Xiong, Yeyun Gong, Paul Bennett, Nick Craswell, Xing Xie, Fan Yang, Bryan Tower, Zheng Liu, Mingqin Li, Chuanjie Liu, Jason (Zengzhong) Li, Rangan Majumder, Jennifer Neville, Harsha Simhadri, Manik Varma, Yujing Wang, Linjun Yang, Mao Yang, Ce Zhang
  
  The Web Conference | May 2024
  
  Publication PDF 访问
- Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversations
  
  Wei Chen, Yeyun Gong, Can Xu, Huang Hu, Bolun Yao, Zhongyu Wei, Zhihao Fan, Xiaowu Hu, Bartuer Zhou, Biao Cheng, Daxin Jiang (姜大昕), Nan Duan
  
  ACL 2022 | April 2022
  
  Publication
- Pull Requests as a Training Signal for Repo-Level Code Editing
  
  Qinglin Zhu, Tianyu Chen, Shuai Lu, Lei Ji, Runcong Zhao, Murong Ma, Xiangxiang Dai, Yulan He, Lin Gui, Peng Cheng, Yeyun Gong
  
  ICML 2026 | May 2026
  
  Publication
- From Patches to Trajectories: Privileged Process Supervision for Software-Engineering Agents
  
  Murong Ma, Tianyu Chen, Yun Lin, Shuai Lu, Qinglin Zhu, Yeyun Gong, Zhiyong Huang, Peng Cheng, Yan Lu, Jin Song Dong
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- Reducing the Costs of Proof Synthesis on Rust Systems by Scaling Up a Seed Training Set
  
  Nongyu Di, Tianyu Chen, Shan Lu, Shuai Lu, Yeyun Gong, Peng Cheng, Jay Lorch, Yuan Yao, Xiaoxing Ma
  
  February 2026
  
  February 2026
  
  Publication 项目
- AutoVerus: Automated Proof Generation for Rust Code
  
  Chenyuan Yang, Xuheng Li, Md Rakib Hossain Misu, Jianan Yao, Weidong Cui, Yeyun Gong, Chris Hawblitzel, Shuvendu Lahiri, Jay Lorch, Shuai Lu, Fan Yang, Ziqiao Zhou, Shan Lu
  
  Object-Oriented Programming, Systems, Languages & Applications (OOPSLA) | October 2025
  
  Distinguished Artifact Award
  
  Publication Github 项目
- Automated Proof Generation for Rust Code via Self-Evolution
  
  Tianyu Chen, Shuai Lu, Shan Lu, Yeyun Gong, Chenyuan Yang, Xuheng Li, Md Rakib Hossain Misu, Hao Yu, Nan Duan, Peng Cheng, Fan Yang, Shuvendu Lahiri, Tao Xie, Lidong Zhou
  
  ICLR 2025 | October 2024
  
  Publication 项目
- Reducing the Costs of Proof Synthesis on Rust Systems by Scaling Up a Seed Training Set
  
  Nongyu Di, Tianyu Chen, Shan Lu, Shuai Lu, Yeyun Gong, Peng Cheng, Jay Lorch, Yuan Yao, Xiaoxing Ma
  
  February 2026
  
  February 2026
  
  Publication 项目
- AutoVerus: Automated Proof Generation for Rust Code
  
  Chenyuan Yang, Xuheng Li, Md Rakib Hossain Misu, Jianan Yao, Weidong Cui, Yeyun Gong, Chris Hawblitzel, Shuvendu Lahiri, Jay Lorch, Shuai Lu, Fan Yang, Ziqiao Zhou, Shan Lu
  
  Object-Oriented Programming, Systems, Languages & Applications (OOPSLA) | October 2025
  
  Distinguished Artifact Award
  
  Publication Github 项目
- Automated Proof Generation for Rust Code via Self-Evolution
  
  Tianyu Chen, Shuai Lu, Shan Lu, Yeyun Gong, Chenyuan Yang, Xuheng Li, Md Rakib Hossain Misu, Hao Yu, Nan Duan, Peng Cheng, Fan Yang, Shuvendu Lahiri, Tao Xie, Lidong Zhou
  
  ICLR 2025 | October 2024
  
  Publication 项目
- MS MARCO Web Search: A Large-scale Information-rich Web Dataset with Millions of Real Click Labels
  
  Qi Chen, Xiubo Geng, Corby Rosset, Carolyn Buractaon, Jingwen Lu, Tao Shen, Kun Zhou, Chenyan Xiong, Yeyun Gong, Paul Bennett, Nick Craswell, Xing Xie, Fan Yang, Bryan Tower, Zheng Liu, Mingqin Li, Chuanjie Liu, Jason (Zengzhong) Li, Rangan Majumder, Jennifer Neville, Harsha Simhadri, Manik Varma, Yujing Wang, Linjun Yang, Mao Yang, Ce Zhang
  
  The Web Conference | May 2024
  
  Publication PDF 访问
- CodeRetriever: A Large Scale Contrastive Pre-Training Method for Code Search.
  
  Xiaonan Li, Yeyun Gong, Yelong Shen, Xipeng Qiu, Hang Zhang, Bolun Yao, Weizhen Qi, Daxin Jiang (姜大昕), Weizhu Chen, Nan Duan
  
  EMNLP 2022 | October 2022
  
  Publication
- DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation
  
  Wei Chen, Yeyun Gong, Song Wang, Bolun Yao, Weizhen Qi, Zhongyu Wei, Xiaowu Hu, Bartuer Zhou, Yi Mao, Weizhu Chen, Biao Cheng, Nan Duan
  
  ACL 2022 | May 2022
  
  Publication
- Adversarial Retriever-Ranker for dense text retrieval
  
  Hang Zhang, Yeyun Gong, Yelong Shen, Jiancheng Lv, Nan Duan, Weizhu Chen
  
  ICLR 2022 | April 2022
  
  Publication Publication
- KFCNet: Knowledge Filtering and Contrastive Learning Network for Generative Commonsense Reasoning
  
  Haonan Li, Yeyun Gong, Jian Jiao, Ruofei Zhang, Timothy Baldwin, Nan Duan
  
  Empirical Methods in Natural Language Processing | September 2021
  
  Publication
- ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation
  
  Weizhen Qi, Yeyun Gong, Yu Yan, Can Xu, Bolun Yao, Bartuer Zhou, Biao Cheng, Daxin Jiang (姜大昕), Jiusheng Chen, Ruofei Zhang, Houqiang Li, Nan Duan
  
  Meeting of the Association for Computational Linguistics | August 2021
  
  Publication Github
- FastSeq: Make Sequence Generation Faster
  
  Yu Yan, Fei Hu, Jiusheng Chen, Nikhil Bhendawade, Ting Ye, Yeyun Gong, Nan Duan, Desheng Cui, Bingyu Chi, Ruofei Zhang
  
  Meeting of the Association for Computational Linguistics | August 2021
  
  Publication Github
- Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation
  
  Dayiheng Liu, Yeyun Gong, Yu Yan, Jie Fu, Bo Shao, Daxin Jiang (姜大昕), Jiancheng Lv,, Nan Duan
  
  Empirical Methods in Natural Language Processing | November 2020
  
  Publication
- Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space
  
  Dayiheng Liu, Yeyun Gong, Jie Fu, Yu Yan, Jiusheng Chen, Jiancheng Lv, Nan Duan, Ming Zhou
  
  Empirical Methods in Natural Language Processing | November 2020
  
  Publication
- ProphetNet-Ads: A Looking Ahead Strategy for Generative Retrieval Models in Sponsored Search Engine
  
  Weizhen Qi, Yeyun Gong, Yu Yan, Jian Jiao, Bo Shao, Ruofei Zhang, Houqiang Li, Nan Duan, Ming Zhou
  
  The 9th CCF International Conference on Natural Language Processing and Chinese Computing(NLPCC2020) | October 2020
  
  Publication Github
- ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training
  
  Weizhen Qi, Yu Yan, Yeyun Gong, Dayiheng Liu, Nan Duan, Jiusheng Chen, Ruofei Zhang, Ming Zhou
  
  EMNLP 2020 | October 2020
  
  Publication Github
- RikiNet: Reading Wikipedia Pages for Natural Question Answering
  
  Dayiheng Liu, Yeyun Gong, Jie Fu, Yu Yan, Jiusheng Chen, Daxin Jiang (姜大昕), Jiancheng Lv, Nan Duan
  
  Meeting of the Association for Computational Linguistics | July 2020
  
  Publication
- CodeRetriever: A Large Scale Contrastive Pre-Training Method for Code Search.
  
  Xiaonan Li, Yeyun Gong, Yelong Shen, Xipeng Qiu, Hang Zhang, Bolun Yao, Weizhen Qi, Daxin Jiang (姜大昕), Weizhu Chen, Nan Duan
  
  EMNLP 2022 | October 2022
  
  Publication
- Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversations
  
  Wei Chen, Yeyun Gong, Can Xu, Huang Hu, Bolun Yao, Zhongyu Wei, Zhihao Fan, Xiaowu Hu, Bartuer Zhou, Biao Cheng, Daxin Jiang (姜大昕), Nan Duan
  
  ACL 2022 | April 2022
  
  Publication

- Beyond Static Dialogues: Benchmarking Realistic, Heterogeneous, and Evolving Long-Term Memory
  
  Han Zhang, Zihao Tang, Xinyan Velocity Yu, Xiao Liu, Yeyun Gong, Haizhen Huang, Yan Lu, Weiwei Deng, Feng Sun, Qi Zhang, Hanfang Yang
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- From Patches to Trajectories: Privileged Process Supervision for Software-Engineering Agents
  
  Murong Ma, Tianyu Chen, Yun Lin, Shuai Lu, Qinglin Zhu, Yeyun Gong, Zhiyong Huang, Peng Cheng, Yan Lu, Jin Song Dong
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- Memory Grafting: Scaling Language Model Pre-training via Offline Conditional Memory
  
  Runxi Cheng, Yuchen Guan, Yongxian Wei, Qianpu Sun, Qixiu Li, Sinan Du, Feng Xiong, Chun Yuan, Yan Lu, Yeyun Gong
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- Reducing the Costs of Proof Synthesis on Rust Systems by Scaling Up a Seed Training Set
  
  Nongyu Di, Tianyu Chen, Shan Lu, Shuai Lu, Yeyun Gong, Peng Cheng, Jay Lorch, Yuan Yao, Xiaoxing Ma
  
  February 2026
  
  February 2026
  
  Publication 项目
- LayerNorm Induces Recency Bias in Transformer Decoders
  
  Junu Kim, Xiao Liu, Zheng-Wen Lin, Lei Ji, Yeyun Gong, Edward Choi
  
  September 2025
  
  arXiv | September 2025
  
  Publication
- DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collections
  
  Haebin Shin, Lei Ji, Xiao Liu, Zhiwei Yu, Qi Chen, Yeyun Gong
  
  August 2025
  
  arXiv | August 2025
  
  DOI Publication
- Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoning
  
  Yiming Huang, Xiao Liu, Yeyun Gong, Zhibin Gou, Yelong Shen, Nan Duan, Weizhu Chen
  
  March 2024
  
  arXiv | March 2024
  
  Publication
- Pull Requests as a Training Signal for Repo-Level Code Editing
  
  Qinglin Zhu, Tianyu Chen, Shuai Lu, Lei Ji, Runcong Zhao, Murong Ma, Xiangxiang Dai, Yulan He, Lin Gui, Peng Cheng, Yeyun Gong
  
  ICML 2026 | May 2026
  
  Publication
- m3BERT: A Modern, Multi-lingual, Matryoshka Bidirectional Encoder
  
  Yaoxiang Wang, Simiao Zuo, Qingguo Hu, Yucheng Ding, Yeyun Gong, Jian Jiao, Jinsong Su
  
  KDD 2026 | May 2026
  
  Publication
- Beyond Sunk Costs: Boosting LLM Pre-training Efficiency via Orthogonal Growth of Mixture-of-Experts
  
  Ruizhe Wang, Yucheng Ding, Xiao Liu, Yaoxiang Wang, Peng Cheng, Baining Guo, Zhengjun Zha, Yeyun Gong
  
  ICML 2026 | May 2026
  
  Publication
- AutoVerus: Automated Proof Generation for Rust Code
  
  Chenyuan Yang, Xuheng Li, Md Rakib Hossain Misu, Jianan Yao, Weidong Cui, Yeyun Gong, Chris Hawblitzel, Shuvendu Lahiri, Jay Lorch, Shuai Lu, Fan Yang, Ziqiao Zhou, Shan Lu
  
  Object-Oriented Programming, Systems, Languages & Applications (OOPSLA) | October 2025
  
  Distinguished Artifact Award
  
  Publication Github 项目
- Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVR
  
  Xiao Liang, Zhong-zhi Li, Yeyun Gong, Yelong Shen, Yingchun Wu, Zhijiang Guo, Weizhu Chen
  
  ICLR 2026 | August 2025
  
  DOI Publication
- Velocitune: A Velocity-based Dynamic Domain Reweighting Method for Continual Pre-training
  
  Zheheng Luo, Xin Zhang, Xiao Liu, Haoling Li, Yeyun Gong, Qi Chen, Peng Cheng
  
  ACL | August 2025
  
  DOI Publication Publication
- Routing Mamba: Scaling State Space Models with Mixture-of-Experts Projection
  
  Zheng Zhan, Liliang Ren, Shuohang Wang, Liyuan Liu, Yang Liu, Yeyun Gong, Yanzhi Wang, Yelong Shen
  
  NeurIPS 2025 | June 2025
  
  DOI Publication Publication
- PeRL: Permutation-Enhanced Reinforcement Learning for Interleaved Vision-Language Reasoning
  
  Yizhen Zhang, Yang Ding, Shuoshuo Zhang, Xinchen Zhang, Haoling Li, Zhong-zhi Li, Peijie Wang, Jie Wu, Lei Ji, Yelong Shen, Yujiu Yang, Yeyun Gong
  
  NeurIPS 2025 | June 2025
  
  DOI Publication Publication
- SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoning
  
  Xiao Liang, Zhong-zhi Li, Yeyun Gong, Yang Wang, Hengyuan Zhang, Yelong Shen, Yingchun Wu, Weizhu Chen
  
  NeurIPS 2025 | June 2025
  
  DOI Publication Publication
- Enhancing Large Language Model Performance with Gradient-Based Parameter Selection
  
  Haoling Li, Xin Zhang, Xiao Liu, Yeyun Gong, Yifan Wang, Qi Chen, Peng Cheng
  
  AAAI Conference on Artificial Intelligence, 2025 | April 2025
  
  Oral Presentation
  
  DOI Publication
- Overcoming Vocabulary Mismatch: Vocabulary-agnostic Teacher Guided Language Modeling
  
  Haebin Shin, Lei Ji, Xiao Liu, Yeyun Gong
  
  ICML 2025 | March 2025
  
  DOI Publication Publication
- Optimizing Large Language Model Training Using FP4 Quantization
  
  Ruizhe Wang, Yeyun Gong, Xiao Liu, Guoshuai Zhao, Ziyue Yang, Baining Guo, Zhengjun Zha, Peng Cheng
  
  ICML 2025 | January 2025
  
  DOI Publication Publication
- Integrative Decoding: Improve Factuality via Implicit Self-consistency
  
  Yi Cheng, Xiao Liang, Yeyun Gong, Wen Xiao, Song Wang, Yuji Zhang, Wenjun Hou, Kaishuai Xu, Wenge Liu, Wenjie Li, Jian Jiao, Qi Chen, Peng Cheng, Wayne Xiong
  
  ICLR 2025 | January 2025
  
  DOI Publication Publication
- Automated Proof Generation for Rust Code via Self-Evolution
  
  Tianyu Chen, Shuai Lu, Shan Lu, Yeyun Gong, Chenyuan Yang, Xuheng Li, Md Rakib Hossain Misu, Hao Yu, Nan Duan, Peng Cheng, Fan Yang, Shuvendu Lahiri, Tao Xie, Lidong Zhou
  
  ICLR 2025 | October 2024
  
  Publication 项目
- Not All Tokens Are What You Need for Pretraining
  
  Yeyun Gong, Xiao Liu, Yelong Shen, Ruochen Xu, Jian Jiao, Nan Duan, Weizhu Chen
  
  Neural Information Processing Systems | October 2024
  
  Best Paper Runner Up
  
  Publication
- MS MARCO Web Search: A Large-scale Information-rich Web Dataset with Millions of Real Click Labels
  
  Qi Chen, Xiubo Geng, Corby Rosset, Carolyn Buractaon, Jingwen Lu, Tao Shen, Kun Zhou, Chenyan Xiong, Yeyun Gong, Paul Bennett, Nick Craswell, Xing Xie, Fan Yang, Bryan Tower, Zheng Liu, Mingqin Li, Chuanjie Liu, Jason (Zengzhong) Li, Rangan Majumder, Jennifer Neville, Harsha Simhadri, Manik Varma, Yujing Wang, Linjun Yang, Mao Yang, Ce Zhang
  
  The Web Conference | May 2024
  
  Publication PDF 访问
- AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation
  
  Tong Wu, Zhihao Fan, Xiao Liu, Yeyun Gong, Yelong Shen, Jian Jiao, Hai-Tao Zheng, Juntao Li, zhongyu wei, Jian Guo, Nan Duan, Weizhu Chen
  
  NeurIPS 2023 | December 2023
  
  Publication
- ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving
  
  Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Minlie Huang, Nan Duan, Weizhu Chen
  
  ICLR 2024 | September 2023
  
  DOI PDF
- Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph
  
  Jiashuo Sun, Chengjin Xu, Lumingyuan Tang, Sai Wang, Chen Lin, Yeyun Gong, Lionel M. Ni, H. Shum, Jian Guo
  
  ICLR 2024 | July 2023
  
  Publication
- CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
  
  Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, Weizhu Chen
  
  ICLR 2024 | May 2023
  
  DOI Publication
- CodeRetriever: A Large Scale Contrastive Pre-Training Method for Code Search.
  
  Xiaonan Li, Yeyun Gong, Yelong Shen, Xipeng Qiu, Hang Zhang, Bolun Yao, Weizhen Qi, Daxin Jiang (姜大昕), Weizhu Chen, Nan Duan
  
  EMNLP 2022 | October 2022
  
  Publication
- DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation
  
  Wei Chen, Yeyun Gong, Song Wang, Bolun Yao, Weizhen Qi, Zhongyu Wei, Xiaowu Hu, Bartuer Zhou, Yi Mao, Weizhu Chen, Biao Cheng, Nan Duan
  
  ACL 2022 | May 2022
  
  Publication
- Distill-VQ: Learning Retrieval Oriented Vector Quantization by Distilling Knowledge from Dense Embeddings
  
  Shitao Xiao, Zheng Liu, Weihao Han, Jianjin Zhang, Defu Lian, Yeyun Gong, Qi Chen, Fan Yang, Hao Sun, Yingxia Shao, Xing Xie
  
  SIGIR 2022 | April 2022
  
  Publication
- Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversations
  
  Wei Chen, Yeyun Gong, Can Xu, Huang Hu, Bolun Yao, Zhongyu Wei, Zhihao Fan, Xiaowu Hu, Bartuer Zhou, Biao Cheng, Daxin Jiang (姜大昕), Nan Duan
  
  ACL 2022 | April 2022
  
  Publication
- Adversarial Retriever-Ranker for dense text retrieval
  
  Hang Zhang, Yeyun Gong, Yelong Shen, Jiancheng Lv, Nan Duan, Weizhu Chen
  
  ICLR 2022 | April 2022
  
  Publication Publication
- KFCNet: Knowledge Filtering and Contrastive Learning Network for Generative Commonsense Reasoning
  
  Haonan Li, Yeyun Gong, Jian Jiao, Ruofei Zhang, Timothy Baldwin, Nan Duan
  
  Empirical Methods in Natural Language Processing | September 2021
  
  Publication
- ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation
  
  Weizhen Qi, Yeyun Gong, Yu Yan, Can Xu, Bolun Yao, Bartuer Zhou, Biao Cheng, Daxin Jiang (姜大昕), Jiusheng Chen, Ruofei Zhang, Houqiang Li, Nan Duan
  
  Meeting of the Association for Computational Linguistics | August 2021
  
  Publication Github
- FastSeq: Make Sequence Generation Faster
  
  Yu Yan, Fei Hu, Jiusheng Chen, Nikhil Bhendawade, Ting Ye, Yeyun Gong, Nan Duan, Desheng Cui, Bingyu Chi, Ruofei Zhang
  
  Meeting of the Association for Computational Linguistics | August 2021
  
  Publication Github
- Poolingformer: Long Document Modeling with Pooling Attention
  
  Hang Zhang, Yeyun Gong, Yelong Shen, Weisheng Li, Jiancheng Lv, Nan Duan, Wei Chen
  
  2021 International Conference on Machine Learning | July 2021
  
  Publication Publication
- EL-Attention: Memory Efficient Lossless Attention for Generation
  
  Yu Yan, Jiusheng Chen, Weizhen Qi, Nikhil Bhendawade, Yeyun Gong, Nan Duan, Ruofei Zhang
  
  2021 International Conference on Machine Learning | July 2021
  
  Publication
- BANG: Bridging Autoregressive and Non-autoregressive Generation with Large Scale Pretraining
  
  Weizhen Qi, Yeyun Gong, Jian Jiao, Yu Yan, Dayiheng Liu, Wei Chen, Kewen Tang, Houqiang Li, Jiusheng Chen, Ruofei Zhang, Ming Zhou, Nan Duan
  
  2021 International Conference on Machine Learning | December 2020
  
  Publication
- Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation
  
  Dayiheng Liu, Yeyun Gong, Yu Yan, Jie Fu, Bo Shao, Daxin Jiang (姜大昕), Jiancheng Lv,, Nan Duan
  
  Empirical Methods in Natural Language Processing | November 2020
  
  Publication
- Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space
  
  Dayiheng Liu, Yeyun Gong, Jie Fu, Yu Yan, Jiusheng Chen, Jiancheng Lv, Nan Duan, Ming Zhou
  
  Empirical Methods in Natural Language Processing | November 2020
  
  Publication
- ProphetNet-Ads: A Looking Ahead Strategy for Generative Retrieval Models in Sponsored Search Engine
  
  Weizhen Qi, Yeyun Gong, Yu Yan, Jian Jiao, Bo Shao, Ruofei Zhang, Houqiang Li, Nan Duan, Ming Zhou
  
  The 9th CCF International Conference on Natural Language Processing and Chinese Computing(NLPCC2020) | October 2020
  
  Publication Github
- ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training
  
  Weizhen Qi, Yu Yan, Yeyun Gong, Dayiheng Liu, Nan Duan, Jiusheng Chen, Ruofei Zhang, Ming Zhou
  
  EMNLP 2020 | October 2020
  
  Publication Github
- RikiNet: Reading Wikipedia Pages for Natural Question Answering
  
  Dayiheng Liu, Yeyun Gong, Jie Fu, Yu Yan, Jiusheng Chen, Daxin Jiang (姜大昕), Jiancheng Lv, Nan Duan
  
  Meeting of the Association for Computational Linguistics | July 2020
  
  Publication
- Alchemy: Amplifying Theorem-Proving Capability through Symbolic Mutation
  
  Shaonan Wu, Shuai Lu, Yeyun Gong, Nan Duan, Ping Wei
  
  October 2024
  
  October 2024
  
  Publication

Yeyun Gong

论文与出版物

Beyond Static Dialogues: Benchmarking Realistic, Heterogeneous, and Evolving Long-Term Memory

Pull Requests as a Training Signal for Repo-Level Code Editing

From Patches to Trajectories: Privileged Process Supervision for Software-Engineering Agents

Memory Grafting: Scaling Language Model Pre-training via Offline Conditional Memory

m3BERT: A Modern, Multi-lingual, Matryoshka Bidirectional Encoder

Beyond Sunk Costs: Boosting LLM Pre-training Efficiency via Orthogonal Growth of Mixture-of-Experts

Reducing the Costs of Proof Synthesis on Rust Systems by Scaling Up a Seed Training Set

AutoVerus: Automated Proof Generation for Rust Code

LayerNorm Induces Recency Bias in Transformer Decoders

Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVR

DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collections

Velocitune: A Velocity-based Dynamic Domain Reweighting Method for Continual Pre-training

Routing Mamba: Scaling State Space Models with Mixture-of-Experts Projection

PeRL: Permutation-Enhanced Reinforcement Learning for Interleaved Vision-Language Reasoning

SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoning

Enhancing Large Language Model Performance with Gradient-Based Parameter Selection

Overcoming Vocabulary Mismatch: Vocabulary-agnostic Teacher Guided Language Modeling

Optimizing Large Language Model Training Using FP4 Quantization

Integrative Decoding: Improve Factuality via Implicit Self-consistency

Automated Proof Generation for Rust Code via Self-Evolution

Alchemy: Amplifying Theorem-Proving Capability through Symbolic Mutation

Not All Tokens Are What You Need for Pretraining

MS MARCO Web Search: A Large-scale Information-rich Web Dataset with Millions of Real Click Labels

Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoning

AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation

ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving

Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph

CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing

CodeRetriever: A Large Scale Contrastive Pre-Training Method for Code Search.

DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation

Distill-VQ: Learning Retrieval Oriented Vector Quantization by Distilling Knowledge from Dense Embeddings

Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversations

Adversarial Retriever-Ranker for dense text retrieval

KFCNet: Knowledge Filtering and Contrastive Learning Network for Generative Commonsense Reasoning

FastSeq: Make Sequence Generation Faster

ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation

EL-Attention: Memory Efficient Lossless Attention for Generation

Poolingformer: Long Document Modeling with Pooling Attention

BANG: Bridging Autoregressive and Non-autoregressive Generation with Large Scale Pretraining

Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation

Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space

ProphetNet-Ads: A Looking Ahead Strategy for Generative Retrieval Models in Sponsored Search Engine

ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training

RikiNet: Reading Wikipedia Pages for Natural Question Answering

Beyond Static Dialogues: Benchmarking Realistic, Heterogeneous, and Evolving Long-Term Memory

Pull Requests as a Training Signal for Repo-Level Code Editing

From Patches to Trajectories: Privileged Process Supervision for Software-Engineering Agents

Memory Grafting: Scaling Language Model Pre-training via Offline Conditional Memory

m3BERT: A Modern, Multi-lingual, Matryoshka Bidirectional Encoder

Beyond Sunk Costs: Boosting LLM Pre-training Efficiency via Orthogonal Growth of Mixture-of-Experts

Reducing the Costs of Proof Synthesis on Rust Systems by Scaling Up a Seed Training Set

AutoVerus: Automated Proof Generation for Rust Code

LayerNorm Induces Recency Bias in Transformer Decoders

Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVR

DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collections

Velocitune: A Velocity-based Dynamic Domain Reweighting Method for Continual Pre-training

Routing Mamba: Scaling State Space Models with Mixture-of-Experts Projection

PeRL: Permutation-Enhanced Reinforcement Learning for Interleaved Vision-Language Reasoning

SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoning

Enhancing Large Language Model Performance with Gradient-Based Parameter Selection

Overcoming Vocabulary Mismatch: Vocabulary-agnostic Teacher Guided Language Modeling

Optimizing Large Language Model Training Using FP4 Quantization

Integrative Decoding: Improve Factuality via Implicit Self-consistency

Automated Proof Generation for Rust Code via Self-Evolution

Alchemy: Amplifying Theorem-Proving Capability through Symbolic Mutation

Not All Tokens Are What You Need for Pretraining

Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoning

AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation

ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving

Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph

CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing

CodeRetriever: A Large Scale Contrastive Pre-Training Method for Code Search.

Distill-VQ: Learning Retrieval Oriented Vector Quantization by Distilling Knowledge from Dense Embeddings

Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversations

Adversarial Retriever-Ranker for dense text retrieval

KFCNet: Knowledge Filtering and Contrastive Learning Network for Generative Commonsense Reasoning

ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation

FastSeq: Make Sequence Generation Faster