Yu Zhang's Homepage

Hi, my name is Yu Zhang ([jy tʃɑŋ], 張宇/张宇 in traditional/simplified Chinese). I am a researcher at Moonshot AI. Prior to this, I received my Ph.D. degree from Soochow University in 2026, advised by Prof. Guohong Fu.

~~My early research focused on structured prediction tasks, specifically dependency parsing and constituency parsing.~~ Currently, my research interests have evolved to focus on developing efficient text generation models. I am particularly intrigued by the prospect of developing hardware-efficient methods for linear-time sequence modeling. As a disciple of parallel programming, I am passionate about exploring techniques that harness the power of parallel computing to develop scalable subquadratic models.

Publications

Semantic Scholar Google Scholar DBLP

* denotes equal contributions

Kimi Linear: An Expressive, Eﬀicient Attention Architecture

Yu Zhang, Zongyu Lin, Xingcheng Yao, Jiaxi Hu, Fanqing Meng, Chengyin Liu, Xin Men, Songlin Yang, Zhiyuan Li, Wentao Li, Enzhe Lu, Weizhou Liu, Yanru Chen, Weixin Xu, Longhui Yu, Yejie Wang, Yu Fan, Longguang Zhong, Enming Yuan, Dehao Zhang, Yizhi Zhang, TY Liu, Haiming Wang, Shengjun Fang, Weiran He, Shaowei Liu, Yiwei Li, Jianlin Su, Jiezhong Qiu, Bo Pang, Junjie Yan, Zhejun Jiang, Weixiao Huang, Bohong Yin, Jiacheng You, Chu Wei, Zhengtao Wang, Chao Hong, Yutian Chen, Guanduo Chen, Yucheng Wang, Huabin Zheng, Feng Wang, Yibo Liu, Mengnan Dong, Zheng Zhang, Siyuan Pan, Wenhao Wu, Yuhao Wu, Longyu Guan, Jiawen Tao, Guohong Fu, Xinran Xu, Yuzhi Wang, Guokun Lai, Yuxin Wu, Xinyu Zhou, Zhilin Yang, Yulun Du

Preprint

arxiv code citation

We introduce Kimi Linear, a novel attention architecture that achieves expressive power comparable to softmax attention while maintaining linear time complexity. Our approach combines the delta rule with gated slot attention to enable efficient sequence modeling. Experimental results demonstrate competitive performance on language modeling benchmarks with significantly improved inference efficiency.

Gated Slot Attention for Efficient Linear-Time Sequence Modeling

Yu Zhang*, Songlin Yang*, Ruijie Zhu, Yue Zhang, Leyang Cui, Yiqiao Wang, Bolun Wang, Freda Shi, Bailin Wang, Wei Bi, Peng Zhou, Guohong Fu

NeurIPS 2024

arxiv code citation

Linear attention has emerged as a promising alternative to softmax attention for its efficient computation and comparable performance. In this work, we propose Gated Slot Attention (GSA), a new linear attention mechanism that combines the benefits of gated linear attention and slot-based memory. GSA achieves strong performance on language modeling tasks while maintaining linear complexity with respect to sequence length.

Parallelizing Linear Transformers with the Delta Rule over Sequence Length

Songlin Yang, Bailin Wang, Yu Zhang, Yikang Shen, Yoon Kim

NeurIPS 2024

arxiv code citation

Linear attention mechanisms offer efficient sequence modeling but lack the expressive power of softmax attention. We propose a novel parallelization scheme for linear transformers using the delta rule, enabling efficient training while maintaining the benefits of linear complexity. Our approach allows for effective modeling of sequential dependencies with significantly reduced computational cost.

Scalable MatMul-free Language Modeling

Ruijie Zhu, Yu Zhang, Ethan Sifferman, Tyler Sheaves, Yiqiao Wang, Dustin Richmond, Peng Zhou, Jason K. Eshraghian

Preprint

arxiv code citation

Matrix multiplication (MatMul) dominates the computational cost of language models. We propose a scalable MatMul-free language modeling approach that eliminates expensive matrix multiplications while maintaining competitive performance. Our method enables efficient deployment on hardware with limited resources.

Non-autoregressive Text Editing with Copy-aware Latent Alignments

Yu Zhang*, Yue Zhang*, Leyang Cui, Guohong Fu

EMNLP 2023

arxiv code citation

Non-autoregressive text editing models offer efficient text generation by predicting edits rather than tokens. We propose copy-aware latent alignments to improve the accuracy of edit-based models by better handling copy operations. Our approach achieves state-of-the-art results on text editing benchmarks with significantly faster inference.

Semantic Role Labeling as Dependency Parsing: Exploring Latent Tree Structures Inside Arguments

Yu Zhang, Qingrong Xia, Shilin Zhou, Yong Jiang, Guohong Fu, Min Zhang

COLING 2022

arxiv code citation

Semantic role labeling (SRL) is typically treated as a sequence labeling or span prediction task. We explore an alternative formulation where SRL is cast as dependency parsing, revealing latent tree structures inside semantic arguments. Our approach leverages dependency parsing techniques to capture hierarchical relationships between arguments.

Fast and Accurate End-to-End Span-based Semantic Role Labeling as Word-based Graph Parsing Best Paper Award

Shilin Zhou, Qingrong Xia, Zhenghua Li, Yu Zhang, Yu Hong, Min Zhang

COLING 2022

arxiv code citation

Span-based semantic role labeling has achieved great success but often requires complex pipeline architectures. We reformulate SRL as word-based graph parsing, enabling fast and accurate end-to-end processing. Our approach won the Best Paper Award at COLING 2022 for its elegant formulation and strong empirical results.

Fast and Accurate Neural CRF Constituency Parsing

Yu Zhang*, Houquan Zhou*, Zhenghua Li

IJCAI 2020

arxiv code citation

Constituency parsing is a fundamental NLP task requiring both accuracy and efficiency. We propose a fast and accurate neural CRF approach that achieves state-of-the-art results with significantly reduced computational cost. Our method combines neural encoding with efficient CRF inference for optimal performance.

Efficient Second-Order TreeCRF for Neural Dependency Parsing

Yu Zhang, Zhenghua Li, Min Zhang

ACL 2020

arxiv code citation

Dependency parsing benefits from structured prediction with TreeCRFs, but second-order models are computationally expensive. We propose efficient algorithms for second-order TreeCRF inference that maintain accuracy while significantly reducing training time. Our approach enables practical use of rich structural features.

Is POS Tagging Necessary or Even Helpful for Neural Dependency Parsing? Best Paper Award

Houquan Zhou*, Yu Zhang*, Zhenghua Li, Min Zhang

NLPCC 2020

arxiv code citation

Part-of-speech (POS) tagging has long been considered essential for dependency parsing. We investigate whether POS tagging is still necessary in the era of neural networks, providing empirical analysis on the contribution of POS tags to parsing performance. Our findings challenge conventional wisdom and won the Best Paper Award at NLPCC 2020.

HLT@SUDA at SemEval-2019 Task 1: UCCA Graph Parsing as Constituent Tree Parsing

Wei Jiang, Zhenghua Li, Yu Zhang, Min Zhang

SemEval 2019

arxiv code citation

Universal Conceptual Cognitive Annotation (UCCA) provides a cross-lingual semantic representation. We approach UCCA graph parsing by reducing it to constituent tree parsing, leveraging existing parsing techniques for tree structures. Our system achieved competitive results at the SemEval-2019 shared task.

Projects

FLA

code v0.2.0

A Triton-Based Library for Hardware-Efficient Implementations of Linear Attention Mechanism.

SuPar

code v1.1.4

State-of-the-art syntactic/semantic parsers. A Python package designed for structured prediction, including reproductions of many state-of-the-art syntactic/semantic parsers (with pretrained models for more than 19 languages), and highly-parallelized implementations of several well-known structured prediction algorithms.

Experience

2025 - present

AI Researcher at Moonshot AI

2024 - 2025

Research Intern at Shanghai AI Lab

mentored by Peng Gao

2023 - 2024

Research Intern at Tencent AI Lab

mentored by Wei Bi

2020 - 2021

Research Intern at Alibaba DAMO Academy

mentored by Yong Jiang

Last updated: