I am a 2nd-year PhD student in the Department of Computer Science and Technology at Tsinghua University. I am supervised by Prof. Zhiyuan Liu. My research interests lie in Pre-trained Models for Natural Language Processing.
* indicates equal contribution.
Zhengyan Zhang, Xu Han, Hao Zhou, Pei Ke, Yuxian Gu, Deming Ye, Yujia Qin, Yusheng Su, Haozhe Ji, Jian Guan, Fanchao Qi, Xiaozhi Wang, Yanan Zheng, Guoyang Zeng, Huanqi Cao, Shengqi Chen, Daixuan Li, Zhenbo Sun, Zhiyuan Liu, Minlie Huang, Wentao Han, Jie Tang, Juanzi Li, Xiaoyan Zhu, Maosong Sun. CPM: A Large-scale Generative Chinese Pre-trained Language Model. [arxiv] [code] [homepage]
Zhengyan Zhang, Fanchao Qi, Zhiyuan Liu, Qun Liu, Maosong Sun. Know What You Don’t Need: Single-Shot Meta-Pruning for Attention Heads. [arxiv]
Yusheng Su, Xu Han, Zhengyan Zhang, Peng Li, Zhiyuan Liu, Yankai Lin, Jie Zhou, Maosong Sun. CokeBERT: Contextual Knowledge Selection and Embedding towards Enhanced Pre-Trained Language Models. [arxiv]
Jie Zhou*, Ganqu Cui*, Zhengyan Zhang*, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, Maosong Sun. Graph Neural Networks: A Review of Methods and Applications. [arxiv]
Yuan Yao, Haoxi Zhong, Zhengyan Zhang, Xu Han, Xiaozhi Wang, Chaojun Xiao, Guoyang Zeng, Zhiyuan Liu, Maosong Sun. Adversarial Language Games for Advanced Natural Language Intelligence. To appear at AAAI Conference on Artifical Intelligence (AAAI 2021). [arxiv]
Xiaozhi Wang, Tianyu Gao, Zhaocheng Zhu, Zhengyan Zhang, Zhiyuan Liu, Juanzi Li, Jian Tang. KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation. To appear at Transactions of the Association for Computational Linguistics. [pdf] [dataset]
Yuxian Gu, Zhengyan Zhang, Xiaozhi Wang, Zhiyuan Liu, Maosong Sun. Train No Evil: Selective Masking for Task-guided Pre-training. The Conference on Empirical Methods in Natural Language Processing (EMNLP 2020). [pdf] [code] (short)
Zhengyan Zhang*, Xu Han*, Zhiyuan Liu, Xin Jiang, Maosong Sun, Qun Liu. ERNIE: Enhanced Language Representation with Informative Entities. The 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019). [arxiv] [code]
Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Maosong Sun, Zhichong Fang, Bo Zhang, Leyu Lin. COSINE: Compressive network embedding on large-scale information networks. IEEE Transactions on Knowledge and Data Engineering. (TKDE) [pdf]
Cunchao Tu, Zhengyan Zhang, Zhiyuan Liu, Maosong Sun. TransNet: Translation-Based Network Representation Learning for Social Relation Extraction. International Joint Conference on Artificial Intelligence. (IJCAI 2017) [pdf] [code]
Cunchao Tu, Xiangkai Zeng, Hao Wang, Zhengyan Zhang, Zhiyuan Liu, Maosong Sun, Bo Zhang, Leyu Lin. A Unified Framework for Community Detection and Network Representation Learning. IEEE Transactions on Knowledge and Data Engineering. (TKDE) [pdf]