Now, I am a Research Scientist (Field of Large Speech Model) in ByteDance . I graduated from the Department of Computer Science, Zhejiang University (浙江大学计算机科学与技术学院), advised by Zhao Zhou (赵洲). Before that, I graduated from Chu Kochen Honors College, Zhejiang University (浙江大学竺可桢学院).

My research interest includes speech & singing synthesis, avatar, and machine translation. I have published over 30 papers at the top international AI conferences such as NeurIPS, ICLR, ICML, AAAI, ACL, with total google scholar citations .

I served as reviewers in EMNLP 21&22, ICML 22, NeurIPS 22&23, ICLR 24, ICASSP 23&24, ACL 23, NAACL 24, TASLP etc.

Previously, I worked as a research intern at Alibaba DAMO Academy , and ByteDance SAMI . I used to have academic cooperation with Microsoft Research Asia .

My selected open-source projects: DiffSinger GitHub Stars ; NATSpeech Github Stars; Diffsinger has been introduced by many popular videos bilibili!

My selected project @ ByteDance: MegaTTS 2.

🔥 News

  • 2024.01: Two papers are accepted by ICLR 2024!
  • 2023.05: Six papers are accepted by ACL 2023!
  • 2023.04: One paper is accepted by ICML 2023!
  • 2023.01: Two papers are accepted by ICLR 2023!
  • 2022.09: Three papers are accepted by NeurIPS 2022!
  • 2022.02: One paper is accepted by ACL 2022!

📝 Publications

Selected Papers

ICLR
sym

MegaTTS 2: Boosting Prompting Mechanisms for Zero-Shot Speech Synthesis
Ziyue Jiang, Jinglin Liu*, Yi Ren, Jinzheng He, Zhenhui Ye, Shengpeng Ji, Qian Yang, Chen Zhang, Pengfei Wei, Chunfeng Wang, Xiang Yin, Zejun MA, Zhou Zhao; & ZJU

Project

  • Brief Introduction: Large Text-to-Speech Model.
AAAI
sym

DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism
Jinglin Liu, Chengxi Li, Yi Ren, Feiyang Chen, Zhou Zhao

Project | | | Hugging Face | Hugging Face

  • Brief Introduction: This paper contains the first acoustic models based on diffusion, including DiffSinger (SVS) and DiffSpeech (TTS). It realizes the high-quality speech/singing synthesis.
  • DiffSinger & DiffSpeech have recieved GitHub Stars & Github Stars, and the downloads number of pre-trained model is downloads.
  • Many video demos created by Bilibili creators are released. And Diffsinger is introduced by a very popular video bilibili!
  • This work is included by many famous speech/music synthesis open-source projects, such as ESPNet , PaddlePaddle/Parakeet , muzic .
NeurIPS
sym

PortaSpeech: Portable and High-Quality Generative Text-to-Speech
Yi Ren*, Jinglin Liu*, Zhou Zhao

Project | | Hugging Face

  • The source codes of this paper are released together with the codes of DiffSpeech. This repository has received Github Stars; It is shown on the Github Daily Trending List.
ACL
sym

Learning the Beauty in Songs: Neural Singing Voice Beautifier
Jinglin Liu, Chengxi Li, Yi Ren, Zhiying Zhu, Zhou Zhao

Project | downloads

Recent Papers

My full paper list is shown at my scholar homepage.

📄 Services

  • Program Committee and Paper Reviewer: EMNLP (2021, 2022), ICML (2022), NeurIPS (2022,2023), ICLR (2024), ICASSP (2023, 2024), ACM-MM (2023), ACL (2023), TASLP (2023).

💻 Industrial Experience

  • 2023.05 - Now, ByteDance, Research Scientist.
  • 2022.07 - 2023.02, Alibaba, Research Intern.
  • 2021.06 - 2021.09, ByteDance, Research Intern.

🎖 Honors and Awards

  • 2023.01 Outstanding Graduate of Zhejiang Province
  • 2023.01 Outstanding Graduate of ZJU
  • 2022.12 Runner-up in China Graduate AI Innovation Competition (2/1217)
  • 2022.10 National Scholarship (Top 1%)
  • 2021.10 National Scholarship (Top 1%)
  • 2021.10 Tencent Scholarship (Top 1%)
  • 2020.06 Excellent Undergraduate Thesis of ZJU
  • 2020.06 Outstanding Graduate of ZJU

📖 Educations

  • 2020.06 - 2023.03, Master, Zhejiang University, Hangzhou.
  • 2016.09 - 2020.06, Undergraduate, Chu Kochen Honors College, Zhejiang Univeristy, Hangzhou.

💬 Invited Talks

  • 2022.02, Audio Synthesis and Re-Synthesis, at MLNLP seminar | [Video]