Audio deepfake detection, speech recognition and synthesis, continual learning.

· University of Chinese Academy of Sciences, Pattern Recognition and Intelligent System, Ph.D. , 2015-2018

· Graduate School of Chinese Academy of Social Sciences, Computational Linguistics, Master, 2007-2010

·  Department of Automation, Tsinghua University, Associate Researcher, 2024-

·  Institute of Automation, Chinese Academy of Sciences, Associate Researcher, 2020-2024

·  Institute of Automation, Chinese Academy of Sciences, Assistant Researcher, 2018-2020

·  Alibaba Group, Cloud Computing Department, Senior R&D Engineer, 2011-2014

·  “Speech Signal Processing”, School of Artificial Intelligence, University of Chinese Academy of Sciences, 2021-2024

·  “Speech Interaction”, School of Computer Science and Technology, University of Chinese Academy of Sciences, 2018-2019

   

Selected Publications:

[1]       Jiangyan Yi, Chenglong Wang, Jianhua Tao, Chu Yuan Zhang et al. SceneFake: An Initial Dataset and Benchmarks for Scene Fake Audio Detection. Pattern Recognition (2024)

[2]       Jiangyan Yi, Jianhua Tao, Ruibo Fu, Tao Wang, Chu Yuan Zhang, Chenglong Wang: Adversarial Multi-Task Learning for Mandarin Prosodic Boundary Prediction with Multi-Modal Embeddings. IEEE ACM Trans. Audio Speech Lang. Process. 31: 2963-2973 (2023)

[3]       Jiangyan Yi, Jianhua Tao, Zhengqi Wen, Ye Bai: Language-Adversarial Transfer Learning for Low-Resource Speech Recognition. IEEE ACM Trans. Audio Speech Lang. Process. 27(3): 621-630 (2019)

[4]       Jiangyan Yi, Jianhua Tao, Ye Bai, Zhengkun Tian, Cunhang Fan: Transfer knowledge for punctuation prediction via adversarial training. Speech Communication. 149: 1-10 (2023)

[5]       Jiangyan Yi, Zhengqi Wen, Jianhua Tao, Hao Ni, Bin Liu: CTC Regularized Model Adaptation for Improving LSTM RNN Based Multi-Accent Mandarin Speech Recognition. J. Signal Process. Syst. 90(7): 985-997 (2018)

[6]       Jiangyan Yi, Ruibo Fu, Jianhua Tao, Shuai Nie, Haoxin Ma, Chenglong Wang, Tao Wang, Zhengkun Tian, Ye Bai, Cunhang Fan, Shan Liang, Shiming Wang, Shuai Zhang, Xinrui Yan, Le Xu, Zhengqi Wen, Haizhou Li: ADD 2022: the First Audio Deep Synthesis Detection Challenge. ICASSP 2022 : 9216-9220

[7]       Jiangyan Yi, Ye Bai, Jianhua Tao, Haoxin Ma, Zhengkun Tian, Chenglong Wang , Tao Wang, Ruibo Fu: Half-Truth: A Partially Fake Audio Detection Dataset. INTERSPEECH 2021 : 1654-1658

[8]       Jiangyan Yi, Jianhua Tao, Zhengkun Tian, Ye Bai, Cunhang Fan: Focal Loss for Punctuation Prediction. INTERSPEECH 2020: 721-725

[9]       Jiangyan Yi, Jianhua Tao, Ye Bai: Language-invariant Bottleneck Features from Adversarial End-to-end Acoustic Models for Low Resource Speech Recognition. ICASSP 2019: 6071-6075

[10]   Jiangyan Yi, Jianhua Tao: Self-attention Based Model for Punctuation Prediction Using Word and Speech Embeddings. ICASSP 2019: 7270-7274

[11]   Jiangyan Yi, Jianhua Tao, Zhengqi Wen, Ye Bai: Adversarial Multilingual Training for Low-Resource Speech Recognition. ICASSP 2018: 4899-4903

[12]   Jiangyan Yi, Jianhua Tao, Zhengqi Wen, Ya Li: Distilling Knowledge from an Ensemble of Models for Punctuation Prediction. INTERSPEECH 2017: 2779-2783

[13]   Jiangyan Yi, Jianhua Tao, Ruibo Fu, Xinrui Yan, Chenglong Wang, Tao Wang, Chu Yuan Zhang, Xiaohui Zhang, Yan Zhao, Yong Ren, Le Xu, Junzuo Zhou, Hao Gu, Zhengqi Wen, Shan Liang, Zheng Lian, Shuai Nie, Haizhou Li: ADD 2023: the Second Audio Deepfake Detection Challenge. DADA@IJCAI 2023: 125-130

[14]   Tao Wang, Jiangyan Yi*, Ruibo Fu, Jianhua Tao, Zhengqi Wen, Chu Yuan Zhang: Emotion selectable end-to-end text-based speech editing. Artificial Intelligence. 329 (2024)

[15]   Cunhang Fan, Jiangyan Yi*, Jianhua Tao, Zhengkun Tian, Bin Liu, Zhengqi Wen: Gated Recurrent Fusion With Joint Training Framework for Robust End-to-End Speech Recognition. IEEE ACM Trans. Audio Speech Lang. Process. 29: 198-209 (2021)

[16]   Ye Bai, Jiangyan Yi*, Jianhua Tao, Zhengqi Wen, Zhengkun Tian, Shuai Zhang: Integrating Knowledge Into End-to-End Speech Recognition From External Text-Only Data. IEEE ACM Trans. Audio Speech Lang. Process. 29: 1340-1351 (2021)

[17]   Ye Bai, Jiangyan Yi*, Jianhua Tao, Zhengkun Tian, Zhengqi Wen, Shuai Zhang: Fast End-to-End Speech Recognition Via Non-Autoregressive Models and Cross-Modal Knowledge Transferring From BERT. IEEE ACM Trans. Audio Speech Lang. Process. 29: 1897-1911 (2021)

[18]   Tao Wang, Jiangyan Yi*, Ruibo Fu, Jianhua Tao, Zhengqi Wen: CampNet: Context-Aware Mask Prediction for End-to-End Text-Based Speech Editing. IEEE ACM Trans. Audio Speech Lang. Process. 30: 2241-2254 (2022)

[19]   Tao Wang, Ruibo Fu, Jiangyan Yi*, Jianhua Tao, Zhengqi Wen: NeuralDPS: Neural Deterministic Plus Stochastic Model With Multiband Excitation for Noise-Controllable Waveform Generation. IEEE ACM Trans. Audio Speech Lang. Process. 30: 865-878 (2022)

[20]   Zhengkun Tian, Jiangyan Yi*, Jianhua Tao, Shuai Zhang, Zhengqi Wen. Hybrid Autoregressive and Non-Autoregressive Transformer Models for Speech Recognition. IEEE Signal Processing Letters. 29: 762-766 (2022)

[21]   Jun Xue, Cunhang Fan, Jiangyan Yi, Jian Zhou, Zhao Lv: Dynamic Ensemble Teacher-Student Distillation Framework for Light-Weight Fake Audio Detection. IEEE Signal Processing Letters. 31: 2305-2309 (2024)

[22]   Ye Bai, Jiangyan Yi*, Jianhua Tao, Zhengqi Wen, Cunhang Fan: A Public Chinese Dataset for Language Model Adaptation. J. Signal Process. Syst. 92(8): 839-851 (2020)

[23]   Xiaohui Zhang, Jiangyan Yi*, Jianhua Tao, Chenglong Wang, Chu Yuan Zhang: Do You Remember? Overcoming Catastrophic Forgetting for Fake Audio Detection. ICML 2023: 41819-41831

[24]   Xiaohui Zhang, Jiangyan Yi*, Chenglong Wang, Chu Yuan Zhang, Siding Zeng, Jianhua Tao: What to Remember: Self-Adaptive Continual Learning for Audio Deepfake Detection. AAAI 2024:19569-19577

[25]   Hao Gu, Jiangyan Yi*, Chenglong Wang, Yong Ren, Jianhua Tao, Xinrui Yan, Yujie Chen, Xiaohui Zhang:Utilizing Speaker Profiles for Impersonation Audio Detection. ACM Multimedia 2024

[26]   Shuai Zhang, Jiangyan Yi*, et al. Code-switching Mediated Sentence-level Semantic Learning. AAAI 2025

[27]   Yujie Chen, Jiangyan Yi*, et al. Region-Based Optimization in Continual Learning for Audio Deepfake Detection. AAAI 2025

·  the EAAI journal Editorial Board, Editor, 2024-

·  Standing Committee Member of the CCF Special Interest Group on Speech Dialogue and Auditory, 2023-

·  Committee Member of the Permanent Body of the NCMMSC, 2018-

·  SL Technical Committee Member of IEEE, 2024-

·  SLA TC Technical Committee Member of APSIPA, 2020-

·  Area Co-Chairs of ICASSP 2024, INTERSPEECH 2020 and 2022

·  Session Chair of INTERSPEECH 2019, 2020 and ICASSP 2021, 2022, 2023, 2024

·  Publication Co-Chairs of APSIPA 2019, NCMMSC 2019

·  Co-Organizers of ADD 2022, 2023

·  Co-Organizers of ACM Multimedia workshop DDAM 2022

·  Co-Organizers of IJCAI workshop DADA 2023

·  the Special Prize of Wu Wenjun Artificial Intelligence Science and Technology Invention Award in 2022 (This is the first time in the 12-year history of the award that a Special Prize has been awarded by the CAAI)

·  the First Prize of the Beijing Invention Patent Award in 2023

·  the First Place in the Few-Shot Track of the Multi-Speaker Multi-Style Voice Cloning Challenge at ICASSP 2021

·  Best Paper Award at the 19th National Conference on Signal Processing 2019

·  Best Student Paper Award at the NCMMSC 2019

·  Best Student Paper Nomination Award at the ISCSLP 2018

·  Best Poster Award at Intel AIDC Beijing 2018

·  1st place in the JD Finance Dialogue Speech Recognition Competition 2018

·  1st place in the Ministry of Industry and Information Technology Personalized Speech Synthesis Competition 2019, 2020

现指导学生

宋瀚林  博士研究生  081203-计算机应用技术  

张豪  博士研究生  081104-模式识别与智能系统  

任勇  博士研究生  081104-模式识别与智能系统  

张一诺  硕士研究生  085400-电子信息  

马一鸣  硕士研究生  085400-电子信息  

程振华  硕士研究生  085400-电子信息  

曾思丁  硕士研究生  085410-人工智能  

徐涛  硕士研究生  085410-人工智能  

杨龙江  硕士研究生  085410-人工智能  

顾浩  博士研究生  081104-模式识别与智能系统