机器学习理论与方法及其在大模型中的延申，包括但不限于：

可信机器学习

• 指标评估、指标优化中的可信理论与方法
• 大模型安全

大模型高效压缩

• 模型蒸馏理论与方法
• 投机解码
• Token 压缩

每年招收硕士、博士生各一名，主要从事机器学习相关研究

2023-现在，副教授，中国科学院大学，计算机科学与技术学院

2021-2023，博士后，中国科学院大学，计算机科学与技术学院

2017-2021，博士，中国科学院信息工程研究所

2013-2017，硕士，北京科技大学

在多个机器学习、人工智能领域顶级会议担任领域主席等职务：

• ICML/NeurIPS/ICLR 领域主席
• IJCAI 2021 高级程序委员会委员（SPC） 
• ICML 2021 专家审稿人（ER） 

在JCR1区期刊担任编委、领域编辑（AE）：

• Information Fusion IF:15.5
• Pattern Recognition IF:7.6
• Neural Networks IF：6.3

在多个机器学习、人工智能领域顶级期刊担任审稿人 

• IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI, IF: 23.6)
• IEEE Transactions on Image Processing (T-IP, IF: 10.6) 
• ​Transactions on Machine Learning Research (TMLR)

见主页https://joshuaas.github.io/

Zhiyong Yang, Qianqian Xu, Shilong Bao, Peisong Wen, Yuan He, Xiaochun Cao, and Qingming Huang. AUC-Oriented Domain Adaptation: From Theor y to Algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI, IF:23.6, doi: 10.1109/TPAMI.2023.3303943), 2023.【CCF-A类期刊】

 Zhiyong Yang, Qianqian Xu, Wenzheng Hou, Shilong Bao, Yuan He, Xiaochun Cao, and Qingming Huang. Revisiting AUC-oriented Adversarial Training with Loss-Agnostic Perturbations. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI, IF:23.6, doi: 10.1109/ TPAMI.2023.3303934), 2023.【CCF-A类期刊】 

Zhiyong Yang, Qianqian Xu, Shilong Bao, Yuan He, Xiaochun Cao, and Qingming Huang. Optimizing Two-way Partial AUC with an End-to-end Framework . IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI, IF:23.6, doi: 10.1109/TPAMI.2022.3185311), 2022.【CCF-A类期刊】 

Zhiyong Yang, Qianqian Xu, Shilong Bao, Xiaochun Cao, and Qingming Huang. Learning with Multiclass AUC: Theory and Algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI, IF:23.6, doi: 10.1109/TPAMI.2021.3101125), 2021.【CCF-A类期刊】

Zhiyong Yang, Qianqian Xu, Xiaochun Cao, and Qingming Huang. Task-Feature Collaborative  Learning with Application to Personalized Attribute Prediction. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI, IF:23.6, doi: 10.1109/TPAMI.2020.2991344), 2021.【CCF-A类期 刊】 

Zhiyong Yang, Qianqian Xu, Shilong Bao, Yuan He, Xiaochun Cao and Qingming Huang. When All We Need is a Piece of the Pie: A Generic Framework for Optimizing Two-way Partial AUC. International Conference on Machine Learning (ICML) 2021 (Long Talk, 录取率3%).【CCF-A类会议】

Shilong Bao, Qianqian Xu, Zhiyong Yang, Yuan He, Xiaochun Cao and Qingming Huang. The Minority Matters: A Diversity-Promoting Collaborative Metric Learning Algorithm. Annual Conference on Neural Information Processing Systems (NeurIPS), 2022 (Oral，录取率1.9%). 【CCF-A类会议】

Zitai Wang, Qianqian Xu, Zhiyong Yang, Yuan He, Xiaochun Cao and Qingming Huang. OpenAUC: Towards AUC-Oriented Open-Set Recognition. Annual Conference on Neural Information Processing Systems (NeurIPS), 2022 (Spotlight，录取率5%).【CCF-A类会议】

Yangbangyan Jiang, Qianqian Xu, Zhiyong Yang, Xiaochun Cao and Qingming Huang. DM2C: Deep Mixed-Modal Clustering. Annual Conference on Neural Information Processing Systems (NeurIPS), 5880–5890, 2019. (Spotlight，录取率2.4%).【CCF-A类会议】

Recently, machine learning and deep learning technologies have been successfully employed in many complicated high-stake decision-making applications such as disease prediction, fraud detection, outlier detection, and criminal justice sentencing. All these applications share a common trait known as risk-aversion in economics and finance terminologies. In other words, the decision-makers tend to have an extremely low risk tolerance. Under this context, the decision-makers will carefully choose their decision parameter to meet the specific requirement. Consequently, the decision parameters for train- and test- might be quite different. To mitigate the decision parameter shift problem, I'm seeking for new decision-invariant machine learning machanisms , on top of which we develop a new framework called Xcurve

The goal of X-curve learning is to learn high-quality models that can adapt to different decision conditions. Inspired by the fundamental principle of the well-known AUC optimization, our library provides a systematic solution to optimize the area under different kinds of performance curves. To be more specific, the performance curve is formed by a plot of two performance functions $x(\lambda)$, $y(\lambda)$ of decision parameter $\lambda$. The area under a performance curve becomes the integral of the performance over all possible choices of different decision conditions. In this way, the learning systems are only required to optimize a decision-invariant metric to avoid the risk aversion issue

Four Kinds of Performance Curves

We are seeking for new principled method to make the current machine learning system trustworthy (e.g. Robustness against Adversarial attacks, OOD examples). On top of Xcurve, I'm especially interested in (a) how to design performance-based metrics for trustworthy machine learning, and (b) how to use the SOTA models and idea of trustworthy machine learning to improve the Xcurve Framework.

Long-tail learning is one of the most challenging problems in machine learning, which aims to train well-performing models from a large number of examples that follow a highly imbalanced class distribution. We find that the long-tail problem could be mitigated by adjusting the optimal decision rule. On top of the Xcurve framework, we are interested in (a) how to design distribution-invariant metrics for long-tail learning to deal with different long-tail distributions, and (b) how to directly optimize such metrics efficiently.

• CCF

  From AUC Optimization to XCurve: A Study of Decision-invariant Learning, 2023

  
• TECHBEAT

  Efficient Partial AUC: Optimization Algorithm and Theory, 2023

  

  • Zhiyong Yang, Qianqian Xu, Shilong Bao, Yuan He, Xiaochun Cao and Qingming Huang. Optimizing Two-way Partial AUC with an End-to-end Framework. TPAMI, 2022.
  • Huiyang Shao, Qianqian Xu, Zhiyong Yang, Shilong Bao, Qingming Huang. Asymptotically Unbiased Instance-wise Regularized Partial AUC Optimization: Theory and Algorithm. NeurIPS, 2022.
• CSIG

  The 19th Young Scientists Conference of The China Society of Image and Graphics, 2023

  
• HKBU

  XCurve: Machine Learning with Decision-Invariant Performance Curve Metrics, 2023

  
• HKBU

  TMLR Young Scientist Seminar@HKBU, 2022   [pdf]

  

  In this talk, I provided a brief introduction of AUROC as a decision-invariant metric, reviewed the history of AUROC and AUROC-oriented machine learning methods, and then presented some of our latest work on AUC-oriented learning with complicated scenarios (Partial AUROC Optimization, Adversarial Learning, and Recommendation):

  • Zhiyong Yang, Qianqian Xu, Shilong Bao, Yuan He, Xiaochun Cao and Qingming Huang. Optimizing Two-way Partial AUC with an End-to-end Framework. TPAMI, 2022.
  • Wenzheng Hou, Qianqian Xu, Zhiyong Yang, Shilong Bao, Yuan He and Qingming Huang. AdAUC: End-to-end Adversarial AUC Optimization Against Long-tail Problems. ICML2022
  • Shilong Bao, Qianqian Xu, Zhiyong Yang, Xiaochun Cao and Qingming Huang. Rethinking Collaborative Metric Learning: Toward an Efficient Alternative without Negative Sampling. IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2021.
• TECHBEAT

  AUC Optimization Under Complicate Scenarios, 2021

  

  • Zhiyong Yang, Qianqian Xu, Shilong Bao, Yuan He, Xiaochun Cao and Qingming Huang. When All We Need is a Piece of the Pie: A Generic Framework for Optimizing Two-way Partial AUC. International Conference on Machine Learning (ICML), 11820–11829, 2021
  • Zhiyong Yang, Qianqian Xu, Shilong Bao, Xiaochun Cao and Qingming Huang. Learning with Multiclass AUC: Theory and Algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2021. (Early Access)
• AI TIME

  When All We Need is a Piece of the Pie: A Generic Framework for Optimizing Two-way Partial AUC, 2021

  

  The Area Under the ROC Curve (AUC) is a crucial metric for machine learning, which evaluates the average performance over all possible True Positive Rates (TPRs) and False Positive Rates (FPRs). Based on the knowledge that a skillful classifier should simultaneously embrace a high TPR and a low FPR, we turn to study a more general variant called Two-way Partial AUC (TPAUC), where only the region with TPR ≥ α, FPR ≤ β is included in the area. Moreover, a recent work shows that the TPAUC is essentially inconsistent with the existing Partial AUC metrics where only the FPR range is restricted, opening a new problem to seek solutions to leverage high TPAUC. Motivated by this, we present the first trial in this paper to optimize this new metric. The critical challenge along this course lies in the difficulty of performing gradient-based optimization with end-to-end stochastic training, even with a proper choice of surrogate loss. To address this issue, we propose a generic framework to construct surrogate optimization problems, which supports efficient end-to-end training with deep-learning. Moreover, our theoretical analyses show that: 1) the objective function of the surrogate problems will achieve an upper bound of the original problem under mild conditions, and 2) optimizing the surrogate problems leads to good generalization performance in terms of TPAUC with a high probability. Finally, empirical studies over several benchmark datasets speak to the efficacy of our framework.

  • Zhiyong Yang, Qianqian Xu, Shilong Bao, Yuan He, Xiaochun Cao and Qingming Huang. When All We Need is a Piece of the Pie: A Generic Framework for Optimizing Two-way Partial AUC. International Conference on Machine Learning (ICML), 11820–11829, 2021
• ICML

  When All We Need is a Piece of the Pie: A Generic Framework for Optimizing Two-way Partial AUC (Long Talk 3%), 2021

  

  The Area Under the ROC Curve (AUC) is a crucial metric for machine learning, which evaluates the average performance over all possible True Positive Rates (TPRs) and False Positive Rates (FPRs). Based on the knowledge that a skillful classifier should simultaneously embrace a high TPR and a low FPR, we turn to study a more general variant called Two-way Partial AUC (TPAUC), where only the region with TPR ≥ α, FPR ≤ β is included in the area. Moreover, a recent work shows that the TPAUC is essentially inconsistent with the existing Partial AUC metrics where only the FPR range is restricted, opening a new problem to seek solutions to leverage high TPAUC. Motivated by this, we present the first trial in this paper to optimize this new metric. The critical challenge along this course lies in the difficulty of performing gradient-based optimization with end-to-end stochastic training, even with a proper choice of surrogate loss. To address this issue, we propose a generic framework to construct surrogate optimization problems, which supports efficient end-to-end training with deep-learning. Moreover, our theoretical analyses show that: 1) the objective function of the surrogate problems will achieve an upper bound of the original problem under mild conditions, and 2) optimizing the surrogate problems leads to good generalization performance in terms of TPAUC with a high probability. Finally, empirical studies over several benchmark datasets speak to the efficacy of our framework.

  • Zhiyong Yang, Qianqian Xu, Shilong Bao, Yuan He, Xiaochun Cao and Qingming Huang. When All We Need is a Piece of the Pie: A Generic Framework for Optimizing Two-way Partial AUC. International Conference on Machine Learning (ICML), 11820–11829, 2021
• NEURIPS

  DM2C: Deep Mixed-Modal Clustering (Spotlight 2.4%), 2019

  

  Data exhibited with multiple modalities are ubiquitous in real-world clustering tasks. Most existing methods, however, pose a strong assumption that the pairing information for modalities is available for all instances. In this paper, we consider a more challenging task where each instance is represented in only one modality, which we call mixed-modal data. Without any extra pairing supervision across modalities, it is difficult to find a universal semantic space for all of them. To tackle this problem, we present an adversarial learning framework for clustering with mixed-modal data. Instead of transforming all the samples into a joint modalityindependent space, our framework learns the mappings across individual modality spaces by virtue of cycle-consistency. Through these mappings, we could easily unify all the samples into a single modality space and perform the clustering. Evaluations on several real-world mixed-modal datasets could demonstrate the superiority of our proposed framework.

  • Jiang, Yangbangyan, et al. Dm2c: Deep mixed-modal clustering. Advances in Neural Information Processing Systems 32 (2019).