Model Evaluation

效能評估

1 Reading List

2 AI 模型效能評估簡介

當我們建立人工智慧模型後,緊接而來的問題便是模型表現如何,由於表現的面相眾多,我們必須要掌握各種不同的衡量方式,才能使模型衡量面面俱到。在這一系列的影片中,我們將介紹各種面向的人工智慧模型效能評估方法。我們將會先介紹衡量分類模型的指標,並且利用其特性連結到衡量分割模型的指標。最後,我們也會針對模型的變化程度、拓展性,與合理性,介紹一些常見的評估方式。

Slides | 

3 多分類的效能評估

4 Statistical Concepts

Statistics Fundamentals・StatQuest・[Playlist]

Confusion Matrix, Sensitivity, and Specificity




ROC, AUC, pROC

Precision–Recall Curve (PRC) for imbalanced datasets

Cross Validation



SHapley Additive exPlanations (SHAP)

A method in machine learning that explains individual predictions by quantifying the contribution of each feature to a model's output.

An Introduction to SHAP Values and Machine Learning Interpretability・DataCamp・9 min read

SHapley Additive exPlanations (SHAP)・Conor O'Sullivan・Playlist [01:10:42]

Introduction to SHAP with Python・Conor O'Sullivan・10 min read

X XYZ

Classification

Segmentation

Detection

Registration

Prognosis

4 Case Study

Deep Learning to Distinguish Pancreatic Cancer Tissue From Non-cancerous Pancreatic Tissue: a Retrospective Study With Cross-racial External Validation
The Lancet Digital Health, Vol. 2, Iss. 6, pp. 303-313, 2020
https://doi.org/10.1016/S2589-7500(20)30078-9  

Detection of Pancreatic Cancer With Two- and Three-dimensional Radiomic Analysis in a Nationwide Population-based Real-world Dataset
BMC Cancer, 23:58, 2023
https://doi.org/10.1186/s12885-023-10536-8 

Radiomic Analysis of Magnetic Resonance Imaging Predicts Brain Metastases Velocity and Clinical Outcome After Upfront Radiosurgery
Neuro-Oncology Advances, vdaa100, 2020
https://doi.org/10.1093/noajnl/vdaa100

Radiomic Features Distinguish Pancreatic Cancer from Non-cancerous Pancreas.
Radiology: Imaging Cancer, Vol. 3, No. 4, 2021.
https://doi.org/10.1148/rycan.2021210010 

Pancreatic cancer detection on CT scans with deep learning: a nationwide population-based study
Radiology, Vol. 306, No.1, pp. 172-182, 2023.
https://pubs.rsna.org/doi/full/10.1148/radiol.220152