-
Notifications
You must be signed in to change notification settings - Fork 3
/
index.html
300 lines (282 loc) · 20.9 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Explainable AI Course</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="">
<meta name="author" content="Hima Lakkaraju">
<style>
.center{
display: block;
margin-left: auto;
margin-right: auto;
width: 100%;
}
.container {
display: block;
margin-left: auto;
margin-right: auto;
width: 100%;
}
</style>
<link href="css/bootstrap.min.css" rel="stylesheet">
<link href="css/style.css" rel="stylesheet">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.3.1/css/all.css" integrity="sha384-mzrmE5qonljUremFsqc01SB46JvROS7bZs3IO2EmfFsd15uHvIt+Y8vEf7N7fWAU" crossorigin="anonymous">
<!-- HTML5 shim, for IE6-8 support of HTML5 elements -->
<!--[if lt IE 9]>
<script src="js/html5shiv.js"></script>
<![endif]-->
<script type="text/javascript" src="js/jquery.min.js"></script>
<script type="text/javascript" src="js/bootstrap.min.js"></script>
<script type="text/javascript" src="js/scripts.js"></script>
</head>
<body>
<div class="container">
<div class="row clearfix">
<div class="col-md-12 column">
<div class="row clearfix">
<br/>
<h1 align="center"><font color="blue">Explainable Artificial Intelligence</font></h1>
<h3 align="center"><font color="blue">From Simple Predictors to Complex Generative Models</font></h3>
<h3 align="center"><font color="green">Spring 2023, Harvard University</font></h3>
<br/>
<!-- <img alt="" class="pull" src="./images/frontfig-cs330-v2.jpg" style="width: 100%">
<br/> -->
<!--img alt="" class="pull-right" src="./images/CS333-Pull-v1.jpg" style="width: 100%"-->
<!--<div class="carousel slide" id="carousel-25687">
<ol class="carousel-indicators">
<li class="active" data-slide-to="0" data-target="#carousel-25687"> </li>
</ol>
<div class="carousel-inner">
<div class="item active">
<img alt="" align="left" src="./images/frontfig-cs333.png">
</div>
</div>
</div>-->
<h3><font color="blue">Overview:</font></h3>
<p align="justify"> As machine learning models are increasingly being employed to aid critical decision making in high-stakes domains such as healthcare, finance, and law, it becomes important to ensure that relevant stakeholders are able to understand the behavior of these models.
Such an understanding helps determine if, when, and how much to rely on the outputs generated by these models.
This graduate level course aims to familiarize students with the recent advances in the emerging field of eXplainable Artificial Intelligence (XAI).
In this course, we will review seminal position papers in the field, understand the notion of explainability from the perspective of different end users (e.g., doctors, ML researchers/engineers),
discuss in detail different classes of interpretable models and post hoc explanations (e.g., rule-based and prototype-based models, feature attributions, counterfactual explanations, mechanistic interpretability),
and explore the connections between explainability and fairness, robustness, and privacy.
This course will also cover latest research on understanding large language models (e.g., GPT-3) and diffusion models (e.g., DALLE 2), and highlight the unique opportunities and challenges that arise when interpreting the behavior of such large generative models.
<h3><font color="blue">Prerequisites:</font></h3>
<p align="justify">Students are expected to be fluent in basic linear algebra, probability, algorithms, and machine learning. Students are also expected to have programming and software engineering skills to work with data sets using Python, numpy, and sklearn.</p>
<h3><font color="blue">Feedback:</font></h3>
<p align="justify">Please use <a href="https://forms.gle/s9WniBW6oV9PVBLV6">this form</a> to provide any feedback and suggestions about the course.
<div class="hrline"> <hr /> </div>
<div class="row clearfix">
<h3 align="center"> <font color="blue">Course Staff</font> </h3>
<br> <br>
<div class="row clearfix">
<div align="center" class="col-sm-4">
<img class="img-thumbnail" src="images/hima.png" alt="Hima Lakkaraju" width="160" height="160">
<h4><font color="green"><b>Hima Lakkaraju</b></font></h4>
<h5><a href="https://himalakkaraju.github.io/">Webpage</a> | <a href="https://twitter.com/hima_lakkaraju">Twitter</a></h5>
</div>
<div align="center" class="col-sm-4">
<img class="img-thumbnail" src="images/jiaqi.jpeg" alt="Ike Lage" width="140" height="140">
<h4><font color="green"><b>Jiaqi Ma</b></font></h4>
<h5> <a href="https://www.jiaqima.com/">Webpage</a> | <a href="https://twitter.com/Jiaqi_Ma_">Twitter</a> </h5>
</div>
<div align="center" class="col-sm-4">
<img class="img-thumbnail" src="images/suraj.jpeg" alt="Ike Lage" width="140" height="140">
<h4><font color="green"><b>Suraj Srinivas</b></font></h4>
<h5> <a href="https://suraj-srinivas.github.io/">Webpage</a> | <a href="https://twitter.com/Suuraj">Twitter</a> </h5>
</div>
</div>
</div>
</div>
<div class="hrline"> <hr /> </div> <br>
<h3 align="center"><font color="blue">Schedule</font></h3>
<table class="table">
<thead>
<tr>
<th style="width: 20%">Week</th>
<th style="width: 40%">Topic</th>
<th style="width: 40%">Readings</th>
</tr>
</thead>
<tbody>
<tr>
<td><br> <b>Week 1</b></td>
<td> <br> <font color="green">Introduction</font> </td>
<td> <br> Doshi-Velez and Kim, 2017, <a href="https://arxiv.org/abs/1702.08608">Towards a Rigorous Science of Interpretable Machine Learning</a> <br> <br>
Weller, 2019, <a href="https://arxiv.org/abs/1708.01870">Transparency: Motivation and Challenges</a> <br>
Lipton, 2017, <a href="https://arxiv.org/abs/1606.03490">The Mythos of Model Interpretability</a> <br> <br>
[ <a href="slides/Lecture_1.pptx">Slides 1</a> | <a href="slides/Lecture_2.pptx">Slides 2</a> ] <br> <br>
</td>
</tr>
<tr>
<td> <br> <b>Week 2</b></td>
<td> <br><font color="green">Human Factors in Explainability </font> </td>
<td>
<br> Hong et. al., 2020, <a href="https://arxiv.org/abs/2004.11440">Human Factors in Model Interpretability: Industry Practices, Challenges</a> <br>
Kaur et. al., 2020, <a href="http://www-personal.umich.edu/~harmank/Papers/CHI2020_Interpretability.pdf">Interpreting Interpretability: Understanding Data Scientists’ Use of Interpretability Tools for Machine Learning</a> <br> <br>
Lage et. al., 2019, <a href="https://ojs.aaai.org/index.php/HCOMP/article/view/5280/5132">Human Evaluation of Models Built for Interpretability</a> <br>
Poursabzi-Sangdeh et. al., 2021, <a href="https://arxiv.org/abs/1802.07810"> Measuring and Manipulating Model Interpretability</a> <br> <br>
[ <a href="slides/Lecture_3.pdf">Slides 1</a> | <a href="slides/Lecture_4.pptx">Slides 2</a> ] <br> <br>
</td>
</tr>
<tr>
<td> <br> <b>Week 3</b></td>
<td> <br> <font color="green">Inherently Interpretable Models</font> </td>
<td> <br> Letham and Rudin, 2015, <a href="https://arxiv.org/abs/1511.01644">Interpretable Classifiers Using Rules and Bayesian Analysis</a> <br>
Lakkaraju et. al., 2016, <a href="https://www-cs-faculty.stanford.edu/people/jure/pubs/interpretable-kdd16.pdf">Interpretable Decision Sets</a> <br> <br>
Caruana et. al., 2015, <a href="http://people.dbmi.columbia.edu/noemie/papers/15kdd.pdf">Intelligible Models for Healthcare</a> <br>
Li et. al., 2017, <a href="https://arxiv.org/abs/1710.04806">Deep Learning for Case Based Reasoning Through Prototypes</a> <br> <br>
[ <a href="slides/Lecture_5.pptx">Slides 1</a> | <a href="slides/Lecture_6.pptx">Slides 2</a> ] <br> <br>
<font color="blue">Additional Readings:</font> <br>
Ustun and Rudin, 2019, <a href="https://arxiv.org/abs/1610.00168">Learning Optimized Risk Scores</a> <br> <br>
</td>
</tr>
<tr>
<td><br> <b>Week 4</b></td>
<td> <br> <font color="green">Post hoc Explanations: Feature Attributions</font> </td>
<td> <br> Ribeiro et. al., 2016, <a href="https://arxiv.org/abs/1602.04938">Why should I trust you? Explaining the Predictions of Any Classifier</a> <br>
Lundberg and Lee, 2017, <a href="https://papers.nips.cc/paper/7062-a-unified-approach-to-interpreting-model-predictions.pdf">A Unified Approach to Interpreting Models</a> <br> <br>
Smilkov et. al., 2017, <a href="https://arxiv.org/abs/1706.03825">Smoothgrad: Removing noise by adding noise</a> <br>
Sundararajan et. al., 2017, <a href="https://arxiv.org/abs/1703.01365">Axiomatic Attribution for Deep Networks</a> <br> <br>
[ <a href="slides/Lecture_7_LIME.pptx">Slides 1</a> | <a href="slides/Lecture_7_SHAP.pptx">Slides 2</a> | <a href="slides/Lecture_8_SmoothGrad.pptx">Slides 3</a> | <a href="slides/Lecture_8_IntegratedGradients.pptx">Slides 4</a> ] <br> <br>
<font color="blue">Additional Readings:</font> <br>
Shrikumar et. al., 2019, <a href="https://arxiv.org/abs/1704.02685">Learning Important Features through Propagating Activation Differences</a> <br> <br>
</td>
</tr>
<tr>
<td><br> <b>Week 5</b></td>
<td> <br> <font color="green">Pitfalls, Challenges, and Evaluation of Feature Attributions</font> </td>
<td> <br> Slack and Hilgard et. al., 2020, <a href="https://arxiv.org/abs/1911.02508">Fooling LIME and SHAP</a> <br>
Dombrowski et. al., 2019, <a href="https://arxiv.org/abs/1906.07983">Explanations can be manipulated and geometry is to blame</a> <br> <br>
Adebayo et. al., 2018, <a href="https://arxiv.org/abs/1810.03292">Sanity Checks for Saliency Maps</a> <br>
Agarwal et. al., 2023, <a href="https://arxiv.org/abs/2206.11104">OpenXAI: Towards a Transparent Evaluation of Model Explanations</a> <br> <br>
[ <a href="slides/Lecture_9_Fooling_LIME_SHAP.pptx">Slides 1</a> | <a href="slides/Lecture_9_Explanations_Manipulated.pptx">Slides 2</a> | <a href="slides/Lecture_10_Adebayo_Sanity_Checks.pptx">Slides 3</a> | <a href="slides/Lecture_10_Agarwal_OpenXAI.pptx">Slides 4</a> ]<br> <br>
<font color="blue">Additional Readings:</font> <br>
Krishna and Han et. al., 2023, <a href="https://arxiv.org/abs/2202.01602">The Disagreement Problem in Explainable Machine Learning</a> <br>
Rudin, 2019, <a href="https://arxiv.org/abs/1811.10154">Stop Explaining Black Box Models</a> <br>
Chen and Subhas et. al., 2022, <a href="https://arxiv.org/abs/2211.05667">What Makes a Good Explanation? A Harmonized View of Properties of Explanations</a> <br>
Chen et. al., 2022, <a href="https://arxiv.org/abs/2206.02256">Use-Case-Grounded Simulations for Explanation Evaluation</a> <br> <br>
</td>
</tr>
<tr>
<td><br> <b>Week 6</b></td>
<td> <br> <font color="green">Counterfactual Explanations (or) Algorithmic Recourse</font> </td>
<td> <br> Wachter et. al., 2018, <a href="https://arxiv.org/abs/1711.00399">Counterfactual Explanations Without Opening the Black Box</a> <br>
Karimi et. al., 2020, <a href="https://arxiv.org/abs/2002.06278">Algorithmic Recourse: From Counterfactual Explanations to Interventions</a> <br> <br>
Upadhyay et. al., 2021, <a href="https://arxiv.org/abs/2102.13620">Towards Robust and Reliable Algorithmic Recourse</a> <br>
Pawelczyk et. al., 2022, <a href="https://arxiv.org/abs/2203.06768">Probabilistically Robust Recourse: Navigating the Trade-offs between Costs and Robustness in Algorithmic Recourse</a> <br> <br>
[ <a href="slides/Lecture_11_Wachter_Algorithmic_Recourse.pptx">Slides 1</a> | <a href="slides/Lecture_11_Karimi_Causal_Recourse.pptx">Slides 2</a> | <a href="slides/Lecture_12_ROAR.pptx">Slides 3</a> | <a href="slides/Lecture_12_PROBE.pptx">Slides 4</a> ] <br> <br>
<font color="blue">Additional Readings:</font> <br>
Pawelczyk et. al., 2020, <a href="https://arxiv.org/abs/1910.09398">Learning Model-Agnostic Counterfactual Explanations for Tabular Data</a> <br>
Rawal et. al., 2020, <a href="https://arxiv.org/abs/2009.07165">Beyond Individualized Recourse: Interpretable and Interactive Summaries of Actionable Recourses</a> <br>
Ustun et. al., 2019, <a href="https://arxiv.org/abs/1809.06514"> Actionable Recourse in Linear Classification</a> <br> <br>
</td>
</tr>
<tr>
<td><br><b>Week 7</b></td>
<td> <br> <font color="green">Attention and Concept Based Explanations</font> </td>
<td> <br> Mullenbach et. al., 2018, <a href="https://www.aclweb.org/anthology/N18-1100.pdf">Explainable Prediction of Medical Codes from Clinical Text</a> <br>
Jain and Wallace, 2019, <a href="https://arxiv.org/abs/1902.10186">Attention is not Explanation</a> <br> <br>
Bau and Zhou et. al., 2017, <a href="http://netdissect.csail.mit.edu/final-network-dissection.pdf">Network Dissection: Quantifying Interpretability of Deep Visual Representations</a> <br>
Kim et. al., 2018, <a href="https://arxiv.org/abs/1711.11279">Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors</a> <br> <br>
[ <a href="slides/Lecture_13_Convolutional_Attention.pptx">Slides 1</a> | <a href="slides/Lecture_13_Attention_Not_Explanation.pptx">Slides 2</a> | <a href="slides/Lecture_14_Network_Dissection.pdf">Slides 3</a> | <a href="slides/Lecture_14_TCAV.pptx">Slides 4</a> ] <br> <br>
</td>
</tr>
<tr>
<br>
<td><br> <b>Week 8</b></td>
<td> <br> <font color="green">Data Attribution and Interactive Explanations</font> </td>
<td> <br> Koh et. al., 2017, <a href="https://arxiv.org/abs/1703.04730">Understanding Black Box Predictions via Influence Functions</a> <br>
Ghorbani et. al., 2019, <a href="https://arxiv.org/abs/1904.02868">What is your data worth? Equitable Valuation of Data</a> <br> <br>
Ghai et. al. 2020, <a href="https://arxiv.org/abs/2001.09219">Explainable Active Learning (XAL): Toward AI Explanations as Interfaces for Machine Teachers</a> <br>
Slack et. al., 2022, <a href="https://arxiv.org/abs/2207.04154">TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversations</a> <br> <br>
[ <a href="slides/Lecture_15_Influence_Function.pptx">Slides 1</a> | <a href="slides/Lecture_15_DataShapley.pdf">Slides 2</a> | <a href="slides/Lecture_16_XAL.pptx">Slides 3</a> | <a href="slides/Lecture_16_Talk_to_Model.pptx">Slides 4</a> ] <br> <br>
</td>
</tr>
<tr>
<br>
<td><br> <b>Week 9</b></td>
<td> <br> <font color="green">Theory of Explainability and Interpreting Generative Models</font> </td>
<td> <br> Covert et. al., 2021, <a href="https://www.jmlr.org/papers/volume22/20-1316/20-1316.pdf">Explaining by Removing: A Unified Framework for Model Explanation</a> <br>
Han et. al., 2022, <a href="https://arxiv.org/abs/2206.01254">Which Explanation Should I Choose? A Function Approximation Perspective to Characterizing Post hoc Explanations</a> <br> <br>
Shen et. al., 2020, <a href="https://arxiv.org/abs/1907.10786">Interpreting the Latent Space of GANs for Semantic Face Editing</a> <br>
Harkonen et. al., 2020, <a href="https://arxiv.org/abs/2004.02546">GANSpace: Discovering Interpretable GAN Controls</a> <br> <br>
[ <a href="slides/Lecture_17_Explaining_by_Removing.pptx">Slides 1</a> | <a href="slides/Lecture_17_LFA.pdf">Slides 2</a> | <a href="slides/Lecture_18_GANSpace.pdf">Slides 3</a> | <a href="slides/Lecture_18_Latent_Space_GAN.pdf">Slides 4</a> ] <br> <br>
<font color="blue">Additional Readings:</font> <br>
Li et. al., 2021, <a href="https://arxiv.org/abs/2011.01205">A Learning Theoretic Perspective on Local Explainability</a> <br> <br>
</td>
</tr>
<tr>
<td><br> <b>Week 10</b></td>
<td> <br> <font color="green">Connections with Robustness, Privacy, Fairness, and Unlearning</font> </td>
<td> <br> Shah et. al., 2021, <a href="https://arxiv.org/abs/2102.12781">Do Input Gradients Highlight Discriminative Features?</a> <br>
Pawelczyk et. al., 2023, <a href="https://arxiv.org/abs/2211.05427">On the Privacy Risks of Algorithmic Recourse</a> <br> <br>
Begley et. al., 2020, <a href="https://arxiv.org/abs/2010.07389">Explainability for Fair Machine Learning</a> <br>
Krishna et. al., 2023, <a href="https://arxiv.org/abs/2302.04288">Towards Bridging the Gaps between the Right to Explanation and the Right to be Forgotten</a> <br> <br>
[ <a href="slides/Lecture_19_DiffROAR.pptx">Slides 1</a> | <a href="slides/Lecture_19_Privacy_Risk.pdf">Slides 2</a> | <a href="slides/Lecture_20_Explainability_Fair_ML.pdf">Slides 3</a> | <a href="slides/Lecture_20_ROCERF.pdf">Slides 4</a> ] <br> <br>
<font color="blue">Additional Readings:</font> <br>
Dai et. al., 2022, <a href="https://arxiv.org/abs/2011.01205">Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanations</a> <br> <br>
</td>
</tr>
<tr>
<td><br> <b>Week 11</b></td>
<td> <br> <font color="green">Mechanistic Interpretability and Compiled Transformers</font> </td>
<td> <br> Olah, 2020, <a href="https://distill.pub/2020/circuits/zoom-in/">An Introduction to Circuits</a> <br>
Olah, 2022, <a href="https://transformer-circuits.pub/2022/mech-interp-essay/index.html">Mechanistic Interpretability, Variables, and the Importance of Interpretable Bases</a> <br> <br>
Lindner et. al., 2023, <a href="https://arxiv.org/abs/2301.05062">Tracr: Compiled Transformers as a Laboratory for Interpretability</a> <br> <br>
[ <a href="slides/Lecture_21_Mechanistic_Interpretability.pptx">Slides 1</a> | <a href="slides/Lecture_22_Tracr.pdf">Slides 2</a> ] <br> <br>
</td>
</tr>
<tr>
<td><br> <b>Week 12</b></td>
<td> <br> <font color="green">Understanding and Reasoning in Large Language Models</font> </td>
<td> <br> Wei et. al., 2022, <a href="https://arxiv.org/abs/2201.11903">Chain of Thought Prompting Elicits Reasoning in Large Language Models</a> <br>
Lampinen et. al., 2022, <a href="https://arxiv.org/abs/2204.02329">Can language models learn from explanations in context?</a> <br> <br>
Rajani et. al., 2019, <a href="https://arxiv.org/abs/1906.02361">Explain Yourself! Leveraging Language Models for Common Sense Reasoning</a> <br>
Yin et. al., 2022, <a href="https://arxiv.org/abs/2202.10419">Interpreting Language Models with Contrastive Explanations</a> <br> <br>
[ <a href="slides/Lecture_23_CoT.pdf">Slides 1</a> | <a href="slides/Lecture_23_Explanations_In_Context.pptx">Slides 2</a> | <a href="slides/Lecture_24_Explain_Yourself.pptx">Slides 3</a> | <a href="slides/Lecture_24_Contrastive_Explanation.pdf">Slides 4</a> ] <br> <br>
<font color="blue">Additional Readings:</font> <br>
Bills et. al., 2023, <a href="https://openaipublic.blob.core.windows.net/neuron-explainer/paper/index.html">Language Models can Explain Neurons in Language Models</a> <br> <br>
</td>
</tr>
<tr>
<td><br> <b>Week 13</b></td>
<td> <br> <font color="green">Understanding and Reasoning in Other Large Models</font> </td>
<td> <br> McGrath et. al., 2022, <a href="https://arxiv.org/abs/2111.09259">Acquisition of Chess Knowledge in Alpha Zero</a> <br> <br>
Tang et. al., 2022, <a href="https://arxiv.org/abs/2210.04885">What the DAAM: Interpreting Stable Diffusion Using Cross Attention</a> <br>
Cho et. al., 2022, <a href="https://arxiv.org/abs/2202.04053">DALL-EVAL: Probing the Reasoning Skills and Social Biases of Text-to-Image Generative Models</a> <br> <br>
[ <a href="slides/Lecture_25_AlphaZero.pptx">Slides 1</a> | <a href="slides/Lecture_25_DAAM.pptx">Slides 2</a> | <a href="slides/Lecture_26_DALL_EVAL.pptx">Slides 3</a> ] <br> <br>
</td>
</tr>
</tbody>
</table>
<br>
<div class="hrline"> <hr /> </div>
<br>
</div>
</div>
</div>
<br>
<!-- The footer -->
<div class="hrline"><hr></div><br>
<div class="row clearfix">
<div class="col-md-3 column">
</div>
<div class="col-md-6 column">
<div class = "text-center">
<p>
© Hima Lakkaraju 2023
</p>
</div>
</div>
<div class="col-md-3 column">
</div>
</div>
<br>
<br>
<br>
<!-- End of document -->
</div>
</body>
</html>