This is the official code release for "DEPICT: Diffusion-Enabled Permutation Importance for Image Classification Tasks" (ECCV 2024).
Explanation methods for image classifiers have historically been limited to the pixel space. On the contrary, tabular-based models can be explained using permutation importance. We propose extending permutation importance to generate concept-based explanations for Image-based classifiers. Rather than trying to figure out which pixels in a real image to manipulate, we propose using text-conditioned diffusion models to permute concepts in text-space, and then map such concepts to the image space.
We provide a full demo of the method, as well as code to recreate the results from the paper's experimence.
[7/2/2024]: We will be updating the entire codebase with all available code in the coming weeks.
Please reach out to sjabbour at umich dot edu or file a Github issue if you have any questions about our work. Thank you!