Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[JSAI24] C3-LRP: Visual Explanation Generation based on Layer-Wise Relevance Propagation for ResNet

[JSAI24] C3-LRP: Visual Explanation Generation based on Layer-Wise Relevance Propagation for ResNet

More Decks by Semantic Machine Intelligence Lab., Keio Univ.

Other Decks in Technology

Transcript

  1. 𝐶!-LRP : Visual Explanation Generation based on Layer-Wise Relevance Propagation

    for ResNet Felix Doublet, Seitaro Otsuki, Tsumugi Iida, Komei Sugiura Keio University
  2. - 2 - Summary : We adapted LRP to handle

    ResNet models • A new Layer-wise Relevance Propagation (LRP) rule to handle residual connections of ResNet • 𝑪𝟑-LRP that selects the most noteworthy area based on the generated relevance regions. We introduce : LRP [Bach+, PLoS15] GradCAM [Selvaraju+, ICCV17] Ours
  3. - 3 - Background : The need for an eXplainable

    AI Deep Neural Networks (DNNs) = black box nature : • Lack of trust by end-users (medical analysis, autonomous vehicles) • Legal need (RGPD, AI Act) to explain decisions Anastasia Grivia : https://impact.universityofgalway.ie/articles/to- black-box-or-not-to-black-box/ ⇒ Explainable AI (XAI) Layer-Wise Relevance Propagation [Bach+, PLoS15] : • Strong theoretical background • Transparency in Decision-Making • Interpretable Relevance scores ⇒ White Box Analysis
  4. - 4 - Background : Generating visual explanations for ResNet

    with LRP Input Image n Layer-Wise Relevance Propagation (LRP) fails to generate insightful explanations for ResNet models LRP Output LRP FAIL !
  5. - 5 - n Layer-Wise Relevance Propagation (LRP) fails to

    generate insightful explanations for ResNet models LRP Output Residual connections ResNet[He+, CVPR16] No designed rule ✕ Background : Generating visual explanations for ResNet with LRP ???
  6. - 6 - n Layer-Wise Relevance Propagation (LRP) fails to

    generate insightful explanations for ResNet models LRP Output Residual connections ResNet[He+, CVPR16] Tailored rule ✓ Our method Background : Generating visual explanations for ResNet with LRP
  7. - 7 - GradCAM [Selvaraju+, ICCV17] Related work : Back-Propagation

    methods GradCAM [Selvaraju+, ICCV17] Gradients flowing to produce a coarse localization map. Input x Gradient [Shrikumar+, ICLR18] Multiplying the input image by the gradient of the output. LRP [Bach+, PLoS15] Propagating the prediction backward through the network layers.
  8. - 8 - Related work : Layer-Wise Relevance Propagation -

    an established theoretical framework LRP [Bach+, PloS15] n Layer-wise Relevance Propagation [Bach+, PLoS15] n Established theoretical framework n Transparent computational processes
  9. - 9 - : the relevance score of neuron 𝑘

    at layer 𝑙 + 1 : activation of neuron 𝑘 : weight of the connection between neuron 𝑖 and neuron 𝑘 Related work : Layer-Wise Relevance Propagation - an established theoretical framework LRP [Bach+, PloS15]
  10. - 10 - : the relevance score of neuron 𝑘

    at layer 𝑙 + 1 : activation of neuron 𝑘 : weight of the connection between neuron 𝑖 and neuron 𝑘 Related work : Layer-Wise Relevance Propagation - an established theoretical framework LRP [Bach+, PloS15] A rule must be defined for each type of layer
  11. - 11 - Proposed Method : The need for a

    new rule for the ResNet models ResNet50 architecture :
  12. - 12 - Proposed Method : The need for a

    new rule for the ResNet models ResNet50 architecture :
  13. - 13 - One bottleneck block: ResNet50 architecture : Proposed

    Method : The need for a new rule for the ResNet models Need for a tailored backpropagation rule
  14. - 14 - IDEA : Considering each bottleneck block as

    a single dense layer Proposed Method : Relevance backpropagation for Bottleneck layers
  15. - 15 - : Relevance of the layer L IDEA

    : Considering each bottleneck block as a single dense layer Proposed Method : Relevance backpropagation for Bottleneck layers
  16. - 16 - : Input of the Bottleneck block :

    Output of the Bottleneck block 𝐶 : Output channels 𝑈 : Height 𝑉 : Width : Avoiding a zero division IDEA : Considering each bottleneck block as a single dense layer : Relevance of the layer L Proposed Method : Relevance backpropagation for Bottleneck layers
  17. - 17 - Choice Contour Component (𝐶") : Removing unnecessary

    information Relevance map (LRP output) Proposed Method : Choice Contour Component : 𝑪𝟑
  18. - 18 - Choice Contour Component (𝐶") : Removing unnecessary

    information Relevance map (LRP output) Proposed Method : Choice Contour Component : 𝑪𝟑 1st Contour (findContours) 1st Component (connectedComponents) C1C [Iida+, SIG-AM23] TRUE Two binary masks are created
  19. - 19 - Choice Contour Component (𝐶") : Removing unnecessary

    information Relevance map (LRP output) Proposed Method : Choice Contour Component : 𝑪𝟑 1st Contour (findContours) 1st Component (connectedComponents) C1C [Iida+, SIG-AM23] AND TRUE
  20. - 20 - Choice Contour Component (𝐶") : Removing unnecessary

    information Relevance map (LRP output) Proposed Method : Choice Contour Component : 𝑪𝟑 1st Contour (findContours) 1st Component (connectedComponents) C1C [Iida+, SIG-AM23] AND Final Relevance map Only the pixels from BOTH the 1st contour and the 1st component are kept TRUE
  21. - 21 - Experimental setup : The CUB-200-2011 dataset n

    The CUB-200-2011 dataset [Wah+, 11] : n 11,788 images of birds n 200 classes n Split : 5,794:200:5,794 (train:val:test) Laysan Albatros Least Auklet
  22. Method Acc. ↑ Insertion↑ Deletion↓ ID score↑ RISE [Petsuik+, RMVC18]

    0.815±0.001 0.371±0.015 0.043±0.004 0.328±0.004 GradCAM [Selvaraju+, ICCV19] 0.815±0.001 0.466±0.019 0.156±0.008 0.310±0.020 LRP [Bach+, PLoS15] 0.815±0.001 0.063±0.007 0.051±0.006 0.011±0.001 ABN [Fukui+, CVPR19] 0.642±0.009 0.282±0.052 0.075±0.011 0.207±0.054 Ours 0.815±0.001 0.685±0.015 0.017±0.001 0.668±0.015 - 22 - Quantitative results : Our method outperformed the baseline methods
  23. - 26 - Qualitative results : Our method provided the

    most insightful explanations Original Ours ABN [Fukui+, CVPR19] LRP [Bach+, PloS15] GradCAM [Selvaraju+, ICCV17] RISE [Petsuik+, BMVC18]
  24. - 27 - Ours Qualitative results : Our method provided

    the most insightful explanations § Only one wing is attended § The attended area is vague Original ABN [Fukui+, CVPR19] LRP [Bach+, PloS15] GradCAM [Selvaraju+, ICCV17] RISE [Petsuik+, BMVC18]
  25. - 28 - Ours Qualitative results : Our method provided

    the most insightful explanations § The bird is correctly attended § The attended area is vague Original ABN [Fukui+, CVPR19] LRP [Bach+, PloS15] GradCAM [Selvaraju+, ICCV17] RISE [Petsuik+, BMVC18]
  26. - 29 - Ours Qualitative results : Our method provided

    the most insightful explanations § Almost no pixel is attended § Not an insightful explanation Original ABN [Fukui+, CVPR19] LRP [Bach+, PloS15] GradCAM [Selvaraju+, ICCV17] RISE [Petsuik+, BMVC18]
  27. - 30 - Ours Qualitative results : Our method provided

    the most insightful explanations § The bird is correctly attended § The background is also attended Original ABN [Fukui+, CVPR19] LRP [Bach+, PloS15] GradCAM [Selvaraju+, ICCV17] RISE [Petsuik+, BMVC18]
  28. - 31 - Ours Qualitative results : Our method provided

    the most insightful explanations § The bird is correctly attended § The shape of the bird’s body is precisely attended Original ABN [Fukui+, CVPR19] LRP [Bach+, PloS15] GradCAM [Selvaraju+, ICCV17] RISE [Petsuik+, BMVC18]
  29. - 32 - Ablation study : 𝑪𝟑 effectively generated high

    quality explanations +𝟎. 𝟒𝟑𝟔
  30. - 33 - Ablation study : 𝑪𝟑 effectively generated high

    quality explanations +𝟎. 𝟎𝟔𝟖
  31. - 34 - Conclusion : We successfully generated high quality

    explanations Contributions: • A method for calculating LRP in models with residual connections • 𝐶!-LRP, which improves the quality of explanations 𝑪𝟑-LRP LRP [Bach+, PloS15] output 𝐶!-LRP output
  32. - 35 - Appendix : Failure case example – All

    the different methods failed to generate an insightful explanation Original Ours ABN [Fukui 19] LRP [Bach 15] GradCAM [Selvaraju 18] RISE [Petsuik 18] ⇒ None of the different methods correctly attend the bird
  33. - 36 - Appendix : Error analysis - The method

    focusing on an insufficient part of the image Error Type IA OA WA #Error 63 21 16 Insufficiently Attended (IA) Over-Attended (OA) Wrongly-Attended (WA) • IA : The area of attention is too small. • OA : The area of relevance is excessively large • WA : The relevance is given to pixels that do not directly contribute to the classification.