Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Detecting Adversarial Audio via Activation Quan...

Detecting Adversarial Audio via Activation Quantization Error

IJCNN 2020. Presented by Heng Liu

Gregory Ditzler

June 19, 2020
Tweet

More Decks by Gregory Ditzler

Other Decks in Research

Transcript

  1. Detecting Adversarial Audio via Activation Quantization Error Heng Liu and

    Gregory Ditzler Department of Electrical & Computer Engineering University of Arizona Tucson, AZ 85721 USA {hengl, ditzler}@email.arizona.edu
  2. Outline • Introduction • Related works • Adversarial audio attack

    and detection • Neural network quantization • Contribution • Experiments • Conclusion and future works
  3. Introduction: Open problem How to detect? • Image applications •

    Image transformation • Feature transformation • Edge detection • Signal Processing based techniques • Quite effective • Audio applications • Techniques adopted from image domain • Limited security due to fundamentally different structure
  4. Introduction: A separate but related topic Neural network quantization •

    Reduce DNN’s memory, and computational resource consumption • Deployment on fog and edge devices • Low latency application Contribution • DNN quantization is beneficial for adversarial detection • We propose to detect adversarial audios by using neural network quantization
  5. Related works Adversarial audio attacks • Gradient based audio attack

    • Carlini and Wagner, 2018, SPW • Gradient based audio attack (over-the-air) • Yukura and Sakuma, IJCAI, 2019 • Black-box audio attack (free of gradient calculation) • Taori et al., SPW, 2019 Adversarial audio detection • Feature transformation • Frequency filters • Temporal dependency-based methods
  6. Related works Neural network quantization • Quantization schemes • Weight

    quantization • Activation quantization • Quantization error • Accuracy loss comparing with full precision model ↵ <latexit sha1_base64="JtDqaCSYHdUsArJlViGZOYtHm8o=">AAAB7XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzAOSJfROJsmY2ZllZlYIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSgQ31ve/vcLa+sbmVnG7tLO7t39QPjxqGpVqyhpUCaXbERomuGQNy61g7UQzjCPBWtH4dua3npg2XMkHO0lYGONQ8gGnaJ3U7KJIRtgrV/yqPwdZJUFOKpCj3it/dfuKpjGTlgo0phP4iQ0z1JZTwaalbmpYgnSMQ9ZxVGLMTJjNr52SM6f0yUBpV9KSufp7IsPYmEkcuc4Y7cgsezPxP6+T2sF1mHGZpJZJulg0SAWxisxeJ32uGbVi4ghSzd2thI5QI7UuoJILIVh+eZU0L6qBXw3uLyu1mzyOIpzAKZxDAFdQgzuoQwMoPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBi4GPGA==</latexit> <latexit sha1_base64="JtDqaCSYHdUsArJlViGZOYtHm8o=">AAAB7XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzAOSJfROJsmY2ZllZlYIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSgQ31ve/vcLa+sbmVnG7tLO7t39QPjxqGpVqyhpUCaXbERomuGQNy61g7UQzjCPBWtH4dua3npg2XMkHO0lYGONQ8gGnaJ3U7KJIRtgrV/yqPwdZJUFOKpCj3it/dfuKpjGTlgo0phP4iQ0z1JZTwaalbmpYgnSMQ9ZxVGLMTJjNr52SM6f0yUBpV9KSufp7IsPYmEkcuc4Y7cgsezPxP6+T2sF1mHGZpJZJulg0SAWxisxeJ32uGbVi4ghSzd2thI5QI7UuoJILIVh+eZU0L6qBXw3uLyu1mzyOIpzAKZxDAFdQgzuoQwMoPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBi4GPGA==</latexit> <latexit sha1_base64="JtDqaCSYHdUsArJlViGZOYtHm8o=">AAAB7XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzAOSJfROJsmY2ZllZlYIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSgQ31ve/vcLa+sbmVnG7tLO7t39QPjxqGpVqyhpUCaXbERomuGQNy61g7UQzjCPBWtH4dua3npg2XMkHO0lYGONQ8gGnaJ3U7KJIRtgrV/yqPwdZJUFOKpCj3it/dfuKpjGTlgo0phP4iQ0z1JZTwaalbmpYgnSMQ9ZxVGLMTJjNr52SM6f0yUBpV9KSufp7IsPYmEkcuc4Y7cgsezPxP6+T2sF1mHGZpJZJulg0SAWxisxeJ32uGbVi4ghSzd2thI5QI7UuoJILIVh+eZU0L6qBXw3uLyu1mzyOIpzAKZxDAFdQgzuoQwMoPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBi4GPGA==</latexit> <latexit sha1_base64="JtDqaCSYHdUsArJlViGZOYtHm8o=">AAAB7XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzAOSJfROJsmY2ZllZlYIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSgQ31ve/vcLa+sbmVnG7tLO7t39QPjxqGpVqyhpUCaXbERomuGQNy61g7UQzjCPBWtH4dua3npg2XMkHO0lYGONQ8gGnaJ3U7KJIRtgrV/yqPwdZJUFOKpCj3it/dfuKpjGTlgo0phP4iQ0z1JZTwaalbmpYgnSMQ9ZxVGLMTJjNr52SM6f0yUBpV9KSufp7IsPYmEkcuc4Y7cgsezPxP6+T2sF1mHGZpJZJulg0SAWxisxeJ32uGbVi4ghSzd2thI5QI7UuoJILIVh+eZU0L6qBXw3uLyu1mzyOIpzAKZxDAFdQgzuoQwMoPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBi4GPGA==</latexit> cap <latexit sha1_base64="uj2ucrtIPZA4UGSlfC0OHr/pfFs=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUCbbTbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIpypo0EYnqhKiZ4JI1DTeCdVLFMA4Fa4fj25nffmJK80Q+mknKghiHkkecorHSA8W071a9mjcHWSV+QapQoNF3v3qDhGYxk4YK1Lrre6kJclSGU8GmlV6mWYp0jEPWtVRizHSQz0+dkjOrDEiUKFvSkLn6eyLHWOtJHNrOGM1IL3sz8T+vm5noOsi5TDPDJF0sijJBTEJmf5MBV4waMbEEqeL2VkJHqJAam07FhuAvv7xKWhc136v595fV+k0RRxlO4BTOwYcrqMMdNKAJFIbwDK/w5gjnxXl3PhatJaeYOYY/cD5/AE44jcw=</latexit> <latexit sha1_base64="uj2ucrtIPZA4UGSlfC0OHr/pfFs=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUCbbTbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIpypo0EYnqhKiZ4JI1DTeCdVLFMA4Fa4fj25nffmJK80Q+mknKghiHkkecorHSA8W071a9mjcHWSV+QapQoNF3v3qDhGYxk4YK1Lrre6kJclSGU8GmlV6mWYp0jEPWtVRizHSQz0+dkjOrDEiUKFvSkLn6eyLHWOtJHNrOGM1IL3sz8T+vm5noOsi5TDPDJF0sijJBTEJmf5MBV4waMbEEqeL2VkJHqJAam07FhuAvv7xKWhc136v595fV+k0RRxlO4BTOwYcrqMMdNKAJFIbwDK/w5gjnxXl3PhatJaeYOYY/cD5/AE44jcw=</latexit> <latexit sha1_base64="uj2ucrtIPZA4UGSlfC0OHr/pfFs=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUCbbTbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIpypo0EYnqhKiZ4JI1DTeCdVLFMA4Fa4fj25nffmJK80Q+mknKghiHkkecorHSA8W071a9mjcHWSV+QapQoNF3v3qDhGYxk4YK1Lrre6kJclSGU8GmlV6mWYp0jEPWtVRizHSQz0+dkjOrDEiUKFvSkLn6eyLHWOtJHNrOGM1IL3sz8T+vm5noOsi5TDPDJF0sijJBTEJmf5MBV4waMbEEqeL2VkJHqJAam07FhuAvv7xKWhc136v595fV+k0RRxlO4BTOwYcrqMMdNKAJFIbwDK/w5gjnxXl3PhatJaeYOYY/cD5/AE44jcw=</latexit> <latexit sha1_base64="uj2ucrtIPZA4UGSlfC0OHr/pfFs=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUCbbTbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIpypo0EYnqhKiZ4JI1DTeCdVLFMA4Fa4fj25nffmJK80Q+mknKghiHkkecorHSA8W071a9mjcHWSV+QapQoNF3v3qDhGYxk4YK1Lrre6kJclSGU8GmlV6mWYp0jEPWtVRizHSQz0+dkjOrDEiUKFvSkLn6eyLHWOtJHNrOGM1IL3sz8T+vm5noOsi5TDPDJF0sijJBTEJmf5MBV4waMbEEqeL2VkJHqJAam07FhuAvv7xKWhc136v595fV+k0RRxlO4BTOwYcrqMMdNKAJFIbwDK/w5gjnxXl3PhatJaeYOYY/cD5/AE44jcw=</latexit> y = (W T X + b) <latexit sha1_base64="JWZ45c5Mtd9es796imEkYxG6+ug=">AAAB+XicbVBNS8NAEJ3Ur1q/oh69LBahIpREBL0IRS8eK/QL2lg22027dLMJu5tCCP0nXjwo4tV/4s1/47bNQVsfDDzem2Fmnh9zprTjfFuFtfWNza3idmlnd2//wD48aqkokYQ2ScQj2fGxopwJ2tRMc9qJJcWhz2nbH9/P/PaESsUi0dBpTL0QDwULGMHaSH3bTtEt6sUjVmk/NToX/nnfLjtVZw60StyclCFHvW9/9QYRSUIqNOFYqa7rxNrLsNSMcDot9RJFY0zGeEi7hgocUuVl88un6MwoAxRE0pTQaK7+nshwqFQa+qYzxHqklr2Z+J/XTXRw42VMxImmgiwWBQlHOkKzGNCASUo0Tw3BRDJzKyIjLDHRJqySCcFdfnmVtC6rrlN1H6/Ktbs8jiKcwClUwIVrqMED1KEJBCbwDK/wZmXWi/VufSxaC1Y+cwx/YH3+AEX1kh0=</latexit> <latexit sha1_base64="JWZ45c5Mtd9es796imEkYxG6+ug=">AAAB+XicbVBNS8NAEJ3Ur1q/oh69LBahIpREBL0IRS8eK/QL2lg22027dLMJu5tCCP0nXjwo4tV/4s1/47bNQVsfDDzem2Fmnh9zprTjfFuFtfWNza3idmlnd2//wD48aqkokYQ2ScQj2fGxopwJ2tRMc9qJJcWhz2nbH9/P/PaESsUi0dBpTL0QDwULGMHaSH3bTtEt6sUjVmk/NToX/nnfLjtVZw60StyclCFHvW9/9QYRSUIqNOFYqa7rxNrLsNSMcDot9RJFY0zGeEi7hgocUuVl88un6MwoAxRE0pTQaK7+nshwqFQa+qYzxHqklr2Z+J/XTXRw42VMxImmgiwWBQlHOkKzGNCASUo0Tw3BRDJzKyIjLDHRJqySCcFdfnmVtC6rrlN1H6/Ktbs8jiKcwClUwIVrqMED1KEJBCbwDK/wZmXWi/VufSxaC1Y+cwx/YH3+AEX1kh0=</latexit> <latexit sha1_base64="JWZ45c5Mtd9es796imEkYxG6+ug=">AAAB+XicbVBNS8NAEJ3Ur1q/oh69LBahIpREBL0IRS8eK/QL2lg22027dLMJu5tCCP0nXjwo4tV/4s1/47bNQVsfDDzem2Fmnh9zprTjfFuFtfWNza3idmlnd2//wD48aqkokYQ2ScQj2fGxopwJ2tRMc9qJJcWhz2nbH9/P/PaESsUi0dBpTL0QDwULGMHaSH3bTtEt6sUjVmk/NToX/nnfLjtVZw60StyclCFHvW9/9QYRSUIqNOFYqa7rxNrLsNSMcDot9RJFY0zGeEi7hgocUuVl88un6MwoAxRE0pTQaK7+nshwqFQa+qYzxHqklr2Z+J/XTXRw42VMxImmgiwWBQlHOkKzGNCASUo0Tw3BRDJzKyIjLDHRJqySCcFdfnmVtC6rrlN1H6/Ktbs8jiKcwClUwIVrqMED1KEJBCbwDK/wZmXWi/VufSxaC1Y+cwx/YH3+AEX1kh0=</latexit> <latexit sha1_base64="JWZ45c5Mtd9es796imEkYxG6+ug=">AAAB+XicbVBNS8NAEJ3Ur1q/oh69LBahIpREBL0IRS8eK/QL2lg22027dLMJu5tCCP0nXjwo4tV/4s1/47bNQVsfDDzem2Fmnh9zprTjfFuFtfWNza3idmlnd2//wD48aqkokYQ2ScQj2fGxopwJ2tRMc9qJJcWhz2nbH9/P/PaESsUi0dBpTL0QDwULGMHaSH3bTtEt6sUjVmk/NToX/nnfLjtVZw60StyclCFHvW9/9QYRSUIqNOFYqa7rxNrLsNSMcDot9RJFY0zGeEi7hgocUuVl88un6MwoAxRE0pTQaK7+nshwqFQa+qYzxHqklr2Z+J/XTXRw42VMxImmgiwWBQlHOkKzGNCASUo0Tw3BRDJzKyIjLDHRJqySCcFdfnmVtC6rrlN1H6/Ktbs8jiKcwClUwIVrqMED1KEJBCbwDK/wZmXWi/VufSxaC1Y+cwx/YH3+AEX1kh0=</latexit>
  7. Contribution: Part 1 • Motivation: Error amplification effect • Adversarial

    perturbations are negligible at the input level, but are progressively amplified, eventually lead to wrong prediction • Effective on image defense techniques to ameliorate adversarial attacks • Hypothesis • We hypothesize that the activation quantization error on DNN’s output layer behaves differently for benign and adversarial audios Activation quantization errors on audios
  8. Contribution: Part 1 • Victim model • DeepSpeech (open-sourced with

    pre-trained model), for ASR task • The benchmark dataset is Mozilla Common Voice • Adversarial audios • 1. Carlini and Wagner; • 2. Yukura and Sakuma; • 3. Taori et al. • Fixed width quantization • Variable activation quantization bit width for FCN and BiRNN layers • Bit quantization levels: 1 - 8 bits • Quantization error • Measured by Character Error Rate (CER): Calculated between transcripts from full precision and quantized models • CER is defined as: (S + D + I)/N FCN BiRNN FCN FCN FCN FCN Audios Transcription Activation Quantization Errors on Audios: Empirical analysis
  9. Contribution: Part 1 Averaged CER: benign V.S. adversarial audios •

    Observations • The benign audios have an overall lower CER than all three types of adversarial audios • The differences vary across different quantization bit widths • Observation holds true for all three adversarial audio attacks
  10. Contribution: Part 2 Adversarial audio detection method • Rule of

    Thumb: Classify audio clips that have a large activation quantization error as adversarial • How to determine the threshold and bit width? • We empirically estimate the best threshold and bit width Pseudo code
  11. Conclusion and future work • We examined the activation quantization

    error for benign and adversarial audios • We proposed an effective and reliable adversarial audio detection method Conclusions Future work • One future work is to analytically investigating the activation quantization error’s behavior Funding • This work was supported by grants from the Department of Energy #DE- NA0003946, Army Research Lab W56KGU-20-C-0002, and National Science Foundation CAREER #1943552