DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars

Transcript

1. Presented by Liang Gong DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars Yuchi Tian, Kexin Pei, Suman Jana, Baishakhi Ray Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley.
2. Recap of DeepXplore Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. • Neuron Coverage • Differential Testing + Gradient Descent • Simulate a few realistic scenarios: • Lightening Conditions, Masks, Dirt • Evaluate on different domains 2
3. DeepTest Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. • Empirically show relations between Neuron Coverage and DNN output • AFL-like guided testing of DNN • Simulate 9 realistic scenarios • Detect thousands of erroneous behaviors 3
4. Uber Auto Car Kills Pedestrian Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 4
5. Uber Auto Car Kills Pedestrian Wearing a black shirt Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 5 The car did not slow down.
6. Uber Auto Car Kills Pedestrian Wearing a black shirt The car did not slow down. Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 6 Cause: Lightening Condition + Image Contrast
7. https://jalopnik.com/this-is-how-bad-self-driving-cars-suck-in-the-rain-1666268433 Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 7 Hyundai Competition Crashed while testing Rain fall
8. Rain fall https://jalopnik.com/this-is-how-bad-self-driving-cars-suck-in-the-rain-1666268433 Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 8 Hyundai Competition Crashed while testing The sensors failed to pick up street signs, lane markings, and even pedestrians due to the angle of the car shifting in rain and the direction of the sun. Cause: Rain Fall + Lighting Condition + Angle
9. https://www.nytimes.com/2017/01/19/business/tesla-model-s-autopilot-fatal-crash.html Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 9 Tesla Autopilot Mode Kills the Driver The camera failed to recognize the white truck against a bright sky. … was killed when his Tesla Model S, which was operating in Autopilot mode, crashed into a tractor-trailer. Krista Kitchen, via Associated Press Cause: Image Contrast
10. Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 10 Google Self-driving Car Accident Hit a bus while shifting lane. https://www.wired.com/2016/02/googles-self-driving-car-may-caused-first-crash/
11. Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 11 Google Self-driving Car Accident Hit a bus while shifting lane. The car assumed that the bus would yield when it attempted to merge back into traffic. https://www.wired.com/2016/02/googles-self-driving-car-may-caused-first-crash/
12. https://www.wired.com/2016/02/googles-self-driving-car-may-caused-first-crash/ Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 12 Google Self-driving Car Accident Hit a bus while shifting lane. The car assumed that the bus would yield when it attempted to merge back into traffic. Cause: failed to estimate speed
13. Outline Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. Empirically demonstrate that: 13 Changes in Neuron Coverage Changes in the Actions of Self- driving Cars correlate • Generate tests to increase neuron coverage. • Simulate realistic scenarios: fog, rain, angle, scale etc. • Found thousands of erroneous behaviors
14. Intuition Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 14 Change the image a little and observe the DNN’s output.
15. Simplified Model of Self-driving Car Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 15 Signal lights etc. CNN / RNN
16. Simplified Model of Self-driving Car Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 16 Signal lights etc. CNN / RNN
17. Guided Testing Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 17 • neuron coverage as a metric to guide the testing • Activation of a neuron: • In a fully connect layer • In a convolutional layer • In a RNN
18. Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 18 • Activation of a neuron: • In a fully connect layer • In a convolutional layer Output of a neuron is a scalar value (threshold 0.2) Neuron Activation
19. Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 19 • Activation of a neuron: • In a fully connect layer • In a convolutional layer A neuron in a conv layer Neuron Activation
20. Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 20 • Activation of a neuron: • In a fully connect layer • In a convolutional layer A neuron in a conv layer Neuron Activation
21. Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 21 • Activation of a neuron: • In a fully connect layer • In a convolutional layer A neuron in a conv layer Neuron Activation
22. Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 22 • Activation of a neuron: • In a fully connect layer • In a convolutional layer A neuron in a conv layer Neuron Activation
23. Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 23 • Activation of a neuron: • In a fully connect layer • In a convolutional layer A neuron in a conv layer Neuron Activation
24. Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 24 • Activation of a neuron: • In a fully connect layer • In a convolutional layer A neuron in a conv layer Neuron Activation
25. Neuron Activation Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 25 • Activation of a neuron: • In a fully connect layer • In a convolutional layer A neuron in a conv layer
26. Neuron Activation Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 26 • Activation of a neuron: • In a fully connect layer • In a convolutional layer A neuron in a conv layer
27. Neuron Activation Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 27 • Activation of a neuron: • In a fully connect layer • In a convolutional layer A neuron in a conv layer Output: a feature map
28. Neuron Activation Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 28 • Activation of a neuron: • In a fully connect layer • In a convolutional layer Output: a feature map Average the map to get a scalar value A neuron in a conv layer
29. Neuron Activation Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 29 • Activation of a neuron: • In a RNN Unroll the RNN
30. Neuron Activation Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 30 • Activation of a neuron: • In a RNN Unroll the RNN
31. Neuron Activation Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 31 • Activation of a neuron: • In a RNN Unroll the RNN cosnider as different neurons
32. Increase Coverage with Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 32 • Nine different transformations • Brightness • Contrast • Translation • Scaling • Horizontal Shearing • Rotation • Blurring • Fog Effect • Rain Effect
33. Increase Coverage with Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 33 • Nine different transformations • Brightness • Contrast • Translation • Scaling • Horizontal Shearing • Rotation • Blurring • Fog Effect • Rain Effect
34. Increase Coverage with Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 34 • Transformation of Brightness • For each pixel +/- a constant value • Transformation of Contrast • For each pixel * a constant value
35. Increase Coverage with Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 35 • Nine different transformations • Brightness • Contrast • Translation • Scaling • Horizontal Shearing • Rotation • Blurring • Fog Effect • Rain Effect Affine Transformations
36. Affine Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 36 • Moving pixels to new positions rotations Move along x/y axis Transformation Matrix
37. Affine Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 37 • Moving pixels to new positions Transformation Matrix
38. Affine Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 38 • Moving pixels to new positions Transformation Matrix
39. Affine Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 39 • Moving pixels to new positions coordinates of all pixels in the original image Transformation Matrix
40. Affine Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 40 • Moving pixels to new positions coordinates of all pixels in the original image Transformation Matrix coordinates of all pixels in the new image
41. Affine Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 41 • Moving pixels to new positions coordinates of all pixels in the original image Transformation Matrix coordinates of all pixels in the new image
42. Affine Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 42
43. Affine Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 43
44. Affine Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 44
45. Affine Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 45
46. Affine Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 46
47. Affine Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 47
48. Affine Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 48
49. Affine Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 49
50. Affine Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 50
51. Affine Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 51
52. Affine Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 52
53. Increase Coverage with Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 53 • Nine different transformations • Brightness • Contrast • Translation • Scaling • Horizontal Shearing • Rotation • Blurring • Fog Effect • Rain Effect Photoshop A combination of built-in convolutional transformations
54. Combine Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 54 • Greedily combine transformations to images • Try to maximize the coverage
55. Combine Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 55 • Greedily combine transformations to images • Try to maximize the coverage Pop a seed image from the stack
56. Combine Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 56 • Greedily combine transformations to images • Try to maximize the coverage For the seed image: • A transformation queue • A fail counter
57. Combine Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 57 • Greedily combine transformations to images • Try to maximize the coverage For the seed image: • A upper bound of transform combos that do not increase coverage
58. Combine Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 58 • Greedily combine transformations to images • Try to maximize the coverage dequeue or randomly pick a transformation with random parameters
59. Combine Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 59 • Greedily combine transformations to images • Try to maximize the coverage randomly pick another transformation with random parameters
60. Combine Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 60 • Greedily combine transformations to images • Try to maximize the coverage if improves coverage of the seed image: • Queue the transformation • Record the new image
61. Combine Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 61 • Greedily combine transformations to images • Try to maximize the coverage Otherwise increase the fail counter
62. Specify Test Oracle (Steering Angle) Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 62
63. Specify Test Oracle (Steering Angle) Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 63 predicted angle of the synthesized image Ground truth of the original image
64. Specify Test Oracle (Steering Angle) Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 64 predicted angle of the synthesized image Ground truth of the original image
65. Specify Test Oracle (Steering Angle) Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 65 predicted angle of the synthesized image Ground truth of the original image predicted angle of the original image
66. Specify Test Oracle (Steering Angle) Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 66 predicted angle of the synthesized image Ground truth of the original image predicted angle of the original image n is the number of original images
67. Specify Test Oracle (Steering Angle) Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 67 predicted angle of the synthesized image Ground truth of the original image predicted angle of the original image n is the number of original images Error of a Transformed Image ≤ λ * Avg Error of Original Images
68. DNN Models Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 68 Udacity self-driving challenge top models Max steering angles
69. DNN Models Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 69 CNN 100 Images 001011010101010… 100 features of 100 images … LSTM Steering angle
70. DNN Models Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 70 CNN Images Steering angle CNN CNN …
71. DNN Models Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 71 CNN Images Steering angle …
72. Relation between NC and Test Output Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 72
73. Relation between NC and Test Output Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 73 Negative Correlation Larger NC  Smaller Angle [-1, 1] Positive Correlation Larger NC  Larger Angle [-1, 1]
74. Relation between NC and Test Output Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 74
75. Relation between NC and Test Output Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 75 Larger NC  More Likely to Turn Right
76. Diff Transformations Activate diff Neurons? Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 76 1. Randomly Pick a set of images 2. For each Image I and each possible pair of transformations: <T1 , T2 > 3. Apply T1 on I and get cov N1 4. Apply T2 on I and get cov N2 5. Get the Distance between N1 and N2
77. Diff Transformations Activate diff Neurons? Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 77 For most models, different transformations activate different set of neurons.
78. Average Cumulative Neuron Coverage per Input Image Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 78
79. Neuron Coverage Increase Caused by Transformations Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 79
80. Is the Guided Approach Useful? Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 80 Baseline 1: 100 seed images Get coverage Baseline 2: Cumulative Transformation Apply single transform on seed images 7000 synthetic Images Get Coverage
81. Is the Guided Approach Useful? Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 81 Neuron Coverage (Higher  Better) Baseline 2
82. Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 82 Erroneous Behaviors Detected by DeepTest λ = 5
83. Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 83 λ = 5 False Positives
84. Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 84 Improvement of Model after Retraining based on Tests from DeepTest Epoch Model