Pythonとラズパイで ラジコンカーを自動操縦に！

PyCon mini Sapporo 2019 での発表資料です。

Transcript

1. 1ZUIPOͱϥζύΠͰ
   ϥδίϯΧʔΛࣗಈૢॎʹʂ
   1Z$PO
   NJOJ
   4BQQPSP
   
   
   .JOPSV
   *OBDIJ
2. ࣗݾ঺հ ໊લɿҴ஍ɹູʢ͍ͳͪɹΈͷΔʣ
   ࢓ࣄɿΤϯλʔϓϥΠζܥͷιϑτ΢ΣΞΤϯδχΞ
   1ZUIPOྺɿ1ZUIPOͷࠒ͔Β
 ʢͨͩ͠ɺ1ZUIPO
   Y΄ͱΜͲ஌Βͳ͍΄ͲϒϥϯΫ௕͠ʣ
   ಀආ׆ಈɿ.JDSP1ZUIPOͷಈ͘ϚΠίϯϘʔυूΊ
6. https://diyrobocars.com/
9. https://www.donkeycar.com/
12. ϩόʢ%POLFZʣ͸࠷ॳظʹࣂ͍ͳΒ͞Εͨӡൖ༻్ͷಈ෺
    ϩό͸ࢠͲ΋ʹ΋҆શ
    ϩό͸όΧͳͷͰݴ͏͜ͱฉ͔ͳ͍͜ͱ͋Δ
14. https://twitter.com/awsreinvent/status/935369886758354944
15. https://weekly.ascii.jp/elem/000/000/423/423904/
16. https://lab-kadokawa73.peatix.com/
18. ૊Έཱͯʹඞཁͳ΋ͷ
19. εϩοτϧ18. εςΞϦϯά18. ిݯڙڅ 7 όοςϦిݯ 7 ը૾ .*1* Ϟʔλ੍ޚ໋ྩ *$ ਪ࿦ 5FOTPS'MPX ,FSBT ˇ
     
    https://www.amazon.co.jp/dp/B07GCKNG72 ˇ
     
    https://www.sengoku.co.jp/mod/sgk_cart/detail.php?code=EEHD-4Z8C ˇ
     https://www.amazon.co.jp//dp/B01D1D0CX2/ ˇ
     
    https://www.amazon.co.jp/dp/B01LH6SVEC/ ˇ
    
    https://www.amazon.co.jp/dp/B00IZ8DRTU/
20. https://www.thingiverse.com/thing:2260575/files
21. https://www.robocarstore.com/
22. ˇ
     ɹ͙Β͍
23. ϥζύΠ͸#͔# ΛࢦఆͰ͖ͳ͍ɻ
 ٕదऔಘࡁͷ΋ͷ͕ಧ͘ͱ͸ݶΒͳ͍ɻ ͳͷͰϥζύΠͳ͠ΛબͿͷ͕ແ೉ όοςϦʔॆి༻"$Ξμϓλ͸
 ʮ64+1
    1MVHʯΛબͿ͜ͱ ˇ
     ɹ͙Β͍ ӡ͕ѱ͍ͱ
    =
     
    ఔ౓ͷؔ੫ͱΒΕΔ
24. https://www.fabo.store/collections/donkeycar
35. Vehicle Memory Timestamp PiCamera {active=true} LocalWebController {active=true} KerasLinear PWMSteering PWMThrottle timestamp cam/image_array cam/image_array user/angle, user/throttle
 user/mode, recording Lambda(pilot_condition) user/mode run_pilot cam/image_array pilot/angle, pilot/throttle Lambda(drive_mode) user/mode
 user/angle, user/throttle
 pilot/angle, pilot/throttle angle, throttle angle throttle TubWriter cam/image_array
 user/angle, user/throttle
 user/mode, timestamp start run run run run run run run
36. Vehicle Memory Timestamp PiCamera {active=true} LocalWebController {active=true} KerasLinear PWMSteering PWMThrottle timestamp cam/image_array cam/image_array user/angle, user/throttle
 user/mode, recording Lambda(pilot_condition) user/mode run_pilot cam/image_array pilot/angle, pilot/throttle Lambda(drive_mode) user/mode
 user/angle, user/throttle
 pilot/angle, pilot/throttle angle, throttle angle throttle TubWriter cam/image_array
 user/angle, user/throttle
 user/mode, timestamp
37. Vehicle Memory Timestamp PiCamera {active=true} LocalWebController {active=true} KerasLinear PWMSteering PWMThrottle timestamp cam/image_array cam/image_array user/angle, user/throttle
 user/mode, recording Lambda(pilot_condition) user/mode run_pilot cam/image_array pilot/angle, pilot/throttle Lambda(drive_mode) user/mode
 user/angle, user/throttle
 pilot/angle, pilot/throttle angle, throttle angle throttle TubWriter cam/image_array
 user/angle, user/throttle
 user/mode, timestamp
39. Vehicle Memory Timestamp PiCamera {active=true} JoyStickController {active=true} KerasLinear PWMSteering PWMThrottle timestamp cam/image_array cam/image_array user/angle, user/throttle
 user/mode, recording Lambda(pilot_condition) user/mode run_pilot cam/image_array pilot/angle, pilot/throttle Lambda(drive_mode) user/mode
 user/angle, user/throttle
 pilot/angle, pilot/throttle angle, throttle angle throttle TubWriter cam/image_array
 user/angle, user/throttle
 user/mode, timestamp https://github.com/coolerking/donkeypart_game_controller
42. Vehicle Memory Timestamp PiCamera {active=true} LocalWebController {active=true} KerasLinear PWMSteering PWMThrottle timestamp cam/image_array cam/image_array user/angle, user/throttle
 user/mode, recording Lambda(pilot_condition) user/mode run_pilot cam/image_array pilot/angle, pilot/throttle Lambda(drive_mode) user/mode
 user/angle, user/throttle
 pilot/angle, pilot/throttle angle, throttle angle throttle TubWriter cam/image_array
 user/angle, user/throttle
 user/mode, timestamp
43. Vehicle Memory Timestamp PiCamera {active=true} LocalWebController {active=true} KerasLinear PWMSteering PWMThrottle timestamp cam/image_array cam/image_array user/angle, user/throttle
 user/mode, recording Lambda(pilot_condition) user/mode run_pilot cam/image_array pilot/angle, pilot/throttle Lambda(drive_mode) user/mode
 user/angle, user/throttle
 pilot/angle, pilot/throttle angle, throttle angle throttle TubWriter cam/image_array
 user/angle, user/throttle
 user/mode, timestamp
44. Vehicle Memory Timestamp PiCamera {active=true} LocalWebController {active=true} KerasLinear PWMSteering PWMThrottle timestamp cam/image_array cam/image_array user/angle, user/throttle
 user/mode, recording Lambda(pilot_condition) user/mode run_pilot cam/image_array pilot/angle, pilot/throttle Lambda(drive_mode) user/mode
 user/angle, user/throttle
 pilot/angle, pilot/throttle angle, throttle angle throttle TubWriter cam/image_array
 user/angle, user/throttle
 user/mode, timestamp
45. https://images.nvidia.com/content/tegra/automotive/images/2016/solutions/pdf/end-to-end-dl-using-px.pdf
46. img_in = Input(shape=(120, 160, 3),
 name='img_in') x = img_in x = Convolution2D(filters=24,
 kernel_size=(5, 5),
 strides=(2, 2),
 activation='relu')(x) x = Convolution2D(filters=32,
 kernel_size=(5, 5),
 strides=(2, 2),
 activation='relu')(x) x = Convolution2D(filters=64,
 kernel_size=(5, 5),
 strides=(2, 2),
 activation='relu')(x) x = Convolution2D(filters=64,
 kernel_size=(3, 3),
 strides=(2, 2),
 activation='relu')(x) x = Convolution2D(filters=64,
 kernel_size=(3, 3),
 strides=(1, 1),
 activation='relu')(x) x = Flatten(name='flattened')(x) x = Dense(units=100, activation='linear')(x) x = Dropout(rate=.1)(x) x = Dense(units=50, activation='linear')(x) x = Dropout(rate=.1)(x) # categorical output of the angle angle_out = Dense(units=1, activation='linear',
 name='angle_out')(x) # continous output of throttle throttle_out = Dense(units=1,
 activation='linear',
 name='throttle_out')(x) 1JMPU/FU
47. Vehicle Memory Timestamp PiCamera {active=true} LocalWebController {active=true} KerasLinear PWMSteering PWMThrottle timestamp cam/image_array cam/image_array user/angle, user/throttle
 user/mode, recording Lambda(pilot_condition) user/mode run_pilot cam/image_array pilot/angle, pilot/throttle Lambda(drive_mode) user/mode
 user/angle, user/throttle
 pilot/angle, pilot/throttle angle, throttle angle throttle TubWriter cam/image_array
 user/angle, user/throttle
 user/mode, timestamp
48. Vehicle Memory Timestamp PiCamera {active=true} LocalWebController {active=true} KerasLinear PWMSteering PWMThrottle timestamp cam/image_array cam/image_array user/angle, user/throttle
 user/mode, recording Lambda(pilot_condition) user/mode run_pilot cam/image_array pilot/angle, pilot/throttle Lambda(drive_mode) user/mode
 user/angle, user/throttle
 pilot/angle, pilot/throttle angle, throttle angle throttle TubWriter cam/image_array
 user/angle, user/throttle
 user/mode, timestamp
49. Vehicle Memory Timestamp PiCamera {active=true} LocalWebController {active=true} KerasLinear TwoWheelDifferencialDrive timestamp cam/image_array cam/image_array user/angle, user/throttle
 user/mode, recording Lambda(pilot_condition) user/mode run_pilot cam/image_array pilot/angle, pilot/throttle Lambda(drive_mode) user/mode
 user/angle, user/throttle
 pilot/angle, pilot/throttle angle, throttle angle, throrttle TubWriter cam/image_array
 user/angle, user/throttle
 user/mode, timestamp https://github.com/autorope/donkeypart_differential_drive
53. Memory Timestamp PiCamera {active=true} LocalWebController {active=true} KerasLinear PWMSteering PWMThrottle timestamp cam/image_array cam/image_array user/angle, user/throttle
 user/mode, recording Lambda(pilot_condition) user/mode run_pilot cam/image_array pilot/angle, pilot/throttle Lambda(drive_mode) user/mode
 user/angle, user/throttle
 pilot/angle, pilot/throttle angle, throttle angle throttle TubWriter cam/image_array
 user/angle, user/throttle
 user/mode, timestamp
54. {࿈൪}_cam_image_array.jpg { "user/mode": “user", "cam/image_array": “1_cam-image_array_.jpg", "user/angle": 0.0, "user/throttle": 0.7027450980392154 "timestamp": "2019-03-16 03:01:33.533054” } record_{࿈൪}.json
55. ૸ߦσʔλ
    ऩू ૸ߦσʔλ
    ֶश ࣗಈ૸ߦ ૸ߦσʔλ ֶशϞσϧ ૸ߦσʔλ௥Ճ Ϟσϧݟ௚͠
56. ૸ߦσʔλ
    ऩू python manage.py drive ૸ߦσʔλ
    ֶश ࣗಈ૸ߦ ૸ߦσʔλ ֶशϞσϧ ૸ߦσʔλ௥Ճ Ϟσϧݟ௚͠
57. ॏཁ՝୊ɿ
    ૸ΒͤΔ৔ॴͷ֬อʂ
62. ૸ߦσʔλ
    ऩू ૸ߦσʔλ
    ֶश python manage.py train ※ PC or Ϋϥ΢υ্ ࣗಈ૸ߦ python manage.py drive —model=ֶशϞσϧ ૸ߦσʔλ ֶशϞσϧ ૸ߦσʔλ௥Ճ Ϟσϧݟ௚͠
67. ૸ߦσʔλ
    ऩू ૸ߦσʔλ
    ֶश ࣗಈ૸ߦ python manage.py drive —model=ֶशϞσϧ ૸ߦσʔλ ֶशϞσϧ ૸ߦσʔλ௥Ճ Ϟσϧݟ௚͠
70. ⾛ ⽬ …
71. https://arxiv.org/pdf/1704.07911.pdf
79. img_in = Input(shape=(120, 160, 3), name='img_in') x = img_in x = Cropping2D(cropping=((40, 0), (0, 0)))(x) # trim 40 pixels off top x = Convolution2D(filters=24, kernel_size=(5, 5), strides=(2, 2),
 activation='relu')(x) x = Convolution2D(filters=32, kernel_size=(5, 5), strides=(2, 2),
 activation='relu')(x) x = Convolution2D(filters=64, kernel_size=(5, 5), strides=(2, 2),
 activation='relu')(x) x = Convolution2D(filters=64, kernel_size=(3, 3), strides=(2, 2),
 activation='relu')(x) x = Convolution2D(filters=64, kernel_size=(3, 3), strides=(1, 1),
 activation='relu')(x) x = Flatten(name='flattened')(x) x = Dense(units=100, activation='linear')(x) x = Dropout(rate=.1)(x) x = Dense(units=50, activation='linear')(x) x = Dropout(rate=.1)(x) # categorical output of the angle angle_out = Dense(units=1, activation=‘linear', name='angle_out')(x) # continous output of throttle throttle_out = Dense(units=1, activation=‘linear', name='throttle_out')(x)
83. ͓஥ؒืू
86. https://donkey-slackin.herokuapp.com/
87. ͓͠·͍