Building an arcade with Android by Éric Boissonneault

Discover how an Android app can interact with hardware I/O components to turn your next game idea into a fully working arcade.

Transcript

1. Building an arcade with Android By Eric Boissonneault Android Developer @ Transit
2. Agenda • Discovering a smart lighting product • Searching for the SDK • Making my own Android library • Various experiments • What about a game? • Adding controller inputs • Future improvements and ideas
3. Twinkly Squares
4. Twinkly Squares • Master tile ◦ USB-C powered ◦ Using ESP32 microcontroller ◦ Wifi capabilities ◦ BT capabilities ◦ 4 ports for tile cables • Extension tiles ◦ 2 ports for tile cables • Maximum of 16 tiles (1 master + 15 extension) • 1 tile = 64 pixels (8x8) • Current configuration is 9 tiles (24x24) • Total of 576 pixels • Pixels mapping is done through the official app
5. Twinkly lights mapping
6. Twinkly product features Effects (animated GIFs) Single color Sound reactive
7. Available SDKs • No official SDK Source: https://xled-docs.readthedocs.io/ • An on-device API exists • API specifications have been reversed-engineered • Various client implementations can now be found • No Android SDK
8. Hypothesis Successfully create an Android SDK capable of updating frames… • Fast enough? • Over a long period of time? • Without experiencing lag?
9. Implementation Requirements • REST Endpoint ◦ Authenticate ◦ Get the pixels layout ◦ Switch to real-time mode • UDP Endpoint ◦ Send frames using the real-time mode
10. Creating a background service
11. Creating a background service
12. Background service: REST API REST API authentication, fetch pixels layout, setup the real-time mode
13. REST API Implementation
14. Background service: Real-time UDP frames Every 25ms: Sends frame if pixels changed OR after 10 seconds idle (keep-alive)
15. UDP datagram (frame) implementation Source: https://xled-docs.readthedocs.io/
16. Board class: Pixels Matrix Representation
17. Experiment 0: Display all pixels in the correct order
18. Experiment 1: Edges and corners
19. Experiment 2: Bitmap
20. Experiment 3: Bitmap horizontal / vertical translation
21. Experiment 4: Text
22. Experiment 5: Tetris
23. Experiment 6: Battleship
24. What about a game? Tug Of War
25. Game implementation
26. Player interaction (input) and sending pixel frames (output)
27. Game controller inputs L2 and R2 buttons
28. Arcade buttons
29. Upcoming game improvements • Modify the board layout (72x8) • Adjustable edges for a game • Pause the game • Add animations before the game starts and after winning • Support multiplayer 2v2, 3v3, 4v4 • Add a leaderboard • Build a plexiglass table to insert the Twinkly Squares and arcade buttons
30. Idea on my bucket list • Twinkly Matrix (new product) • Challenge with mapping the lights ◦ X,Y coordinates aren’t accurate ◦ Custom mapping tool needed • Face recognition using a camera with OpenCV ◦ Select face area ◦ Convert to a pixel matrix • Project the face pixels into the Twinkly Matrix • Add animations and various interactions • Extend the layout from 10x50 lights to 200x40 lights
31. References • Twinkly products: https://twinkly.com/ • Twinkly unofficial SDKs documentation: https://xled-docs.readthedocs.io/ • ChatGPT: https://chat.openai.com/ • GitHub Copilot: https://github.com/features/copilot • Arcade buttons: Search for SJ@JX Android on Amazon
32. Questions? Thank you!