How We Moved from Android Studio to Slack

Our team ships fast. New features, bug fixes, hotfixes, all moving through the pipeline constantly. But we kept hitting the same bottleneck: QA finds a bug, opens a ticket, a dev picks it up, opens the project, makes the fix, opens a PR, waits for review. For small fixes this cycle felt heavier than the fix itself.

So we asked a simple question: what if QA could get a bug fixed without leaving Slack?

We built an AI agent that lives inside Slack. QA opens a thread with the bug details, a dev adds technical context if needed, and someone mentions the bot. From there, the agent reads the full thread, clones the repo, makes the fix, runs linters and tests, verifies the build passes, and opens a PR. No one had to open an IDE.

In the first 3 days on our first project, 20 bugs were fixed this way. Today it runs across 10 projects.

In this talk I'll walk through how we built this system and what we learned along the way:

• How we designed the agent to read Slack threads, understand context from screenshots and messages, and translate that into code changes
• Why we used Docker containers per job, with dedicated images for each platform (Android SDK, Flutter, .NET, Ruby) to keep everything isolated and reproducible
• The Android-specific challenges: Gradle builds inside containers, SDK setup, running verification without emulators
• Connecting the agent to external tools through MCP, and why authentication turned out to be one of the hardest problems to solve, especially with services like Figma
• What went well and what we'd do differently next time

Transcript

1. DROIDKAIGI 2026 · TOKYO How we moved from Android Studio to Slack Four months of production data from an AI agent that fixes bugs where the bug is reported. SLIDES speakerdeck.com/thisaay/how-we-moved-from-android-studio-to-slack Ahmed Ali neverstop
2. HELLO Ahmed Ali 12 years writing code Mobile engineer Developer tools At Infinum DroidKaigi 2024 Cairo, Egypt My passion is building developer tools, and that is why I am here today. neverstop 02
3. I know you are tired of hearing AI, AI, AI everywhere. I am tired too. So this talk has no concepts and no predictions in it. Only what we ran in production, and what it actually cost. neverstop 03
4. How long does it take to fix a one-line bug? 04
5. 01 The classic cycle
6. HOW A BUG GOT FIXED The classic cycle 01 neverstop QA finds a bug 06
7. HOW A BUG GOT FIXED The classic cycle 01 02 neverstop QA finds a bug Writes a ticket 06
8. HOW A BUG GOT FIXED The classic cycle 01 02 03 neverstop QA finds a bug Writes a ticket Waits to be assigned 06
9. HOW A BUG GOT FIXED The classic cycle 01 02 03 04 neverstop QA finds a bug Writes a ticket Waits to be assigned A dev opens the project 06
10. HOW A BUG GOT FIXED The classic cycle 01 02 03 04 05 neverstop QA finds a bug Writes a ticket Waits to be assigned A dev opens the project Fix and open a PR 06
11. HOW A BUG GOT FIXED The classic cycle 01 02 03 04 05 06 neverstop QA finds a bug Writes a ticket Waits to be assigned A dev opens the project Fix and open a PR Waits for review 06
12. Nothing here is wrong. For a real feature, this is exactly right. neverstop 06
13. THE SAME CYCLE Where the time actually goes 01 02 QA finds a bug HANDOFF Writes a ticket HANDOFF 03 04 Waits to be assigned HANDOFF A dev opens the project HANDOFF 05 06 neverstop Fix and open a PR HANDOFF Waits for review 07
14. For a small bug, the handoffs cost more than the fix. 07
15. SCOPE · 1 OF 3 The bugs were small Some logic not working A total that does not add up. A state that does not reset. neverstop 08
16. SCOPE · 2 OF 3 The bugs were small UI that does not match the design Wrong colour token, wrong icon, wrong spacing. neverstop 08
17. SCOPE · 3 OF 3 The bugs were small Navigation issues Back goes to the wrong place. A screen you cannot leave. neverstop 08
18. None of this is architecture. Most of it is one line. neverstop 08
19. 02 The script
20. We were short on release. Still had features to develop, and bugs to fix, and sometimes really small bugs. neverstop 10
21. MY WORKAROUND What I did instead 01 neverstop Copy the QA description 11
22. MY WORKAROUND What I did instead 01 Copy the QA description 02 Leave my own worktree neverstop 11
23. MY WORKAROUND What I did instead 01 Copy the QA description 02 Leave my own worktree Create a new one 03 neverstop 11
24. MY WORKAROUND What I did instead 01 Copy the QA description 02 03 Leave my own worktree Create a new one 04 Give Claude the same text neverstop 11
25. MY WORKAROUND What I did instead 01 Copy the QA description 02 Leave my own worktree Create a new one 03 04 05 neverstop Give Claude the same text Go back to my worktree 11
26. It was faster than fixing it myself. It was still a context switch. neverstop 11
27. Claude fixed the bug. I still switched context. neverstop 12
28. THE FIRST VERSION So I cut out the middle man 01 neverstop Mention it in Slack 13
29. THE FIRST VERSION So I cut out the middle man 01 Mention it in Slack 02 It starts a session in a fresh worktree neverstop 13
30. THE FIRST VERSION So I cut out the middle man 01 Mention it in Slack 02 It starts a session in a fresh worktree It opens a PR, I review later 03 neverstop 13
31. AhmedBot ∞ Named after me, because it was mine neverstop 13
32. THE FIRST THREAD Message 1 of 3 QA WRITES IT THE WAY SHE ALWAYS DID M Mia 5:11 PM Only one gift card showing in till summary Pay one basket with two gift cards, 15 and 30. The till summary shows 30 instead of 45. neverstop 14
33. THE FIRST THREAD Message 2 of 3 BUT A DEVELOPER STILL HAD TO POINT AT THE CODE A Ahmed 5:14 PM I think we need to look into FiscalDayTrackingServiceImpl and check why there is an issue calculating the gift card @AhmedBot fix neverstop 14
34. THE FIRST THREAD Message 3 of 3 IT ANSWERS WITH THE CAUSE, NOT JUST A PATCH ∞ AhmedBot APP 5:14 PM Implementing the fix Running Detekt (lint check) Creating pull request Root cause: the gift-card branch accumulated onto creditCardAmount , so every new gift card restarted the sum. Fix: one line — accumulate onto giftCardAmount instead. PR opened neverstop 14
35. WHAT I WROTE AT THE TIME The bot participated in around 7 bugs. It fixed most of them, and when it failed, it still provided a strong starting point. In some cases, it quickly proved that a specific solution was not correct, saving time. neverstop 15
36. 7 FIRST WEEK bugs handled by a script one person wrote in the middle of a release. neverstop 16
37. 03 When a script becomes infrastructure
38. “Can you start AhmedBot? I know you are on vacation.” 18
39. WHAT THAT MESSAGE MEANT Five constraints 01 neverstop Only ran while my laptop was awake 19
40. WHAT THAT MESSAGE MEANT Five constraints 01 02 neverstop Only ran while my laptop was awake One Mac, too many Gradle builds 19
41. WHAT THAT MESSAGE MEANT Five constraints 01 02 03 neverstop Only ran while my laptop was awake One Mac, too many Gradle builds Every project validates differently 19
42. WHAT THAT MESSAGE MEANT Five constraints 01 02 03 04 neverstop Only ran while my laptop was awake One Mac, too many Gradle builds Every project validates differently No machine holds every toolchain 19
43. WHAT THAT MESSAGE MEANT Five constraints 01 02 03 04 05 neverstop Only ran while my laptop was awake One Mac, too many Gradle builds Every project validates differently No machine holds every toolchain An agent with shell access 19
44. CONSTRAINT 01 The team asked me to start it while I was on vacation, and while I was sick. It had to become a service, not a person. neverstop 20
45. CONSTRAINT 02 Several people mentioned it at once and my machine fell over. A queue, with limits per platform and per project. neverstop 21
46. CONSTRAINT 03 Android needs Gradle. Flutter needs FVM. Web needs npm. A fix nobody verified is not a fix. Every project declares how to prove its own build. neverstop 22
47. CONSTRAINT 04 Installing every one of those toolchains on the same machine was never going to hold. One image per platform. neverstop 23
48. CONSTRAINT 05 An agent with shell access was running directly on a machine with everything else on it. One container per job. Deleted when it finishes. neverstop 24
49. ONE MORE THING HAD TO CHANGE A bot named after one person cannot belong to everyone AhmedBot Phantom It works in a room it is not in. You only ever see the pull request. neverstop 25
50. 04 The new cycle
51. FOUR MONTHS LATER Message 1 of 2 NO TICKET. NO DEVELOPER. J Jana 12:15 PM [Bug] Inbox: “Return” option incorrect colour — should be the colour as designed: Policy documentation is ready Return Delete Inbox — swipe actions, as designed @Phantom fix neverstop 27
52. FOUR MONTHS LATER Message 2 of 2 IT NAMED THE EXACT WRONG TOKEN Phantom APP 12:18 PM Fixed! The swipe action used colorScheme.tertiary instead of colorScheme.tertiaryContainer . PR opened — one line changed. neverstop 27
53. WHAT WAS DIFFERENT THIS TIME No developer in the thread 01 neverstop The bug title is the bug report 27
54. WHAT WAS DIFFERENT THIS TIME No developer in the thread 01 The bug title is the bug report 02 Nobody added a technical pointer neverstop 27
55. WHAT WAS DIFFERENT THIS TIME No developer in the thread 01 The bug title is the bug report 02 Nobody added a technical pointer 03 The attached design is what it checked against neverstop 27
56. HOW A BUG GETS FIXED NOW The new cycle 01 neverstop QA posts in a thread 28
57. HOW A BUG GETS FIXED NOW The new cycle 01 QA posts in a thread 02 Mentions the bot neverstop 28
58. HOW A BUG GETS FIXED NOW The new cycle 01 QA posts in a thread 02 Mentions the bot 03 A verified PR appears neverstop 28
59. HOW A BUG GETS FIXED NOW The new cycle 01 QA posts in a thread 02 Mentions the bot 03 A verified PR appears 04 A developer reviews it neverstop 28
60. The developer is still here. Just at the end, where the judgement is. neverstop 28
61. 05 What it actually did
62. 378 FOUR MONTHS OF LOGS agent runs across 19 projects: Android, Flutter, iOS, JVM backends, .NET, Ruby and web. Most of that volume is not Android. neverstop 30
63. FOUR MONTHS OF LOGS Who is using it? yes, that one was the CEO neverstop QA 42.9% 152 runs Software Engineering 39.3% 139 runs Project Management 16.7% 59 runs DevOps & Infrastructure 0.8% 3 runs Leadership 0.3% 1 run 31
64. THE SAME DATA, TWO BARS Six out of ten runs came from someone who is not a software engineer. 60.7% 215 runs 39.3% 139 runs QA, project management, infrastructure Software engineers QA 152 · PM 59 · DevOps 3 · Leadership 1 Mobile 103 · Java 22 · .NET 11 · JavaSc The people closest to the bug are the ones sending it. neverstop 32
65. 3.4 MEDIAN, MENTION TO PULL REQUEST min Nine out of ten finish inside ten minutes. Short enough that QA stays in the thread and waits. neverstop 33
66. $0.68 COST PER RUN $255.85 of API spend, four months, the whole production bot. neverstop 34
67. WHY IT IS THAT CHEAP Only 2.7% of the bill is input we paid full price for 43% Cache reads 30% Cache writes Output 24% Uncached input, 2.7% The repository, the system prompt and the project rules are identical on every run. After the first time, the agent is mostly re-reading something it already paid for. neverstop 35
68. THE ONE WE DID NOT PLAN FOR 108 of 378 runs were not fixes. They were questions. 223 108 47 fix, change the code explain, understand it everything else We built something to change code. Almost a third of the value turned out to be letting people read it: a PM checking how a rule works, QA confirming expected behaviour before filing anything at all. neverstop 36
69. NOBODY DESIGNED THIS WORKFLOW Message 1 of 2 QA INVENTED THIS ON HER OWN M Marco 12:45 PM github.com/company/mobile-app/pull/171 github.com/company/mobile-app/pull/165 github.com/company/mobile-app/pull/166 github.com/company/mobile-app/pull/174 @Phantom build-android neverstop 37
70. NOBODY DESIGNED THIS WORKFLOW Message 2 of 2 ONE BUILD, FOUR FIXES, THIRTEEN MINUTES Phantom APP 12:58 PM APK build complete. Four branches merged, no conflicts. app-dev-release.apk 92.2 MB neverstop 37
71. 06 When it does not fix it
72. BACK TO THE FIRST WEEK A run that does not produce a fix still tells you something 01 02 03 Add the missing context Suggest another approach Take it seriously Someone drops in the detail the thread never had, and runs it again. A developer says where to look, in the same thread, without opening anything. This one is not small. It earned a developer, and now everyone knows that. Knowing an approach will not work is worth something. That is triage, not failure. neverstop 39
73. 07 Then the industry shipped it
74. WHAT HAPPENED NEXT Then the industry shipped it MAR Phantom’s first run A container per job on our own machine, fixing bugs from a Slack thread. neverstop 41
75. WHAT HAPPENED NEXT Then the industry shipped it MAR Phantom’s first run APR CodeRabbit Agent for Slack An agent that lives in the thread, investigates and opens PRs. Very close to Phantom. neverstop 41
76. WHAT HAPPENED NEXT Then the industry shipped it MAR Phantom’s first run APR CodeRabbit Agent for Slack JUN Anthropic ships Claude Tag Tag @Claude in a channel and delegate the task. neverstop 41
77. We were about three months early. That mostly proves the problem was real. neverstop 41
78. SO WHY KEEP OURS Three reasons that held up 01 It lives where the bug is reported QA and PMs do not open pull requests. They open Slack threads. neverstop 42
79. SO WHY KEEP OURS Three reasons that held up 01 It lives where the bug is reported 02 It runs on our own machine, on our own repos Including client code that cannot leave our infrastructure, and Azure DevOps repos most tools do not support. neverstop 42
80. SO WHY KEEP OURS Three reasons that held up 01 It lives where the bug is reported 02 It runs on our own machine, on our own repos 03 It reads the same rules our developers do The same project rule files a developer’s own agent reads, so the code it writes looks like ours. neverstop 42
81. Buy theirs, or build one that fits your team. Both are reasonable answers now. 42
82. 08 If you want to build one
83. MENTION TO PULL REQUEST What happens after you press enter Container the toolchain for that project Slack thread Orchestrator the bug, the images channel → project Back in the thread Orchestrator Tool server progress, cause, the PR the only Slack credentials the container has no Slack access neverstop Clone Agent works Validate 44
84. INSIDE THE BASE IMAGE The agent is baked in, and it cannot reach Slack ONE CONTAINER, ONE JOB THE TOOLS IT MAY CALL Claude Code SDK git, platform CLIs No Slack token Set the status Post a message Open a pull request ORCHESTRATOR Holds the only Slack credentials in the system ✕ no path from the container to your workspace Deleted when the job finishes. neverstop 45
85. THE IMAGES One base, one layer per platform base git, the CLIs, the agent android JDK 17, SDK 34 flutter needs the Android SDK maui The base image carries git, the platform CLIs and the agent itself. Everything above it is just a toolchain. Adding a platform means adding one layer, not redesigning anything. jvm · dotnet · ruby · web neverstop 46
86. THE ANDROID IMAGE Running Gradle when nobody is watching BASE git and the platform CLIs BASE The Claude Code SDK BASE No Slack token, no workspace access ANDROID JDK 17 AGP 8 will not run on anything older. Gradle toolchains can still fetch another JDK to compile with. ANDROID SDK 34, build tools 34 The platform the projects target, and nothing spare. Every extra package is minutes on a cold image. neverstop 47
87. WHY THE PULL REQUEST IS WORTH OPENING The agent has to prove its own fix Make the change Run the project's linter Three attempts to get clean. Each project names its own linter and its own compile step, so the agent runs what your CI would run. neverstop Read the errors, fix, retry Compile, then open the PR This is how it catches its own mistakes. 48
88. WHY IT IS MINUTES, NOT HALF AN HOUR Nothing clones from the network twice 01 A mirror per repository, kept warm 02 Jobs clone from the mirror, not the internet 03 One lock per repository First run on a large Android repository: twenty minutes. Every run after it: seconds. neverstop 49
89. So how does a new project get in? 50
90. REGISTERING A PROJECT Message 1 of 3 A REPOSITORY URL IS THE WHOLE REQUEST A Ahmed 12:51 PM let’s add this one github.com/company/kmp-toolkit @Phantom register neverstop 51
91. REGISTERING A PROJECT Message 2 of 3 THEN IT INTERVIEWS YOU Phantom APP 12:51 PM What platform does this project use? flutter neverstop android jvm ruby dotnet maui ios web 51
92. REGISTERING A PROJECT Message 3 of 3 FOUR QUESTIONS LATER Phantom APP 12:59 PM All done. kmp-toolkit is live. Lint ./gradlew detekt Tests ./gradlew allTests Anyone can trigger it with @Phantom toolkit fix neverstop 51
93. Nineteen projects are registered. I set up almost none of them. neverstop 51
94. THE HONEST GAP Where the architecture runs out iOS is a passenger Nothing here runs the app There is no Swift toolchain image yet, so those projects run on the plain base image. The agent reads and changes Swift, but it cannot compile it, so it cannot prove anything, and those pull requests are drafts. No emulator, no device, no screenshot of the fixed state. Compile and lint tell you the code is sane. They do not tell you the bug is gone. Someone still opens the app. neverstop 52
95. WHAT WE ACTUALLY BUILT Not one bot. A place to plug agents in. LIVE LIVE TESTED NEXT Fix a bug Ask the codebase Review a pull request Whatever we need next One orchestrator · one set of images · one validation loop neverstop T H E I N F R A ST R U C T U R E 53
96. 09 What we learned
97. IF YOU ARE THINKING ABOUT THIS Three things we learned 01 Match the tool to the size of the problem Small, well described bugs. Not architecture, not features. neverstop 55
98. IF YOU ARE THINKING ABOUT THIS Three things we learned 01 Match the tool to the size of the problem 02 Start with the small version A script that works is a better argument than a proposal. neverstop 55
99. IF YOU ARE THINKING ABOUT THIS Three things we learned 01 Match the tool to the size of the problem 02 Start with the small version 03 The API bill is not the cost Someone maintains this. Count that time before you call it cheap. neverstop 55
100. STILL TO DO Connect it to the ticketing system, so a fix in a thread closes the ticket that started it. 55
101. TAKE IT WITH YOU These slides are online speakerdeck.com/thisaay how-we-moved-from-android-studioto-slack Every number in this talk, and the parts I went past quickly. neverstop 56
102. OVER TO YOU Questions neverstop 56
103. Every step we automated made the fix faster. None of them made the handoff cheaper. So we deleted the handoff. Thank you. Ahmed Ali RATE THIS TALK neverstop