[Droidcon Orlando '26] The Android Lens: Applying Mobile Forensics to AI Performance

Modern LLMs like Ollama are technically ground-breaking but suffer from significant thermal and energy inefficiencies on resource-constrained hardware. This is often an overlooked cost of LLM's deterministic nature of token generation along with the heavy, unoptimized CPU operations within the underlying math engines.

High energy demand translates to high water usage and thermal dissipation needs. For many communities, this environmental footprint makes local AI inaccessible or unsustainable. To solve this, we must adopt a more "frugal" philosophy, the way Android development does.

This talk explores the forensics of loading Llama 3.2 (1B) onto a Raspberry Pi 4 to emulate resource constraint conditions. Through the lens of Android and Kotlin multiplatform development - molded and developed in its nature of resource-constrained hardware - we will audit Ollama's source code and profile how it performs in real time. We will move past the high-level Go wrappers and into the unforgiving C++ threading and memory management, identifying the leaks and bottlenecks leading AI to drown in thermal throttling and hallucinations.

Attendees will learn:
- How to apply mobile performance patterns (like scoped locking) to AI engines.
- How to monitor real-time hardware telemetry (thermal/RAM) against token generation speed.
- Why "frugal computing philosophy" is an ethical necessity for sustainable AI deployment needed to save our fresh water resources.

Transcript

1. The Android Lens: Applying Mobile Forensics to AI Performance Amanda Hinchman-Dominguez Droidcon Orlando 2026
2. What are Amino Acids? Introduction to DNA 2 • Assumes a unlimited resource fungibility in computing. "Scaling horizontally" is a Western ideology
3. 3 Web, backend, desktop, and all AI/ML are built on unbounded runtime + resources • Scaling horizontally - "Just throw more hardware at it" • Ignores the basic laws of nature: that there are finite resources to extract. The Android Lense: Applying Mobile Forensics to AI Performance @amanda-hinchman.bsky.social @amanda
4. What are Amino Acids? Introduction to DNA 4 AI has a resource crisis. We can no longer afford inefficient code, and it's costing us.
6. All development platforms leans on horizontal scaling.... except for mobile. Android, specifically. Android engineers have spent a decade surviving: • Limited RAM and computing power • Aggressive OS Process Killing • Complicated life cycles prone to memory leaks 6 The Android Lense: Applying Mobile Forensics to AI Performance @amanda-hinchman.bsky.social @amanda
7. What are Amino Acids? Introduction to DNA 7 AI needs Android developers to solve their largest problems today: performance
8. 8 Applying Android Performance Frameworks to AI Once you see it, you won't be able to unsee it. The Android Lense: Applying Mobile Forensics to AI Performance @amanda-hinchman.bsky.social @amanda
9. 9 Performance Framework - Use Tools to Confirm! 2. Profiling data + interpreting data to diagnose issues 1. Recognize the signs of performance bottleneck 3. Targeted fixes using patterns The Android Lense: Applying Mobile Forensics to AI Performance 4. Validate results with data @amanda-hinchman.bsky.social @amanda
10. Recognizing the signs of performance bottlenecks
11. 11 Performance Bottlenecks in Android - Water in a Measuring cup Measuring Cup Metaphor Technical State Customer Complaint Level 5 OOMs/Overflow OS crashes/terminates the process repeatedly [Insert curse words]. Crashes every 5 minutes. Uninstalling. Leaving you for another vendor. Level 4 Brimming Crashes are symptomatic - different kinds of crashes occur consistently System "always very slow", freezes during busy/over long periods of time. New install, churning after a few days. Level 3 Half-Full OS slow + laggy. Screen is stuck Level 2 Rising Water The OS listener or small object is retained Lagging Level 1 Empty Cup Minor leak. A single listener or small object retained Gets slow over time but resolved when turned on/off
12. 12 Performance Bottlenecks in AI - Radio Signal Radio Signal Metaphor Technical State What it looks like to you Level 5 - Complete process failure Total Static/Dead Air API times out. Context window overflows. Model enters infinite output loop until terminated. Am I listening to other conversations? Why are all models ignoring my prompts and unreadable? Level 4 - Severe behavioral regression Heavy Inference Model completely ignores system prompts, breaks JSON schemas, constantly hallucinates It's generating gibberish code, ignoring half of my instructions :( Level 3 - Heavy latency + gets lost in the middle Fading Signal Time to First Token spikes. Model begins omitting details from prompts. Excessively repeats phrases. Feels slow and starting to give repetitive answers. I have to keep reminding it of things I said. Level 2 - Verbosity drift + compliance issues Mild Crackle Bloated output length. Model over-refuses prompts. It's getting real wordy. Level 1 - Minor, isolated edge-case failures. Solid Reception Model works for like 95% of tasks. Works well, but I need to doublecheck details.
13. 1. Use Profiling to Target Critical Areas
14. Profiling for Android Performance - AS Profiling 14 The Android Lense: Applying Mobile Forensics to AI Performance
15. Profiling for Android Performance - Perfetto 15 The Android Lense: Applying Mobile Forensics to AI Performance
16. Framework for using Android Tooling 16 Problem-Solving Methodology 1. Track Memory Consumption with AS Memory Profiler - Look at how memory moves to see if memory is being held hostage. 2. Find the largest culprits with AS Memory Usage 3. Diagnose with LeakCanary - gives you the direct path Fluctuates: Memory is allocated and released correctly Constantly Rising: Memory is allocated but never released (leaks!) Healthy Memory Management: Rollercoaster Unhealthy Memory Management: Stairs @amanda-hinchman.bsky.social The Android Lense: Applying Mobile Forensics to AI Performance @amanda
17. Identifying Red Flags in Android Performance Degradation What to Look For • Steady climb in memory graphs • Staircase effect - each step up usually represents Fragment/Activities opened but never fully released • Even after Garbage Collection, the baseline continues to rise • As memory fills up, the Android OS works harder to find space (the cup stays full) • Leads to Jank (dropped frames) and eventually OOM crashes Clues Pointing to Performance Degradation 17 @amanda-hinchman.bsky.social The Android Lense: Applying Mobile Forensics to AI Performance @amanda
18. Profiling for AI Models Created a CMP baseline and visual reporting with Ollama benchmarking and RAGAS evaluation to track: • CPU • Temperature by Core • Token Expenditure • Gen Speed • Token Speed/Ingestion • Hallucination Score @amanda-hinchman.bsky.social @amanda
20. 2. Target bottlenecks with pattern matching
21. 21 Look for patterns that matches the following criteria Effort: LOW Impact: HIGH/MED Risk: LOW The Android Lense: Applying Mobile Forensics to AI Performance @amanda-hinchman.bsky.social @amanda
22. 22 Android Specific Patterns 1. Statically-saved Android UI components Remove static referencing 2. Android UI interaction/DI within non-Android classes Move UI interaction to Activity/Fragment level 3. Android UI references in background threads Remove Android UI from background threads The Android Lense: Applying Mobile Forensics to AI Performance @amanda-hinchman.bsky.social @amanda
23. Memory Leaks & Performance Considerations: An Android Cookbook Amanda Hinchman-Dominguez Kotlin GDE 20-24
24. 24 Generic Patterns for Performance Improvements 1. Deadlock on Exceptions w/ Mutex - Naked lock/unlock calls - Globally exposed Mutex - Restrict access to mutex calls - Ensure ordering of locks are reflective - Kotlin: Scope-safe locks i.e. mutex.withLock {...} 2. Blocking on an unbuffered channel - wrap in a non-blocking select - included cancellation/error handling - Kotlin: use cooperative cancellation 3. Reduce the for-looping on intensive math - flatten the number of for-loops The Android Lense: Applying Mobile Forensics to AI Performance @amanda-hinchman.bsky.social @amanda
25. 1. Deadlock Exceptions on Mutex ⟶ add guardrails #include "ggml-threading.h" #include <mutex> std::mutex ggml_critical_section_mutex; void ggml_critical_section_start() { ggml_critical_section_mutex.lock(); } void ggml_critical_section_end(void) { ggml_critical_section_mutex.unlock(); } ml/backend/ggml/ggml/src/ggml-threading.cpp The Android Lense: Applying Mobile Forensics to AI Performance 25 @amanda-hinchman.bsky.social • Naked deadlocks = vulnerable to deadlock if there is: ◦ an early return ◦ breaks out mid-loop ◦ the code deadlocks • The Rule: always prefer RAII guards (i.e. std::lock_guard) @amanda
26. 1. Deadlock Exceptions on Mutex ⟶ restrict access #include "ggml-threading.h" #include <mutex> std::mutex ggml_critical_section_mutex; void ggml_critical_section_start() { ggml_critical_section_mutex.lock(); } void ggml_critical_section_end(void) { ggml_critical_section_mutex.unlock(); } ml/backend/ggml/ggml/src/ggml-threading.cpp The Android Lense: Applying Mobile Forensics to AI Performance 26 @amanda-hinchman.bsky.social • Globally exposed mutexes ◦ Danger: Allows any other file to link/call on it ◦ The Rule: Mutexes should be as private as possible ▪ class members ▪ declare static to restrict scope @amanda
27. 1. Deadlock Exceptions on Mutex (Fixed) #include "ggml-threading.h" #include <mutex> std::mutex ggml_critical_section_mutex; void ggml_critical_section_start() { ggml_critical_section_mutex.lock(); } void ggml_critical_section_end(void) { ggml_critical_section_mutex.unlock(); } static std::mutex ggml_critical_section_mutex; class GGMLCriticalSection { public: GGMLCriticalSection() : lock(ggml_critical_section_mutex) {} private: std::lock_guard<std::mutex> lock; }; void process_ggml_tensors() { GGMLCriticalSection critical_section; } ml/backend/ggml/ggml/src/ggml-threading.cpp The Android Lense: Applying Mobile Forensics to AI Performance 27 @amanda-hinchman.bsky.social @amanda
28. @amanda-hinchman.bsky.social @amanda ml/backend/ggml/ggml/src/ggml-threading.cpp
29. 1. Deadlock Exceptions on Mutex (but Kotlin) val mutex = Mutex() suspend fun processTensors() { mutex.lock() try { nestedHelper() // DEADLOCK } finally { mutex.unlock() } } suspend fun nestedHelper() { mutex.lock() /* shortened for brevity */ mutex.unlock() } The Android Lense: Applying Mobile Forensics to AI Performance 29 @amanda-hinchman.bsky.social • Danger: Kotlin Mutexes are NOT reentrant. ◦ If a coroutine holds a lock and tries to acquire it again, it holds forever • Rule: use .withLock { ... } ◦ Guarantees the lock is released even on exception @amanda
30. 1. Deadlock Exceptions on Mutex (but Kotlin) val mutex = Mutex() suspend fun processTensors() { mutex.lock() try { nestedHelper() // DEADLOCK } finally { mutex.unlock() } } suspend fun nestedHelper() { mutex.lock() /* shortened for brevity */ mutex.unlock() } val mutex = Mutex() suspend fun processTensors() { mutex.withLock { nestedHelperUnsafe() } } private fun nestedHelperUnsafe() { // ... do work ... } The Android Lense: Applying Mobile Forensics to AI Performance 30 @amanda-hinchman.bsky.social @amanda
31. 2. Blocking on an Unbuffered Channel ⟶ use select ml/backend/ggml/ggml/src/ggml-threading.cpp The Android Lense: Applying Mobile Forensics to AI Performance 31 @amanda-hinchman.bsky.social go func() { /* shortened for brevity*/ ch <- result // <-----Naked send! }() • Danger: ch <- v is a blocking operation ◦ If there is a loss in connection/cancellation, the main handler stops reading and exits • Rule: use sendCh that wraps channel send in non-blocking .select ◦ Forces waiting on multiple channel operations @amanda
32. 2. Blocking on an Unbuffered Channel ⟶ use select The Android Lense: Applying Mobile Forensics to AI Performance 32 @amanda-hinchman.bsky.social sendCh := func(v any) bool { select { case ch <- v: return true case <- c.Request.Context().Done(): return false } } @amanda • Danger: ch <- v is a blocking operation ◦ If there is a loss in connection/cancellation, the main handler stops reading and exits • Rule: use sendCh that wraps channel send in non-blocking .select ◦ Forces waiting on multiple channel operations
34. 2. Blocking on an Unbuffered Channel (in Kotlin) The Android Lense: Applying Mobile Forensics to AI Performance 34 @amanda-hinchman.bsky.social val channel = Channel<String>() scope.launch { try { val result = performHeavyMath() channel.send(result) } catch (e: Exception) { channel.send("Error: ${e.message}") } } • Danger: If the collector has cancelled/disconnected, this suspends forever ◦ Coroutine Leaks! • Rule: Cancel the scope ◦ Or use Kotlin's select<*> to automatically disconnect @amanda
36. Validating results with profiling
37. Validating Results Android Profiling Fixes Once identifying the culprits and fixing them, make sure there is a hypothesis of expected behavior to validate against. • Before a release ◦ What is the memory/CPU consumption after? ◦ What is the time of screen rendering and navigation? ◦ Benchmarking results with scripts/logging or use tools like Macrobenchmarking in Android • After a release ◦ How many reported crashes? ◦ How many customer support cases for the same area are made? ◦ Read for customer feedback as well. ◦ Check for revenue @amanda-hinchman.bsky.social @amanda
38. Benchmarking Design for Ollama-Forensics-Monitoring Why run benchmarking? Allows for comparative runs in different scenarios so we can compare between a baseline and post-performance improvement Test Goal/Prompt Strategy What it Stresses Expected Failure Mode 1. Baseline Control Standard text generation without strict performance constraints. Normal token generation speed (t/s). Should maintain a low/zero hallucination index. 2. Adversarial Math Forcing explicit numeric telemetry claims inside the text output. Factual accuracy under forced constraints. High probability of hallucination if unchecked. 3. Concurrency Stress Forcing complex transaction code while banning core Java tokens Instruction-following under high prompt cognitive load. Fallback tokens leaking into bytecode. @amanda-hinchman.bsky.social @amanda
43. 43 Final Thoughts We could wait until we launch AI into space to finally solve our resource issues. Or, we could start leveraging Android developers to solve performance bottlenecks today - on the hardware we already have. The Android Lense: Applying Mobile Forensics to AI Performance
44. 44 Access slides, workshop info, and research here: Thanks for attending my talk! The Android Lense: Applying Mobile Forensics to AI Performance Thanks for attending my talk!