Hybrid AI with Apple Intelligence and Firebase AI Logic

Apple Intelligence brings powerful, private, and fast on-device models directly to our users. But what about users on older hardware? And what happens when a task is too complex or the input is too large for the local model's context window? We can't just show an error.

This is where a **Hybrid AI** strategy becomes essential.

In this session, we'll explore how to build a smart, resilient system that gets the best of both worlds: the instant speed of on-device inference and the power of large-scale cloud models. We'll build a practical solution in Swift from the ground up.

You will learn how to:

- Create a clean, protocol-oriented architecture to abstract local and remote inference services.
- Use Apple's new `FoundationModels` API to run fast, on-device generation.
- Implement a seamless fallback to a powerful cloud model, like Google's Gemini via Firebase AI.
- Strategically decide *when* to fall back—from simple OS availability checks to handling specific errors like `LMError.contextWindowExceeded`.

By the end of this talk, you'll have a robust pattern for building intelligent features that are fast, capable, and—most importantly—available to *all* your users, not just those on the latest hardware.

Transcript

1. @peterfriese.dev Created by Mamank from Noun Project https: / / peterfriese.dev peterfriese Peter Friese, Staff Developer Relations Engineer, Google Hybrid AI with Apple Intelligence and Gemini
3. NSLondon 2024
4. Codename Sofia ✨Add links via iOS Share Sheet ✨Extract OpenGraph Metadata ✨Simplified formatting ✨Offline reading
5. … in the meantime…
7. WWDC 2025
8. Foundation Models
9. On-device AI - benefits 1/ Available offline 2/ No cost 3/ Private
10. Summarisation w/ Apple Foundation Model let instructions = """ **Task:** Create a short, snappy, and informative summary of the following text. **Constraints:** - The summary must be a single paragraph, containing no more than 3 sentences. - Capture the single most important takeaway from the text. - Use clear, direct, and easy-to-understand language. **Text to Summarize:** """ let input = // fetched from database let session = LanguageModelSession(instructions: instructions) let response = try await session.respond(to: input) let responseText = response.content System instructions
11. Summarisation w/ Apple Foundation Model let instructions = """ **Task:** Create a short, snappy, and informative summary of the following text. **Constraints:** - The summary must be a single paragraph, containing no more than 3 sentences. - Capture the single most important takeaway from the text. - Use clear, direct, and easy-to-understand language. **Text to Summarize:** """ let input = // fetched from database let session = LanguageModelSession(instructions: instructions) let response = try await session.respond(to: input) let responseText = response.content
12. Summarisation w/ Apple Foundation Model let instructions = """ **Task:** Create a short, snappy, and informative summary of the following text. **Constraints:** - The summary must be a single paragraph, containing no more than 3 sentences. - Capture the single most important takeaway from the text. - Use clear, direct, and easy-to-understand language. **Text to Summarize:** """ let input = // fetched from database let session = LanguageModelSession(instructions: instructions) let response = try await session.respond(to: input) let responseText = response.content
13. Summarisation w/ Apple Foundation Model let instructions = """ **Task:** Create a short, snappy, and informative summary of the following text. **Constraints:** - The summary must be a single paragraph, containing no more than 3 sentences. - Capture the single most important takeaway from the text. - Use clear, direct, and easy-to-understand language. **Text to Summarize:** """ let input = // fetched from database let session = LanguageModelSession(instructions: instructions) let response = try await session.respond(to: input) let responseText = response.content Text we want to summarise
14. Summarisation w/ Apple Foundation Model let instructions = """ **Task:** Create a short, snappy, and informative summary of the following text. **Constraints:** - The summary must be a single paragraph, containing no more than 3 sentences. - Capture the single most important takeaway from the text. - Use clear, direct, and easy-to-understand language. **Text to Summarize:** """ let input = // fetched from database let session = LanguageModelSession(instructions: instructions) let response = try await session.respond(to: input) let responseText = response.content
15. Demo!
16. On-device AI - drawbacks 1/ Not available on all phones 2/ Not available in all regions 3/ Model not downloaded (yet) 4/ Feature disabled (MDM policy) 5/ Token window size
17. hybrid
18. /ˈhʌɪbrɪd/
19. /ˈhʌɪbrɪd/ 1/ 2/ Of mixed character; composed of different elements. Of mixed character; composed of different elements. A thing made by combining two different elements. A thing made by combining two different elements.
20. Requirements 1/ Automatic fallback 2/ Specify priority (local / remote) 3/ Unified API 4/ Extensible 5/ Configurable (e.g. prompts)
22. Strategy pattern
23. Inferencer /// A protocol defining the contract for a summarization provider. /// /// This allows for a strategy pattern where different summarization /// methods (e.g., local, remote) can be used interchangeably. @MainActor protocol Inferencer { / // The system instructions to be used by the generative model. var instructions: String { get } / // A boolean indicating whether the inferencer is available for use. var isAvailable: Bool { get } / // Performs the summarization. func infer(_ input: String) async throws - > String? } E.g. device specific availability
24. Inferencer /// A protocol defining the contract for a summarization provider. /// /// This allows for a strategy pattern where different summarization /// methods (e.g., local, remote) can be used interchangeably. @MainActor protocol Inferencer { / // The system instructions to be used by the generative model. var instructions: String { get } / // A boolean indicating whether the inferencer is available for use. var isAvailable: Bool { get } / // Performs the summarization. func infer(_ input: String) async throws - > String? } Perform the actual work
25. Inferencer /// A protocol defining the contract for a summarization provider. /// /// This allows for a strategy pattern where different summarization /// methods (e.g., local, remote) can be used interchangeably. @MainActor protocol Inferencer { / // The system instructions to be used by the generative model. var instructions: String { get } / // A boolean indicating whether the inferencer is available for use. var isAvailable: Bool { get } / // Performs the summarization. func infer(_ input: String) async throws - > String? } If you want to use custom instructions
26. Inferencer /// A protocol defining the contract for a summarization provider. /// /// This allows for a strategy pattern where different summarization /// methods (e.g., local, remote) can be used interchangeably. @MainActor protocol { / // The system instructions to be used by the generative model. get / // A boolean indicating whether the inferencer is available for use. get / // Performs the summarization. var instructions: String { } var isAvailable: Bool { } func infer(_ input: String) async throws - > String? Inferencer }
27. LocalSummarizer: Inferencer struct LocalSummarizer: { func infer(_ input: String) async throws - > String? Inferencer var instructions: String { } var isAvailable: Bool { } } { }
28. struct LocalSummarizer: { """ **Task:** Create a short, snappy, and informative summary of the following text. **Constraints:** - The summary must be a single paragraph, containing no more than 3 sentences. - Capture the single most important takeaway from the text. - Use clear, direct, and easy-to-understand language. **Text to Summarize:** """ LocalSummarizer: Inferencer var instructions: String { Inferencer func infer(_ input: String) async throws - > String? } var isAvailable: Bool { } { Same instructions as in first code sample
29. LocalSummarizer: Inferencer struct LocalSummarizer: { func infer(_ input: String) async throws - > String? Inferencer var isAvailable: Bool { } } { } var instructions: String { }
30. if #available(iOS 26.0, *) { return SystemLanguageModel.default.isAvailable } return false LocalSummarizer: Inferencer struct LocalSummarizer: { func infer(_ input: String) async throws - > String? Inferencer var isAvailable: Bool { } } { } var instructions: String { } Local model only available on iOS 26 and above
31. if #available(iOS 26.0, *) { return SystemLanguageModel.default.isAvailable } return false LocalSummarizer: Inferencer struct LocalSummarizer: { func infer(_ input: String) async throws - > String? Inferencer var isAvailable: Bool { } } { } var instructions: String { } Model might be disabled / not yet downloaded
32. { } if #available(iOS 26.0, *) { guard SystemLanguageModel.default.isAvailable else { throw SummarizationError.modelNotAvailable } let session = LanguageModelSession(instructions: instructions) let response = try await session.respond(to: input) let responseText = response.content return responseText } else { throw SummarizationError.modelNotAvailable } struct LocalSummarizer: { var instructions: String { LocalSummarizer: Inferencer func infer(_ input: String) async throws - > String? Inferencer } var isAvailable: Bool { } } Better safe than sorry - model might have become unavailable in the meantime!
33. { } if #available(iOS 26.0, *) { guard SystemLanguageModel.default.isAvailable else { throw SummarizationError.modelNotAvailable } let session = LanguageModelSession(instructions: instructions) let response = try await session.respond(to: input) let responseText = response.content return responseText } else { throw SummarizationError.modelNotAvailable } struct LocalSummarizer: { var instructions: String { LocalSummarizer: Inferencer func infer(_ input: String) async throws - > String? Inferencer } var isAvailable: Bool { } } Same code as in initial code sample for summarising text
35. RemoteSummarizer: Inferencer struct RemoteSummarizer: Inferencer { var isAvailable: Bool { / / network availability / feature flags / subscription status } private var model: GenerativeModel { let modelName = configuration.remoteModelName let firebaseAI = FirebaseAI.firebaseAI(backend: .vertexAI()) return firebaseAI.generativeModel( modelName: modelName, systemInstruction: ModelContent(role: "system", parts: [instructions]) ) } public func infer(_ input: String) async throws -> String? { let response = try await model.generateContent(input) let responseText = response.text return responseText } } Availability depends on other factors
36. RemoteSummarizer: Inferencer struct RemoteSummarizer: Inferencer { var isAvailable: Bool { / / network availability / feature flags / subscription status } private var model: GenerativeModel { let modelName = configuration.remoteModelName let firebaseAI = FirebaseAI.firebaseAI(backend: .vertexAI()) return firebaseAI.generativeModel( modelName: modelName, systemInstruction: ModelContent(role: "system", parts: [instructions]) ) } public func infer(_ input: String) async throws -> String? { let response = try await model.generateContent(input) let responseText = response.text return responseText } } Pro tip: use Remote Config to set model name!
37. RemoteSummarizer: Inferencer struct RemoteSummarizer: Inferencer { var isAvailable: Bool { / / network availability / feature flags / subscription status } private var model: GenerativeModel { let modelName = configuration.remoteModelName let firebaseAI = FirebaseAI.firebaseAI(backend: .vertexAI()) return firebaseAI.generativeModel( modelName: modelName, systemInstruction: ModelContent(role: "system", parts: [instructions]) ) } public func infer(_ input: String) async throws -> String? { let response = try await model.generateContent(input) let responseText = response.text return responseText } } Generate summary using Firebase AI Logic
38. SummarizerService public func summarize(_ input: String) async throws -> String? { var lastError: Error? let priorityOrder = configuration.summarizerPriorityOrder for inferencerKey in priorityOrder { guard let inferencer = inferencers[inferencerKey], inferencer.isAvailable else { continue } do { if let result = try await inferencer.infer(input) { return result } } catch { logger.error("Inferencer \(type(of: inferencer)) failed: \(String(describing: error))") lastError = error / / Try the next available inferencer } } / / If all inferencers fail, throw the last error we encountered if let lastError { throw lastError } / / If no suitable inferencer was found at all throw SummarizationError.noSuitableInferencerFound } Iterate over registered inferences
39. What ’ s next 1/ Use local model as “orchestrator” 2/ Hybrid AI in Firebase AI Logic 3/ AnyLanguageModel
41. mattt/AnyLanguageModel import let instructions = """ **Task:** Create a short, snappy, and informative summary of the following text. **Constraints:** - The summary must be a single paragraph, containing no more than 3 sentences. - Capture the single most important takeaway from the text. - Use clear, direct, and easy-to-understand language. """ let input = // fetched from database let session = LanguageModelSession(instructions: instructions) let response = try await session.respond(to: input) let responseText = response.content FoundationModels AnyLanguageModel Drop-in replacement
42. Thanks!
43. Thanks! @peterfriese.dev Created by Mamank from Noun Project https: / / peterfriese.dev peterfriese peterfriese Slides for this talk
44. Q&A @peterfriese.dev Created by Mamank from Noun Project https: / / peterfriese.dev peterfriese peterfriese