5000万ダウンロードを超える漫画サービスを支えるログ基盤の設計開発の全て

「iOSDC Japan 2025」の登壇資料です。
https://iosdc.jp/2025/

Transcript

1. 5000 支 iOSDC Japan 2 02 5 202 5 . 0 9 . 21 10 : 3 0 Track C @dsxsxsxs © LINE Digital Frontier Corporation
2. 自己 人 日 LINE Digital Frontier VTuber 　 🏍💨 @dsxsxsxs
3. 日 行 Swift 6 . 1 . 0 行 Firebase Remote Config
4. Source Repository: https://github.com/dsxsxsxs/Tracker
5. 行
8. SDK
11. 行 面 Screen Impression Tap etc … UX ⾒ 方
12. 行 行 行 行
13. 力
14. Tracker Concurrent 非 Queueing, Priority: Utility gzip, deflate Exponential Backoff [String: Sendable] Swift Package swift-tools-version: 6 . 1 . 0
15. Swift 6 Sendable ⾒ actor Sendable ⾒ ⾒ @unchecked Sendable @unchecked Sendable @MainActor
16. Digest Loop 一 while dataStore.haveData() { let dataToSend = dataStore.getData(count: chunkCount) do { try await networkClient.send(data: dataToSend) dataStore.deleteData(ids: dataToSend.map(\.id)) } catch { stopDigesting() break } }
18. Entity(Domain Object) TrackingData Use Case(Interactor, Controller) TrackingDispatcher Data Layer TrackingSQLiteDataStore, TrackingNetworkClient Public Interface Tracker
19. Clean
20. Clean
21. Clean
22. public struct TrackingData: Sendable { let id: Int let data: Data } Entity - TrackingData
23. Use Case
24. public protocol TrackingDispatcherProtocol: Sendable { func sendLogs(name: String, payloads: [[String: Sendable]]) } public protocol TrackingDataStoreProtocol: Sendable { func save(data: [Data]) throws -> [TrackingData] func getData(count: Int) throws -> [TrackingData] func deleteData(ids: [Int]) throws } public protocol TrackingNetworkClientProtocol: Sendable { func send(data: [TrackingData]) async throws } Use Case All Sendable!!!!!
25. Data Layer TrackingSQLiteDataStore TrackingNetworkClient Public Interface Tracker
26. DataStore
27. DataStore SQLite (String ) private let tableName = "Tracking" private let primaryKey = "id" private let dataKey = "rawData" 
 CREATE TABLE IF NOT EXISTS \(tableName) ( \(primaryKey) INTEGER PRIMARY KEY AUTOINCREMENT, \(dataKey) BLOB );
28. DataStore - SQLite iOS
29. DataStore - Sendable import SQLite3 public final class TrackingSQLiteDataStore: TrackingDataStoreProtocol { private var db: OpaquePointer? public func save(data: [Data]) throws -> [TrackingData] { public func selectLogs(limit: Int) throws -> [TrackingData] { public func deleteLogs(ids: [Int]) throws { } Stored property 'db' of 'Sendable'-conforming class 'TrackingSQLiteDataStore' has non-sendable type 'OpaquePointer?' OpaquePointer͸SendableͰ͸ͳ͍ OSAllcatedUnfairLockͰϥοϓͯ͠΋μϝ SE 03 31 - Remove Sendable conformance from unsafe pointer types
30. DataStore final class DatabasePointer: @unchecked Sendable { private let lock = NSRecursiveLock() private var _pointer: OpaquePointer? var pointer: OpaquePointer? { get { defer { lock.unlock() } lock.lock() return _pointer } set { defer { lock.unlock() } lock.lock() _pointer = newValue } } } Workaround 😅
31. DataStore public final class TrackingSQLiteDataStore: TrackingDataStoreProtocol { private let databasePointer = DatabasePointer() private var db: OpaquePointer? { get { databasePointer.pointer } set { databasePointer.pointer = newValue } } public func save(data: [Data]) throws -> [TrackingData] {} public func getData(count: Int) throws -> [TrackingData] {} public func deleteData(ids: [Int]) throws {} }
32. DataStore - Test private let sut = TrackingSQLiteDataStore() @Suite(.serialized) final class TrackingDataStoreTests { func save10Logs() throws { // sutʹLogΛ10݅อଘ.... } func delete10Logs() throws { // sutʹLogΛ10݅࡟আ.... } } Swift Testing Concurrent
33. NetworkClient
34. NetworkClient Networking 入 5KB body protocol TrackingNetworkClientNetworking: Sendable { func execute(request: URLRequest) async throws -> (Data, URLResponse) } private let minimumDeflateSize: Int32 = 5120
35. NetworkClient public final class TrackingNetworkClient: TrackingNetworkClientProtocol { let networking: TrackingNetworkClientNetworking let suspend: @Sendable (Int) async throws -> Void let maxRetryCount: Int public func send(data: [TrackingData]) async throws {} } Networking retry 入 5KB body
36. NetworkClient - Send - Compression public func send(data: [TrackingData]) async throws { var request = URLRequest(url: urlComponents.url!) // URLRequestΛઃఆ.... let rawData: Data = try data.asData() if rawData.count > minimumDeflateSize, let deflated = try? rawData.deflated() { request.httpBody = deflated request.setValue("deflate", forHTTPHeaderField: "Content-Encoding") request.setValue("\(deflated.count)", forHTTPHeaderField: "Content-Length") } else { request.httpBody = rawData request.setValue("\(rawData.count)", forHTTPHeaderField: "Content-Length") }
37. NetworkClient - Send - Exponential Backoff var retryAttempts = 0 repeat { do { if retryAttempts > 0 { let delay = pow(2.0, Double(retryAttempts)) try await suspend(Int(delay)) } try await networking.execute(request: request) break } catch { self.holdError(error) retryAttempts += 1 } } while retryAttempts <= maxRetryCount try throwErrorIfNeeded() }
38. NetworkClient - Compression import zlib extension Data { private static let chunk = 1 << 14 // 16384bytes func deflated() throws -> Data { var stream = z_stream() var status: Int32 // লུ var data = Data(capacity: Self.chunk) // লུ data.count = Int(stream.total_out) return data } } zlib 用 Gzip, deflate Compression Framework
39. final class TrackingNetworkClientTests: @unchecked Sendable { var suspendSeconds: [Int] = [] private var networking: ImmediateNetworking! private var sut: TrackingNetworkClient! init() { self.networking = ImmediateNetworking(statusCode: 200) sut = TrackingNetworkClient( maxRetryCount: 3, suspend: { self.suspendSeconds.append($0) }, networking: networking ) } NetworkClient - Test
40. NetworkClient - Test @Test func failHTTP400ThenRetry3Times() async throws { // লུ await #expect { try await self.sut.send(data: oneData) } throws: { error in let nsError = error as NSError return nsError.code == 400 } #expect(networking.receivedRequests.count == 4) #expect(suspendSeconds == [2, 4, 8]) }
41. Dispatcher
42. Dispatcher actor TrackingDispatcher: TrackingDispatcherProtocol { private let dataStore: TrackingDataStoreProtocol private let networkClient: TrackingNetworkClientProtocol private let errorHandler: TrackingErrorHandler private(set) var digestingTask: Task<Void, Never>? var isDigesting: Bool { digestingTask != nil } }
43. Dispatcher actor TrackingDispatcher: TrackingDispatcherProtocol { private let dataStore: TrackingDataStoreProtocol private let networkClient: TrackingNetworkClientProtocol private let errorHandler: TrackingErrorHandler private let executor: LogSerialExecutor private(set) var digestingTask: Task<Void, Never>? let unownedExecutor: UnownedSerialExecutor var isDigesting: Bool { digestingTask != nil } public init(dataStore: TrackingDataStoreProtocol, networkClient: TrackingNetworkClientProtocol, errorHandler: @escaping TrackingErrorHandler) { self.dataStore = dataStore self.networkClient = networkClient self.errorHandler = errorHandler executor = LogSerialExecutor() self.unownedExecutor = executor.asUnownedSerialExecutor() }
44. Dispatcher - Executer private final class LogSerialExecutor: SerialExecutor { private let serialQueue = DispatchQueue(label: "Executor", qos: .utility) nonisolated func enqueue(_ job: UnownedJob) { serialQueue.async { job.runSynchronously(on: self.asUnownedSerialExecutor()) } } func asUnownedSerialExecutor() -> UnownedSerialExecutor { UnownedSerialExecutor(ordinary: self) } Actor ⾒ 止
45. await Task.yield() Dispatcher - Digest Loop func startDigesting() { if self.isDigesting { return } digestingTask = Task(priority: .utility) { [weak self] in guard let self else { return } var shouldContinue = true while shouldContinue { shouldContinue = await self.digestDataStore() sleep k } await self.stopDigesting() } }
46. Dispatcher private func digestDataStore() async -> Bool { guard isDigesting else { return false } if Task.isCancelled { return false } do { let dataToSend = try dataStore.getData(count: 50) if dataToSend.isEmpty || Task.isCancelled { return false } try await networkClient.send(data: dataToSend) let ids = dataToSend.map { $0.id } try dataStore.deleteData(ids: ids) return true } catch { errorHandler(error) return false } } re-entry actor
47. Dispatcher func saveAndStartDigesting(data: [Data]) { do { _ = try dataStore.save(data: data) startDigesting() } catch { errorHandler(error) } } private func stopDigesting() { digestingTask?.cancel() digestingTask = nil } Digest Loop Digest Loop Clean up
48. Dispatcher nonisolated func sendLogs(name: String, payloads: [[String: Sendable]]) { if payloads.isEmpty { return } guard let encoded = try? Self.makeLogs(name: name, payloads: payloads) else { return } Task.detached(priority: .utility) { await self.saveAndStartDigesting(data: encoded) } } protocol ⾒ Loop
49. Dispatcher - Test final class TrackingDispatcherTests: @unchecked Sendable { private let dataStore = MockDataStore() private let networkClient = MockNetworkClient() private var sut: TrackingDispatcher! init() throws { sut = .init( dataStore: dataStore, networkClient: networkClient, errorHandler: { _ in } ) }
50. Dispatcher - Test @Test func digestLoopStopWhenEmpty() async throws { let data70 = Array(repeating: Data(), count: 70) _ = try dataStore.save(data: data70) dataStore.operations = [] await sut.saveAndStartDigesting(data: [Data()]) await sut.digestingTask?.value #expect(dataStore.operations == [ .insert, .select, .delete, .select, .delete, .select ]) #expect(networkClient.sentDataList.count == 2) #expect(networkClient.sentDataList.flatMap { $0 }.count == 71) } 70 入 1 入 Loop expect: 71 2
51. Dispatcher - Test func testConcurrentDigestCalls() async throws { // sutʹLogΛ10݅อଘ͓ͯ͘͠.... await withTaskGroup(of: Void.self) { group in group.addTask { await self.sut.saveAndStartDigesting(data: [Data()]) await self.sut.digestingTask?.value } group.addTask { await self.sut.startDigesting() } group.addTask { await self.sut.startDigesting() } } #expect(networkClient.sentDataList.flatMap { $0 }.count == 71) } expect: 71
52. Tracker
53. Tracker public final class Tracker: Sendable { let dispatcher: TrackingDispatcherProtocol let dataStore: TrackingDataStoreProtocol, let networkClient: TrackingNetworkClientProtocol public func sendLogs(name: String, payloads: [[String: Sendable]]) { dispatcher.sendLogs(name: name, payloads: payloads) } }
54. Tracker 方 let tracker = Tracker( dataStore: try TrackingSQLiteDataStore(), networkClient: try TrackingNetworkClient( maxRetryCount: 3, suspend: { try await Task.sleep(for: .seconds($0)) }, networking: TrackingNetworkClient.DefaultNetworking() ) ) tracker.sendLogs(name: "some_log_name", payloads: [ ["item_id": "12345", "event": "app_start"], ["item_id": "67890", "event": "app_end"] ]) Task.sleep URLSession
55. Test Coverage
57. 行
58. Dual Send Adapter protocol OldLogger { func logEvents(name: String, parameters: [[String: Any]]) } struct DualSendTracker: Sendable { let oldLogger: OldLogger let tracker: Tracker func sendLogs(name: String, parameters: [[String: Sendable]]) { oldLogger.logEvents(name: name, parameters: parameters) tracker.sendLogs(name: name, payloads: parameters) } }
59. 😭
60. Call ⾒ 大 入
61. func sendLogs(name: String, parameters: [[String: Sendable]]) { oldLogger.logEvents(name: name, parameters: parameters) tracker.sendLogs(name: name, payloads: editedParameters) } 大 入 var editedParameters: [[String: Sendable]] = parameters for (index, parameter) in editedParameters.enumerated() { if let someValue = parameter["some_key"] { editedParameters[index]["some_key"] = nil editedParameters[index]["some_new_key"] = someValue } if let someValue = parameter["some_key"] { editedParameters[index]["some_key"] = nil editedParameters[index]["some_new_key"] = someValue } if let someValue = parameter["some_key"] { editedParameters[index]["some_key"] = nil editedParameters[index]["some_new_key"] = someValue } if let someValue = parameter["some_key"] { editedParameters[index]["some_key"] = nil editedParameters[index]["some_new_key"] = someValue } if let someValue = parameter["some_key"] { editedParameters[index]["some_key"] = nil editedParameters[index]["some_new_key"] = someValue } if let someValue = parameter["some_key"] { editedParameters[index]["some_key"] = nil editedParameters[index]["some_new_key"] = someValue } if let someValue = parameter["some_key"] { editedParameters[index]["some_key"] = nil editedParameters[index]["some_new_key"] = someValue } if let someValue = parameter["some_key"] { editedParameters[index]["some_key"] = nil editedParameters[index]["some_new_key"] = someValue } if let someValue = parameter["some_key"] { editedParameters[index]["some_key"] = nil // ͦͷଞॾʑϚοϐϯά }
62. Call Call 生 Call 大
63. - 見
64. struct OldTapEventLog { let name: String let parameter: [String: Sendable] func send() { oldLogger.logEvents( name: name, parameters: [parameter] ) } } 立 面 extension Tracker { struct TapEvent { let name: String let payload: [String: Sendable] func send() { tracker.sendLog( name: name, payload: payload ) } } }
65. struct OldTapEventLog { let name: String let parameter: [String: Sendable] func send() { oldLogger.logEvents( name: name, parameters: [parameter] ) } func sendToNewTracker() -> Self { let tapEvent = Tracker.TapEvent(name: name, payload: parameter) tapEvent.send() return self } }
66. Call let tapEvent = OldTapEventLog( name: "home.like_button.tap", parameter: [ "item_id": "12345" ]) tapEvent.sendToNewTracker().send()
67. Call let logName = "home.like_button.tap" let itemID = “12345" let oldTapEvent = OldTapEventLog( name: logName, parameter: [ "item_id": itemID ] ) oldTapEvent.send() let tapEvent = Tracker.TapEvent( name: logName, payload: [ "new_item_id": itemID ] ) tapEvent.send() "item_id" "new_item_id"
68. 大 人生 見 payload
70. 行 見 小 止血 Hot Fix 小
71. 行
72. 行 方 二 止
73. Firebase Remote Config shouldSendLogToOldLogger: true default: false shouldSendLogToNewTracker: false default: true 行 Config struct RemoteConfig { let shouldSendLogToOldLogger: Bool let shouldSendLogToNewTracker: Bool }
74. 2024 10 月 true 100% v 2 4 . 1 1 . 1 0 false
75. 2025 1 月 false 100% v 2 5 . 0 4 . 1 0 5 ⾒ 5 % 5 % 月 25 % 0 ~ 5 % 月 50 % 50 % 月 75 % 75 % GW 10 0 % 10 0 %
76. Hot Fix ++ 　心 ++ Clean up
78. 🏁
79. 1 Digest Loop 　 　 Clean SOLID Swift 6 自
80. 2 行 見
81. 3 Firebase Remote Config true false false true Config Free ⾒
82. CPU 用 用 ⾒ Disk InMemory Data Store Reachability
83. References Source Repository: https://github.com/dsxsxsxs/Tracker zlib doc: https://zlib.net/manual.html Compression Framework: https://developer.apple.com/documentation/ Accelerate/compressing-and-decompressing-data-with-buffer-compression
84. https://connpass.com/event/ 3 6 98 2 6 / 10 月 17 日 ( 金 ) 19:00 @ 木　 11F
85. LINE 支
86. End Of doc.