TextKit 2 時代の iOS のキーボードとテキスト入力と表示のすべて

iOS は一見、テキスト入力や表示のような基本的実装はとても簡単に思えますが、実際には一度はその動作に頭を悩ましたことがあるでしょう。

iOSDC 2020 では「iOS のキーボードと文字入力のすべて」と題してこれらの問題について詳細に検討しました。
そして、この3年間で iOS は大きく変化し、SwiftUI が広く使われるようになり、全く新しいAPI、TextKit 2 が登場しました。
TextKit 2 は大きく変化しおり、互換性に関して注意すべき点が多くあります。
このセッションでは、過去の事例と比較して、多くのユーザが使う iOS アプリにおけるリッチテキストの実装などを踏まえて、
キーボードの挙動や SwiftUI、そして TextKit 2 によって変わったテキスト処理について実装例を用いて検討していきます。

対象とする方: iOS アプリ開発の経験がある中・上級者

Transcript

1. TextKit 2 ࣌୅ͷ iOS ͷΩʔϘʔυͱςΩετೖྗ ͱදࣔͷ͢΂ͯ @niw 9/2/2023 — Tokyo, Japan iOSDC Japan 2023
2. Yoshimasa Niwa @niw
3. ࣌͸ 2020 ೥…
4. iOS ͷΩʔϘʔυͱจࣈೖྗͷ͢΂ͯ @niw 09/21/2020 — Online iOSDC Japan 2020
5. iOS ͷςΩετදࣔͷ͢΂ͯ @niw 9/2/2023 — Tokyo, Japan iOSDC Japan 2023 TextKit 2 ରԠ൛ New!
6. TextKit 1
7. ಛ௃ TextKit 1 ݹ͔͘Βଓ͘ NSLayoutManager Λத৺ͱͨ͠ΫϥεͰߏ ੒͞ΕΔ API ΋ͱ΋ͱ TextKit 1 ͳͲͱ͸ݺ͹Ε͍ͯͳ͔͕ͬͨɺTextKit 2 ͷ API ͱ۠ผ͢ΔͨΊʹͦ͏ݺ͹ΕΔΑ͏ʹͳͬͨ
8. NSTextStorage NSLayoutManager NSTextContainer UITextView Ϗϡʔ ίϯτϩʔϥʔ Ϟσϧ
9. let textStorage = NSTextStorage() let layoutManager = NSLayoutManager() textStorage.addLayoutManager(layoutManager) let textContainer = NSTextContainer() layoutManager.addTextContainer(textContainer) let textView = UITextView( 
 frame: view.bounds, 
 textContainer: textContainer 
 )
10. ಛ௃ TextKit 1 ૢ࡞ͷ࠷খ୯Ґ͸จࣈͷཁૉ Ͱ͋ΔάϦϑ Ϟσϧʹ૬౰͢Δจࣈྻͷൣ ғͱରԠ͢ΔάϦϑͷૢ࡞͕ Մೳ ͋ A ಟ
11. let layoutManager = textView.layoutManager let glyphRange = layoutManager.glyphRange( forCharacterRange: NSRange( 
 location: 0, length: 1 
 ), actualCharacterRange: nil ) let rect = layoutManager.boundingRect( forGlyphRange: glyphRange, in: textView.textContainer )
12. ར༻ྫ
13. U+3042 U+3044 U+3046 U+3048 U+304A
14. U+0C85 U+0C95 U+0CCD U+0C9F U+0CCB U+0CAC U+0CB0 U+0CCD
15. Meet TextKit 2 (WWDC 21) 
 https://developer.apple.com/videos/play/wwdc2021/10061/
16. ͱ͍͏͔…
19. ·ͱΊ TextKit 1 จࣈྻͷൣғͱϏϡʔͷۣܗ͕άϦϑ୯ҐͰରԠ෇ΒΕΔ จࣈྻͷൣғͱάϦϑͷൣғ͕ରԠ͚ͮΒΕͳ͍ݴޠ͕ଘࡏ ͍ͯ͠Δ ਖ਼֬ʹಈ࡞͠ͳ͍έʔε͕͋Δ
20. TextKit 2
21. ಛ௃ TextKit 2 NSTextLayoutManager Λத৺ʹ৽͘͠௥Ճ͞Εͨ API Ͱɺ TextKit 1ͱ͸ಠཱͯ͠ಈ࡞͢Δ ΑΓਖ਼֬ʹಈ࡞͢ΔΑ͏ʹͳͬͨ ෆมͷ஋ΫϥεΛ࢖͏
22. NSTextStorage NSLayoutManager NSTextContainer UITextView Ϗϡʔ Ϟσϧ ίϯτϩʔϥʔ
23. NSTextContentStorage NSTextLayoutManager NSTextContainer UITextView Ϗϡʔ Ϟσϧ ίϯτϩʔϥʔ
24. let textContentStorage = NSTextContentStorage() let textLayoutManager = NSTextLayoutManager() textContentStorage 
 .addTextLayoutManager(textLayoutManager) let textContainer = NSTextContainer() textLayoutManager.textContainer = textContainer let textView = UITextView( 
 frame: view.bounds, 
 textContainer: textContainer )
25. จࣈྻͱϨΠΞ΢τ
26. ಛ௃ TextKit 2 ߴ౓ʹந৅Խ͞ΕͨΠϯλʔϑΣΠε TextKit 1 ͷૢ࡞ͷ࠷খ୯ҐͰ͋ͬͨάϦϑ͸Ӆṭ͞ΕɺάϦ ϑͷ৘ใ΍ͦͷۣܗͳͲ͸Ұ੾खʹೖΒͳ͘ͳͬͨ TextKit 2 ͷૢ࡞ͷ࠷খ୯Ґ͸ߦ
27. NSTextStorage
 NSAttributedString NSTextContentStorage NSTextLayoutFragment NSTextLineFragment
28. TextKit 2 จࣈྻͱϨΠΞ΢τ ೚ҙͷจࣈྻͷൣғ͔ΒɺߦΑΓࡉ͔͍ը໘্ͷۣܗΛٻΊ Δ͜ͱ͸Ͱ͖ͳ͍ ը໘্ͷ೚ҙͷҐஔ͔Βจࣈྻ্ͷൣғʹରԠ͢Δ͔Ͳ͏͔ ͱ͍ͬͨώοτςετ͸ಠࣗͷΞτϦϏϡʔτΛ࢖͏͜ͱͰ ͋Δఔ౓Մೳ
29. จࣈྻ͔ΒϨΠΞ΢τ
30. TextKit 2 จࣈྻ͔ΒϨΠΞ΢τ NSTextLayoutManager ͷ enumerateTextLayoutFragments(from:options:u sing) ͰจࣈྻͷҐஔ͔Β NSTextLayoutFragment Λ Ұཡ textLineFragments Ͱ NSTextLineFragment ΛҰཡ NSTextLineFragment ͷ typographicBounds Λ࢖͏
31. όά͕͋Γ·͢ typographicBounds typographicBounds ͸ layoutFragmentFrame ͷ origin ͔Βͷ࠲ඪͷ͸ͣ iOS 17 Ͱ͸ͦͷ௨Γʹͳ͍ͬͯΔ iOS 16 Ͱ͸ x ํ޲͚ͩ 0.0 ͔Βͷ࠲ඪʹͳ͍ͬͯΔ
32. ޗഐ͸ೣͰ͋Δɻ໊લ͸·ͩແ ͍ɻ 
 Ͳ͜ͰੜΕ͔ͨͱΜͱݟ౰͕ͭ ͔͵ɻԿͰ΋ബ҉͍͡Ί͡Ί͠ ͨॴͰχϟʔχϟʔٽ͍͍ͯͨ ࣄ͚ͩ͸هԱ͍ͯ͠Δɻ Text Container layoutFragmentFrame typographicBounds
33. if #available(iOS 16.0, *) { let textLayoutManager = textView.textLayoutManager! let textContentStorage = textLayoutManager.textContentManager textLayoutManager.enumerateTextLayoutFragments( 
 from: textContentStorage?.documentRange.location 
 ) { layoutFragment in for lineFragment in layoutFragment.textLineFragments { let bounds: CGRect if #available(iOS 17.0, *) { bounds = lineFragment.typographicBounds } else { bounds = lineFragment.typographicBounds.offsetBy( dx: -layoutFragment.layoutFragmentFrame.minX, dy: 0.0 ) } // Use `bounds`... } return true }
34. ϨΠΞ΢τ͔Βจࣈྻ
35. TextKit 2 ϨΠΞ΢τ͔Βจࣈྻ NSTextLayoutManager ͷ textLayoutFragment(for:) Λ࢖ͬͯ NSLayoutFragment ΛऔಘɺlineFragments Λ ࢖ͬͯͦΕͧΕͷ typographicBounds Λώοτςετ NSLineFragment ͷ characterIndex(for:) Λ࢖ͬͯจ ࣈྻ಺ͷҐஔΛऔಘ attributes(at:effectiveRange:)Ͱจࣈྻ্ͷӨڹൣ ғΛऔಘ
36. όά͕͋Γ·͢ typographicBounds typographicBounds ͸ layoutFragmentFrame ͷ origin ͔Βͷ࠲ඪͷ͸ͣ iOS 17 Ͱ͸ͦͷ௨Γʹͳ͍ͬͯΔ iOS 16 Ͱ͸ x ํ޲͚ͩ 0.0 ͔Βͷ࠲ඪʹͳ͍ͬͯΔ
37. let attributedText = NSMutableAttributedString( 
 "ޗഐ͸ೣͰ͋Δɻ໊લ͸·ͩແ͍" 
 ) let userNameAttributeKey = NSAttributedString.Key( 
 rawValue: "UserName" 
 ) 
 attributedText.addAttribute( 
 userNameAttributeKey, 
 value: "@neko", 
 range: NSRange(location: 0, length: 2) 
 ) 
 
 textView.attributedText = attributedText 
 
 ...
38. let pointInTextView = // ... let pointInTextContainer = CGPoint( x: pointInTextView.x - textView.textContainerInset.left, y: pointInTextView.y - textView.textContainerInset.top ) 
 if let layoutFragment = textView.textLayoutManager? 
 .textLayoutFragment( 
 for: pointInTextContainer 
 ) { let pointInLayoutFragment = CGPoint( x: pointInTextContainer.x - 
 layoutFragment.layoutFragmentFrame.minX, y: pointInTextContainer.y - 
 layoutFragment.layoutFragmentFrame.minY ) ...
39. ... for lineFragment in layoutFragment.textLineFragments { let typographicBounds: CGRect if #available(iOS 17.0, *) { typographicBounds = lineFragment.typographicBounds } else { typographicBounds = 
 lineFragment.typographicBounds.offsetBy( dx: -layoutFragment.layoutFragmentFrame.minX, dy: 0.0 ) } ...
40. ... 
 if typographicBounds.contains(pointInLayoutFragment) { let pointInLineFragment = CGPoint( x: pointInLayoutFragment.x - typographicBounds.minX, y: pointInLayoutFragment.y - typographicBounds.minY ) let characterIndex = lineFragment.characterIndex( 
 for: pointInLineFragment 
 ) var characterRange = 
 NSRange(location: NSNotFound, length: 0) let attributes = textView.attributedText.attributes( 
 at: characterIndex, 
 effectiveRange: &characterRange 
 ) if let userName = attributes[userNameAttributeKey] { // Use `userName` and `characterRange` ... } } } }
41. ͭΒΈ͕ଟ͍ͷͰ͕͢…
42. ศརʹͳͬͨػೳ
43. TextKit 2 ͷศརʹͳͬͨػೳ NSTextAttachmentViewProvider NSTextAttachmentViewProvider Λ࢖ͬͯ೚ҙͷϏϡʔΛ؆୯ ʹจதʹຒΊࠐΊΔΑ͏ʹͳͬͨ TextKit 1 Ͱ͸ը૾͚͚ͩͩͰɺ೚ҙͷϏϡʔ͸ؤுͬͯࣗ෼ ͰϨΠΞ΢τ͢Δඞཁ͕͋ͬͨ
45. final class PlayerView: UIView { // ... override init(frame: CGRect) { // ... let player = AVQueuePlayer(playerItem: playerItem) playerLayer = AVPlayerLayer(player: player) playerLayer.videoGravity = .resizeAspectFill super.init(frame: frame) layer.addSublayer(playerLayer) player.play() } 
 // ... }
46. final class AttachmentViewProvider: NSTextAttachmentViewProvider { override func loadView() { view = PlayerView() } override func attachmentBounds( for attributes: [NSAttributedString.Key : Any], location: NSTextLocation, textContainer: NSTextContainer?, proposedLineFragment: CGRect, position: CGPoint ) -> CGRect { CGRect( 
 x: 0.0, 
 y: 0.0, 
 width: proposedLineFragment.width, 
 height: 100.0 
 ) } }
47. final class Attachment: NSTextAttachment { override func viewProvider( for parentView: UIView?, location: NSTextLocation, textContainer: NSTextContainer? ) -> NSTextAttachmentViewProvider? { let viewProvider = AttachmentViewProvider( textAttachment: self, parentView: parentView, textLayoutManager: textContainer?.textLayoutManager, location: location ) viewProvider.tracksTextAttachmentViewBounds = true return viewProvider } }
48. let attributedText = NSMutableAttributedString() 
 attributedText.append(NSAttributedString( 
 string: "ޗഐ͸ೣͰ͋Δɻ", 
 attributes: attributes 
 )) attributedText.append(NSAttributedString( 
 attachment: Attachment() 
 )) attributedText.append(NSAttributedString( 
 string: "໊લ͸·ͩແ͍", attributes: attributes 
 ))
49. ஫ҙ఺ NSTextAttachmentViewProvider NSAttributedString ʹΑͬͯɺຒΊࠐΉϏϡʔΛؚΊ ͯϦιʔε͕ॴ༗͞ΕΔ ϝϞϦϦʔΫ͕ى͜Βͳ͍Α͏ʹɺ॥؀ࢀরʹ஫ҙ
50. ޓ׵ੑ
51. ରԠ͢Δ Ϗϡʔ iOS ͷόʔδϣϯ iOS 14 iOS 15 iOS 16 Ҏ߱ UITextField 􀁣 􀁣 UITextView 􀁣
52. UITextView ࣗಈϑΥʔϧόοΫ iOS 15 ͸ TextKit 1 ͚ͩ iOS 16 Ҏ߱͸ TextKit 2 ͕σϑΥϧτ ͔͠͠ɺTextKit 1 ͷ API ʹͪΐͬͱͰ΋৮ΕΔͱ TextKit 1 ʹ੾ΓସΘͬͯ͠·͏ Ұ౓ TextKit 1 ʹͳΔͱɺTextKit 2 ʹ͸໭Βͳ͍
53. TextKit 1 ͚ͩΛ࢖͏৔߹ TextKit Λ࢖͏ ॳظԽ࣌ʹ TextKit 1 Λ࢖͏Α͏ʹ͢Δ ࣮ߦ࣌ʹίετͷ͔͔ΔࣗಈϑΥʔϧόοΫΛىͤ͜͞ͳ͍
54. TextKit 2 Λ࢖͏৔߹ TextKit Λ࢖͏ TextKit 2 ͚ͩΛ࢖͏৔߹ UITextView Λ೿ੜͤͯ͞ɺTextKit 1 ͷ API Λ௵͢ 
 _UITextViewEnablingCompatibilityMode Ͱ֬ೝ ࣮ߦ࣌ʹ൑அ͢Δ৔߹ #available(iOS 16.0, *) Ͱ iOS 16 Λ࣮ߦ࣌ʹࣝผ
55. // TextKit 1 ͚ͩΛ࢖͏৔߹ let textView = UITextView(usingTextLayoutManager: false) // TextKit 2 ͚ͩΛ࢖͏৔߹ final class TextView: UITextView { override var layoutManager: NSLayoutManager { fatalError("ͩΊͰ͢") } }
56. // ৚݅ʹΑͬͯ੾Γସ͑Δ৔߹ let textView: UITextView if #available(iOS 16.0, *) { textView = UITextView(usingTextLayoutManager: true) 
 } else { 
 textView = UITextView() } 
 final class TextView: UITextView { override var layoutManager: NSLayoutManager { if #available(iOS 16.0, *) { fatalError("ͩΊͰ͢") } else { super.layoutManager } } }
57. ·ͱΊ TextKit 2 UITextView Ͱ iOS 16 Ҏ߱Ͱ TextKit 2 ͕࢖͑ΔΑ͏ʹͳͬͨ ࠷খ୯Ґ͕ߦͳͷͰɺͦΕҎԼͷૢ࡞͕ඞཁͳ৔߹͸޻෉͢ Δඞཁ͕͋Δ Ґஔܭࢉ͕ iOS 16 Ͱ͸όάͬͯΔ TextKit 1 ʹࣗಈϑΥʔϧόοΫ͢ΔͷͰڍಈΛ؅ཧ͢Δ
58. SwiftUI
59. Text, TextEditor SwiftUI ඇৗʹ൥ࡶͳ AttributedString ͕ Text ϏϡʔͰ࢖͑Δ͕ػ ೳ͕ݶΒΕΔ NSTextAttachment ͸࢖͑ͳ͍ TextKit ͷ API ΁ͷΞΫηε͸ͳ͍
60. ·ͱΊ SwiftUI TextKit ʹ͸ରԠͯ͠ͳ͍ Text Ϗϡʔ΍ TextEditor Ϗϡʔ͸ػೳ͕͔ͳΓݶΒΕΔ TextKit ͷػೳ͕ඞཁͳ৔߹͸ૉ௚ʹ UITextView ΛຒΊࠐΉ
61. ͱ͍͏͜ͱͰ…
62. ·ͱΊ TextKit 1 ͸จࣈྻ͔ΒάϦϑ୯ҐͰૢ࡞Մೳɻͨͩ͠ਖ਼͘͠ ಈ࡞͠ͳ͍έʔε͕͋Δ TextKit 2 ͸ศརͳػೳ͕௥Ճ͞Ε͍ͯΔ͕ɺจࣈྻ͔Βߦ୯ ҐͰͷૢ࡞ʹݶΒΕΔɻtypographicBounds ͸όάͬͯΔ SwiftUI ͷText, TextEditor Ϗϡʔ͸ػೳ͕ݶΒΕΔ
63. ·ͱΊ ͲΕ΋׬ᘳͰ͸ͳ͍ ΞϓϦέʔγϣϯͷ໨త΍ඞཁͳػೳʹԠͯ͡ɺTextKit 1, TextKit 2, SwiftUI ΛબͿ