Share
Embed
Learning to (Learn at Test Time): RNNs with Expressive Hidden States
Hiroto Kurita
August 22, 2025
Transcript
31 slides
-
ಡΈख: ܀ా ਓ (౦େֶ) @ ୈ17ճ࠷ઌNLPษڧձ Learning to (Learn at
Test Time): RNNs with Expressive Hidden States Yu Sun, Xinhao Li, Karan Dalal, Jiarui Xu, Arjun Vikram, Genghan Zhang, Yann Dubois, Xinlei Chen, Xiaolong Wang, Sanmi Koyejo, Tatsunori Hashimoto, Carlos Guestrin ICML 2025 (Spotlight poster) https://arxiv.org/abs/2407.04620 ※ ಛʹஅΓ͕ͳ͍߹ɺਤදσʔλհจ͔ΒҾ༻͍ͯ͠·͢ -
֓ཁ • ϞσϧΞʔΩςΫνϟఏҊܥจɽRNN ͷվળ͍͖ͬͯ • Test Time Training (TTT)ʹΑΓɼೖྗτʔΫϯͨͪΛRNNͷӅΕঢ়ଶʹ”͏·͘” ѹॖ͢Δํ๏Λߟ͑Δ
• બఆཧ༝ • Modern RNN / SSM (hybrid ΞʔΩςΫνϟؚΉ) ͷ࣮͕ਐΜͰ͍Δɽࠃͩ ͱ PFNࣾͷ PLaMo 2ɽCartesia ͷ TTS Ϟσϧ SSM ͱ͍͏ᷚɽ • TTT Λ͏ͱ͍͏ɼࠓ·Ͱͷ SSM ͱগ͠ҧͬͨΞϓϩʔνΛऔΔ 2 -
෮शɿRNN લͷঢ়ଶͱݱࡏͷೖྗΛͬͯঢ়ଶΛߋ৽ 3 ঢ়ଶ ೖྗ ग़ྗ s0 s1 st−1 st
x1 xt−1 xt z1 zt−1 zt … = θzs st +θzx xt = σ(θss st−1 +θsx xt ) 🟦 : ޡࠩٯ๏Ͱߋ৽͞ΕΔϞσϧͷॏΈ -
෮शɿRNN લͷঢ়ଶͱݱࡏͷೖྗΛͬͯঢ়ଶΛߋ৽ 4 ঢ়ଶ ೖྗ ग़ྗ s0 s1 st−1 st
x1 xt−1 xt z1 zt−1 zt … = θzs st +θzx xt = σ(θss st−1 +θsx xt ) 🟦 : ޡࠩٯ๏Ͱߋ৽͞ΕΔϞσϧͷॏΈ ܭࢉɿ ϝϞϦɿ O(1) O(1) 🥰 -
෮शɿRNN લͷঢ়ଶͱݱࡏͷೖྗΛͬͯঢ়ଶΛߋ৽ 5 ঢ়ଶ ೖྗ ग़ྗ s0 s1 st−1 st
x1 xt−1 xt z1 zt−1 zt … = θzs st +θzx xt = σ(θss st−1 +θsx xt ) 🟦 : ޡࠩٯ๏Ͱߋ৽͞ΕΔϞσϧͷॏΈ ܭࢉɿ ϝϞϦɿ O(1) O(1) Self-attention: ܭࢉɿ ϝϞϦɿ O(t) O(t) 🥰 😖 -
😢 (Modern) RNN ܥྻʹऑ͍ 7 🟥 Mamba: ࠷ॳ͍͍ײ͡ 🟥 Mamba:
ޙͰανΔ RNN ܥྻͰͦ͜ਅՁ ͕ൃش͞ΕΔͷʹ…. -
😢 (Modern) RNN ܥྻʹऑ͍ 8 🟥 Mamba: ࠷ॳ͍͍ײ͡ 🟥 Mamba:
ޙͰανΔ RNN ܥྻͰͦ͜ਅՁ ͕ൃش͞ΕΔͷʹ…. 🟦🍊: ఏҊख๏ 🤔 աڈจ຺ͷେྔͷτʔΫϯɼ ͲͷΑ͏ʹѹॖ͢Εʁ -
💡ࣗݾڭࢣ͋Γֶश ≈ ֶशσʔλͷѹॖ 9 ֶशσʔληοτ Ϟσϧ (ॏΈ) (ࣗݾڭࢣ͋Γ)ֶश E.g., ࣍୯ޠ༧ଌ
E.g., Wikipedia ≈ѹॖ൛ֶशσʔλ Q: ຊͷट? A: ౦ژ -
💡ࣗݾڭࢣ͋Γֶश ≈ ֶशσʔλͷѹॖ 10 ֶशσʔληοτ Ϟσϧ (ॏΈ) (ࣗݾڭࢣ͋Γ)ֶश E.g., ࣍୯ޠ༧ଌ
E.g., Wikipedia x1 x2 x3 … ίϯςΩετͷ ֤τʔΫϯ ֶशσʔληοτ Ϟσϧ (ॏΈ) (ࣗݾڭࢣ͋Γ)ֶश ԿΒ͔ͷλεΫ ≈ѹॖ൛ֶशσʔλ Q: ຊͷट? A: ౦ژ 💡 จ຺ͷ֤τʔΫϯͷ ྑ͍ѹॖʹͳΔͣ 💡 RNN ͷঢ়ଶΛϞσϧʢͷॏ Έʣͱݟͯɼจ຺ͷτʔΫϯ Λֶश͞Εྑ͍ͷͰʁ -
ఏҊख๏ɿঢ়ଶ = খ͞ͳػցֶशϞσϧͱݟͯɼޯ߱Լ๏ͰॏΈߋ৽ 11 ঢ়ଶ ೖྗ ग़ྗ W0 W1 Wt−1
Wt x1 xt−1 xt z1 zt−1 zt … = Wt−1 −η∇l(xt ; Wt−1 ) = f(xt ; Wt ) 🟦 : ࣮ࡍͷޡࠩٯ๏Ͱߋ৽͞ΕΔॏΈɽ֎ଆϧʔϓ 🟥 : લ͖ܭࢉதͷޯ߱Լ๏ʹΑΓߋ৽͞ΕΔ ͷॏΈɽଆϧʔϓɽ f : ΛॏΈͱ͢Δখ͞ͳϞσϧɽE.g., ઢܗɼMLPɼ… f( ⋅ ; W⋅ ) W⋅ 1εςοϓͷঢ়ଶߋ৽ = 1εςοϓͷޯ߱Լ -
ఏҊख๏ɿঢ়ଶ = খ͞ͳػցֶशϞσϧͱݟͯɼޯ߱Լ๏ͰॏΈߋ৽ 12 ঢ়ଶ ೖྗ ग़ྗ W0 W1 Wt−1
Wt x1 xt−1 xt z1 zt−1 zt … = Wt−1 −η∇l(xt ; Wt−1 ) = f(xt ; Wt ) 🟦 : ࣮ࡍͷޡࠩٯ๏Ͱߋ৽͞ΕΔॏΈɽ֎ଆϧʔϓ 🟥 : લ͖ܭࢉதͷޯ߱Լ๏ʹΑΓߋ৽͞ΕΔ ͷॏΈɽଆϧʔϓɽ f : ΛॏΈͱ͢Δখ͞ͳϞσϧɽE.g., ઢܗɼMLPɼ… f( ⋅ ; W⋅ ) W⋅ 1εςοϓͷঢ়ଶߋ৽ = 1εςοϓͷޯ߱Լ લϖʔδͷʮֶश=ѹॖʯ ͷؾ࣋ͪ -
ఏҊख๏ɿঢ়ଶ = খ͞ͳػցֶशϞσϧͱݟͯɼޯ߱Լ๏ͰॏΈߋ৽ 13 ঢ়ଶ ೖྗ ग़ྗ W0 W1 Wt−1
Wt x1 xt−1 xt z1 zt−1 zt … = Wt−1 −η∇l(xt ; Wt−1 ) = f(xt ; Wt ) 🟦 : ࣮ࡍͷޡࠩٯ๏Ͱߋ৽͞ΕΔॏΈɽ֎ଆϧʔϓ 🟥 : લ͖ܭࢉதͷޯ߱Լ๏ʹΑΓߋ৽͞ΕΔ ͷॏΈɽଆϧʔϓɽ f : ΛॏΈͱ͢Δখ͞ͳϞσϧɽE.g., ઢܗɼMLPɼ… f( ⋅ ; W⋅ ) W⋅ 1εςοϓͷঢ়ଶߋ৽ = 1εςοϓͷޯ߱Լ 🟥 લ͖ܭࢉͰޯ߱Լ ͍ͤͯ͞Δ͜ͱʹҙ લϖʔδͷʮֶश=ѹॖʯ ͷؾ࣋ͪ -
۩ମతͳࣗݾڭࢣ͋ΓλεΫ ͷઃܭ l 14 Wt−1 Wt xt zt = Wt−1
−η∇l(xt ; Wt−1 ) = f(xt ; Wt ) ঢ়ଶ ೖྗ ग़ྗ 🟦 : ࣮ࡍͷޡࠩٯ๏Ͱߋ৽͞ΕΔॏΈɽ֎ଆϧʔϓ 🟥 : લ͖ܭࢉதͷޯ߱Լ๏ʹΑΓߋ৽͞ΕΔ ͷॏΈɽଆϧʔϓɽ f : ΛॏΈͱ͢Δখ͞ͳϞσϧɽE.g., ઢܗɼMLPɼ… f( ⋅ ; W⋅ ) W⋅ -
۩ମతͳࣗݾڭࢣ͋ΓλεΫ ͷઃܭ l 15 Wt−1 Wt xt zt = Wt−1
−η∇l(xt ; Wt−1 ) = f(xt ; Wt ) ঢ়ଶ ೖྗ ग़ྗ l(xt ; Wt−1 ) = ∥f(˜ xt ; Wt−1 ) − xt ∥2 🟦 : ࣮ࡍͷޡࠩٯ๏Ͱߋ৽͞ΕΔॏΈɽ֎ଆϧʔϓ 🟥 : લ͖ܭࢉதͷޯ߱Լ๏ʹΑΓߋ৽͞ΕΔ ͷॏΈɽଆϧʔϓɽ f : ΛॏΈͱ͢Δখ͞ͳϞσϧɽE.g., ઢܗɼMLPɼ… f( ⋅ ; W⋅ ) W⋅ ೖྗ Λ࠶ߏ͢ΔλεΫ x -
۩ମతͳࣗݾڭࢣ͋ΓλεΫ ͷઃܭ l 16 Wt−1 Wt xt zt = Wt−1
−η∇l(xt ; Wt−1 ) = f(xt ; Wt ) ঢ়ଶ ೖྗ ग़ྗ l(xt ; Wt−1 ) = ∥f(θK xt ; Wt−1 ) − xt ∥2 ೖྗ Λ࠶ߏ͢ΔλεΫ x 🟦 : ࣮ࡍͷޡࠩٯ๏Ͱߋ৽͞ΕΔॏΈɽ֎ଆϧʔϓ 🟥 : લ͖ܭࢉதͷޯ߱Լ๏ʹΑΓߋ৽͞ΕΔ ͷॏΈɽଆϧʔϓɽ f : ΛॏΈͱ͢Δখ͞ͳϞσϧɽE.g., ઢܗɼMLPɼ… f( ⋅ ; W⋅ ) W⋅ ࣍ݩʹࣹӨͯ͠յ͢ -
۩ମతͳࣗݾڭࢣ͋ΓλεΫ ͷઃܭ l 17 Wt−1 Wt xt zt = Wt−1
−η∇l(xt ; Wt−1 ) = f(xt ; Wt ) ঢ়ଶ ೖྗ ग़ྗ l(xt ; Wt−1 ) = ∥f(θK xt ; Wt−1 ) − xt ∥2 ೖྗ Λ࠶ߏ͢ΔλεΫ x 🟦 : ࣮ࡍͷޡࠩٯ๏Ͱߋ৽͞ΕΔॏΈɽ֎ଆϧʔϓ 🟥 : લ͖ܭࢉதͷޯ߱Լ๏ʹΑΓߋ৽͞ΕΔ ͷॏΈɽଆϧʔϓɽ f : ΛॏΈͱ͢Δখ͞ͳϞσϧɽE.g., ઢܗɼMLPɼ… f( ⋅ ; W⋅ ) W⋅ ࣍ݩʹࣹӨͯ͠յ͢ ೖྗͷͲͷಛ͕େࣄ͔ -
۩ମతͳࣗݾڭࢣ͋ΓλεΫ ͷઃܭ l 18 Wt−1 Wt xt zt = Wt−1
−η∇l(xt ; Wt−1 ) = f(xt ; Wt ) ঢ়ଶ ೖྗ ग़ྗ l(xt ; Wt−1 ) = ∥f(θK xt ; Wt−1 )−θV xt ∥2 ೖྗ Λ࠶ߏ͢ΔλεΫ x 🟦 : ࣮ࡍͷޡࠩٯ๏Ͱߋ৽͞ΕΔॏΈɽ֎ଆϧʔϓ 🟥 : લ͖ܭࢉதͷޯ߱Լ๏ʹΑΓߋ৽͞ΕΔ ͷॏΈɽଆϧʔϓɽ f : ΛॏΈͱ͢Δখ͞ͳϞσϧɽE.g., ઢܗɼMLPɼ… f( ⋅ ; W⋅ ) W⋅ ࣍ݩʹࣹӨͯ͠յ͢ ೖྗͷͲͷಛ͕େࣄ͔ ͲͷΑ͏ͳϥϕϧΛ࡞Εྑ͍͔ -
۩ମతͳࣗݾڭࢣ͋ΓλεΫ ͷઃܭ l 19 Wt−1 Wt xt zt = Wt−1
−η∇l(xt ; Wt−1 ) ঢ়ଶ ೖྗ ग़ྗ l(xt ; Wt−1 ) = ∥f(θK xt ; Wt−1 )−θV xt ∥2 ೖྗ Λ࠶ߏ͢ΔλεΫ x 🟦 : ࣮ࡍͷޡࠩٯ๏Ͱߋ৽͞ΕΔॏΈɽ֎ଆϧʔϓ 🟥 : લ͖ܭࢉதͷޯ߱Լ๏ʹΑΓߋ৽͞ΕΔ ͷॏΈɽଆϧʔϓɽ f : ΛॏΈͱ͢Δখ͞ͳϞσϧɽE.g., ઢܗɼMLPɼ… f( ⋅ ; W⋅ ) W⋅ ࣍ݩʹࣹӨͯ͠յ͢ ೖྗͷͲͷಛ͕େࣄ͔ ͲͷΑ͏ͳϥϕϧΛ࡞Εྑ͍͔ ࣍୯ޠ༧ଌʹޮ͘ ͷಛΛநग़ xt = f(θQ xt ; Wt ) -
۩ମతͳࣗݾڭࢣ͋ΓλεΫ ͷઃܭ l 20 Wt−1 Wt xt zt = Wt−1
−ηθlr (xt )∇l(xt ; Wt−1 ) ঢ়ଶ ೖྗ ग़ྗ l(xt ; Wt−1 ) = ∥f(θK xt ; Wt−1 )−θV xt ∥2 ೖྗ Λ࠶ߏ͢ΔλεΫ x 🟦 : ࣮ࡍͷޡࠩٯ๏Ͱߋ৽͞ΕΔॏΈɽ֎ଆϧʔϓ 🟥 : લ͖ܭࢉதͷޯ߱Լ๏ʹΑΓߋ৽͞ΕΔ ͷॏΈɽଆϧʔϓɽ f : ΛॏΈͱ͢Δখ͞ͳϞσϧɽE.g., ઢܗɼMLPɼ… f( ⋅ ; W⋅ ) W⋅ ࣍ݩʹࣹӨͯ͠յ͢ ೖྗͷͲͷಛ͕େࣄ͔ ͲͷΑ͏ͳϥϕϧΛ࡞Εྑ͍͔ ࣍୯ޠ༧ଌʹޮ͘ ͷಛΛநग़ xt = f(θQ xt ; Wt ) ೖྗʹԠֶͯ͡शΛௐ -
۩ମతͳࣗݾڭࢣ͋ΓλεΫ ͷઃܭ l 21 Wt−1 Wt xt zt = Wt−1
−ηθlr (xt )∇l(xt ; Wt−1 ) ঢ়ଶ ೖྗ ग़ྗ l(xt ; Wt−1 ) = ∥f(θK xt ; Wt−1 )−θV xt ∥2 ೖྗ Λ࠶ߏ͢ΔλεΫ x 🟦 : ࣮ࡍͷޡࠩٯ๏Ͱߋ৽͞ΕΔॏΈɽ֎ଆϧʔϓ 🟥 : લ͖ܭࢉதͷޯ߱Լ๏ʹΑΓߋ৽͞ΕΔ ͷॏΈɽଆϧʔϓɽ f : ΛॏΈͱ͢Δখ͞ͳϞσϧɽE.g., ઢܗɼMLPɼ… f( ⋅ ; W⋅ ) W⋅ ࣍ݩʹࣹӨͯ͠յ͢ ೖྗͷͲͷಛ͕େࣄ͔ ͲͷΑ͏ͳϥϕϧΛ࡞Εྑ͍͔ ࣍୯ޠ༧ଌʹޮ͘ ͷಛΛநग़ xt 🟦: ͲͷΑ͏ͳࣗݾڭࢣ͋Γֶश(=ίϯ ςΩετѹॖ)Λલ͖ܭࢉͰߦ͑ɼ ࣍୯ޠ༧ଌʹ༗ޮ͔? ΛֶͿɽ (λεΫࣗମͷબ, ֶͼํΛֶशʣ = f(θQ xt ; Wt ) ೖྗʹԠֶͯ͡शΛௐ -
۩ମతͳࣗݾڭࢣ͋ΓλεΫ ͷઃܭ l 22 Wt−1 Wt xt zt = Wt−1
−ηθlr (xt )∇l(xt ; Wt−1 ) ঢ়ଶ ೖྗ ग़ྗ l(xt ; Wt−1 ) = ∥f(θK xt ; Wt−1 )−θV xt ∥2 ೖྗ Λ࠶ߏ͢ΔλεΫ x 🟦 : ࣮ࡍͷޡࠩٯ๏Ͱߋ৽͞ΕΔॏΈɽ֎ଆϧʔϓ 🟥 : લ͖ܭࢉதͷޯ߱Լ๏ʹΑΓߋ৽͞ΕΔ ͷॏΈɽଆϧʔϓɽ f : ΛॏΈͱ͢Δখ͞ͳϞσϧɽE.g., ઢܗɼMLPɼ… f( ⋅ ; W⋅ ) W⋅ ࣍ݩʹࣹӨͯ͠յ͢ ೖྗͷͲͷಛ͕େࣄ͔ ͲͷΑ͏ͳϥϕϧΛ࡞Εྑ͍͔ ࣍୯ޠ༧ଌʹޮ͘ ͷಛΛநग़ xt 🟦: ͲͷΑ͏ͳࣗݾڭࢣ͋Γֶश(=ίϯ ςΩετѹॖ)Λલ͖ܭࢉͰߦ͑ɼ ࣍୯ޠ༧ଌʹ༗ޮ͔? ΛֶͿɽ (λεΫࣗମͷબ, ֶͼํΛֶशʣ = f(θQ xt ; Wt ) ೖྗʹԠֶͯ͡शΛௐ -
Ϟσϧͷશମ૾ 24 TTT layer LayerNorm / Conv TTT layer Conv
LayerNorm Gate Transformer backbone Mamba backbone ock, the basic building block for Transformers. The sequence modeling block nts: the Transformer backbone and Mamba backbone. Middle: TTT layer The LN before O comes from NormFormer [60]. Right: TTT layer in the [25] and Griffin [18]. Following these two architectures, ω here is GELU [29]. rameters of the gate without changing the embedding dimension, we simply TTT layer LayerNorm / Conv TTT layer Conv LayerNorm Gate Transformer backbone Mamba backbone the basic building block for Transformers. The sequence modeling block the Transformer backbone and Mamba backbone. Middle: TTT layer he LN before O comes from NormFormer [60]. Right: TTT layer in the ] and Griffin [18]. Following these two architectures, ω here is GELU [29]. eters of the gate without changing the embedding dimension, we simply projection. : ΛॏΈͱ͢Δখ͞ͳϞσϧɽE.g., ઢܗɼMLPɼ… f( ⋅ ; W⋅ ) W⋅ Test Time Training (TTT) Layer -
࣮ݧɿܥྻͰͷεέʔϦϯά 25 Figure 10. Evaluations for context lengths 2k and
8k on the Pile. Details in Subsection 3.1. TTT-Linear has comparable performance as Mamba at 2k context, and better performance at 8k. (M): Mamba, (T): Transformer backbone Linear/MLP: ͷϞσϧ f -
࣮ݧɿܥྻͰͷεέʔϦϯά 26 Figure 10. Evaluations for context lengths 2k and
8k on the Pile. Details in Subsection 3.1. TTT-Linear has comparable performance as Mamba at 2k context, and better performance at 8k. (M): Mamba, (T): Transformer backbone Linear/MLP: ͷϞσϧ f TTT-Linear/MLP (T) ͕গ͠ѱ͍ ଞಉ -
࣮ݧɿܥྻͰͷεέʔϦϯά 27 Figure 10. Evaluations for context lengths 2k and
8k on the Pile. Details in Subsection 3.1. TTT-Linear has comparable performance as Mamba at 2k context, and better performance at 8k. (M): Mamba, (T): Transformer backbone Linear/MLP: ͷϞσϧ f TTT-Linear/MLP (T) ͕গ͠ѱ͍ ଞಉ TTT-Linear/MLP (M) ͕Ϥγ. ܥྻͩͱྑ ͘ͳΓ͕ͪ TTT-Linear/MLP (T) ͦ͜·Ͱ Mamba ʹಧ͔ͣ -
࣮ݧɿܥྻͰͷεέʔϦϯά 28 on Books. Details in Subsection 3.2. Our complete
results g Transformer finetuning, are in Figure 15 (in Appendix). TTT-MLP (T) ͕͍͍ײ͡ɽܥྻͰ Transformer backbone ͷར͋Δ͔? 😅 ͜ͷลΓͩͱ΄΅ಉ͡… -
࣮ݧɿܥྻͰͷεέʔϦϯά 29 on Books. Details in Subsection 3.2. Our complete
results g Transformer finetuning, are in Figure 15 (in Appendix). TTT-MLP (T) ͕͍͍ײ͡ɽܥྻͰ Transformer backbone ͷར͋Δ͔? 😅 ͜ͷลΓͩͱ΄΅ಉ͡… Sequence modeling block MLP block LayerNorm LayerNorm TTT layer LayerNorm / Conv TTT layer Conv LayerNorm Gate Residual block Transformer backbone Mamba backbone Figure 13. Left: A residual block, the basic building block for Transformers. The sequence modeling block is instantiated into two variants: the Transformer backbone and Mamba backbone. Middle: TTT layer in the Transformer backbone. The LN before O comes from NormFormer [60]. Right: TTT layer in the backbone inspired by Mamba [25] and Griffin [18]. Following these two architectures, ω here is GELU [29]. To accommodate the extra parameters of the gate without changing the embedding dimension, we simply combine εK and εQ into a single projection. ৗʹ TTT-MLP (T) > TTT-Linear (T) Ͱ TTT-MLP (M) ≈ TTT-Linear (M) ͳͷɼ Mamba ͷ 1D conv ͕͍͍ࣄ͍ͯ͠Δ͔. N-gram తͳಛΛݟΔׂ -
࣮ݧɿprefill / decode 30 Figure 12. Latency on an
NVIDIA A100 GPU with 80G HBM and PCIe connections. pre fi ll / decode ͱʹܥྻ͕৳ͼͯ const. -
ײ • աڈจ຺Λ͍͔ʹѹॖ͢Δ͔ͱ͍͏ࠓ·Ͱͷ Modern RNN / SSM Ͱ ۷ΒΕ͍͕ͯͨɼTest Time
Training ͱ݁ͼ͚Δͷ৽͘͠ײͨ͡ • Modern RNN / SSM ͩͱɼRNN ͷঢ়ଶભҠߦྻͷߏࣗମʹண͢Δ ݚڀ͕ଟ͔ͬͨ? S4, Mamba ͳͲ • RNN ʹݶΒͣɼTTT Test time compute scaling ͱͷ૬ੑྑͦ͞͏ɽ • ARC Challenge Ͱ TTT Λ͏ͱਖ਼͕άϯͱ্͕Δͱ͍͏ݚڀ͕ ICML 2025 ʹ࠾ [Akyürek+’25] • (ֶश࣌ʹ) Կʹ GPU Λແବͳ͑͘Δ͔͕՝͔ 31