Implementation and Application of High-Performance Empirical Dynamic Modeling

౦๺େֶαΠόʔαΠΤϯεηϯλʔߴڮܛஐ Gerald M. Pao, Salk Institute for Biological Studies Implementation
and Application of High- Performance Empirical Dynamic Modeling ֶࡍେن໛৘ใج൫ڞಉར༻ɾڞಉݚڀڌ఺ ୈճγϯϙδ΢Ϝ ೥݄೔ jh220050ࠃࡍڞಉݚڀ՝୊

&NQJSJDBM%ZOBNJD.PEFMJOH &%. w ඇઢܗ࣌ܥྻσʔλͷ෼ੳɾ༧ଌͷͨΊͷ৽͍͠ख๏܈ w ϞσϧϑϦʔͳख๏Ͱ͋Γɼ؍ଌσʔλͷΈʹΑΓσʔλۦಈతʹ෼ੳ w 5BLFOTͷຒΊࠐΈఆཧ< >ʹج͖ͮ஗Ԇ࠲ඪܥΛ༻͍ͯྗֶܥΛ෮ݩ 2
ݩͷྗֶܥ ࠶ߏ੒͞Εͨྗֶܥ ؍ଌ࣌ܥྻσʔλ ඍ෼ಉ૬ ˺ہॴతͳۙ๣ؔ܎͕ҡ࣋ [1] Floris Takens, “Detecting strange attractors in turbulence,” LectureNotes in Mathematics, vol. 898, 1981, pp. 366-381.   [2] Ethan Deyle and George Sugihara, “Generalized theorems for nonlinear state space reconstruction,” PLoS One, vol. 6, no.3, 2011.

&%.ͷద༻ྫҨ఻ࢠൃݱྔͷ࣌ؒมԽ 3 300 250 200 150 100 50 0 180
160 140 120 100 80 60 90 80 70 60 50 40 30 20 10 0 YHP1 YHP1 CLN3 YOX1 0 10 20 30 40 50 60 70 80 90 :)1 :09 $-/   Ҩ఻ࢠͷൃݱྔ   ૬ؔ͸ඇৗʹऑ͍ $-/ͱ:09ͷ   ຒΊࠐΈ $-/ͱ:)1ͷ   ຒΊࠐΈ :)1 :09 $-/ͷ   ຒΊࠐΈ Ҩ఻ࢠൃݱྔͷଞʹ΋ɼੜଶܥͷಈଶɼਆܦ׆ಈɼಓ࿏ަ௨໢ͷަ௨ྔɼͳͲɼ   ඇઢܗྗֶܥͱͯ͠ϞσϧԽͰ͖ΔγεςϜͷଟ͘Ͱ༗ޮͰ͋Δ͜ͱ͕ࣔ͞Ε͍ͯΔɽ

4JNQMFY1SPKFDUJPOʹΑΔ୹ظ༧ଌ 4 ࣌ܥྻΛ࣌ؒ஗Ε࠲ඪܥΛ༻͍ͯঢ়ଶۭؒʹຒΊࠐΉ ݱࡏͷঢ়ଶͷۙ๣఺͕ɼ࣍ͷ࣌ؒεςοϓͰͲͷΑ͏ʹಈ͍͔ͨௐ΂Δ   ঢ়ଶ͕ྨࣅ͍ͯ͠Δ఺͸ɼ࣍ͷ࣌ؒεςοϓͰ΋ঢ়ଶ͕ྨࣅ͍ͯ͠Δͱߟ͑ΒΕΔ
࣍ͷ࣌ؒεςοϓʹ͓͚Δ֤ۙ๣఺ͷঢ়ଶͷॏΈ෇͖ฏۉ͔Βɼݱࡏͷঢ়ଶͷ   ࣍ͷ࣌ؒεςοϓʹ͓͚Δঢ়ଶΛ༧ଌ͢Δ ࣍ͷ   ࣌ؒεςοϓ

4JNQMFY1SPKFDUJPOʹΑΔ୹ظ༧ଌ 5 ؍ଌ࣌ܥྻͷ࣌ؒ஗ΕΛ ࣍ݩͷঢ়ଶۭؒʹຒΊࠐΈ ঢ়ଶۭؒʹ͓͚Δ ͷ
ݸͷۙ๣఺Λ୳ࡧ ۙ๣఺ͷ εςοϓޙͷࢦ਺ॏΈ෇͖ฏۉΛܭࢉ ͍ۙۙ๣఺͸ॏΈ͕େ͖͍ E X(t) x(t) = (X(t), X(t − τ), …, X(t − (E − 1)τ)) x(t) E + 1 x(t1 ), x(t2 ), …, x(tE+1 ) Tp ̂ x(t + Tp ) = E+1 ∑ i=1 wi ∑E+1 j=1 wj ⋅ x(ti + Tp ) wi = exp { − ∥x(t) − x(ti )∥ min ∥x(t) − x(ti )∥ } εςοϓޙ Tp x(t1 + Tp ) x(t2 ) x(t1 ) x(t2 + Tp ) x(t3 ) x(t3 + Tp )

࣌ܥྻ ͱ ΛͦΕͧΕঢ়ଶۭؒʹຒΊࠐΉ ঢ়ଶۭؒʹ͓͚Δ ͷۙ๣఺Λ୳ࡧ ͷۙ๣఺ͷ৘ใΛ༻͍ɼ
ͷະདྷΛ༧ଌ       ͕ Λߴ͍ਫ਼౓Ͱ༧ଌͰ͖ΔͳΒɼ ͸l$$.DBVTFTz X(t) Y(t) x(t) x(t1 ), x(t2 ), …, x(tE+1 ) x(t) y(t) X(t) Y(t) Y(t) X(t) $POWFSHFOU$SPTT.BQQJOH $$. ʹΑΔҼՌ෼ੳ 6 ̂ y(t + Tp ) = E+1 ∑ i=1 wi ∑E+1 i=1 wi ⋅ y(ti + Tp ) wi = exp { − ∥x(t) − x(ti )∥ min ∥x(t) − x(ti )∥ } ˞࣮ࡍ͸࠷దͳຒΊࠐΈ࣍ݩɾ࣌ؒ஗Εͷܾఆɼσʔλ఺ͷ૿ՃʹΑͬͯ   ༧ଌਫ਼౓͕޲্͢Δ͔ͷऩଋੑ൑ఆͳͲ͕͞Βʹඞཁ

λʔήοτΞϓϦέʔγϣϯ w $$.ҼՌ෼ੳʹΑΓɼΧϧγ΢ϜΠϝʔδϯάʹΑΓܭଌͨ͠શ೴ͷ   ਆܦࡉ๔ؒͷҼՌؔ܎ωοτϫʔΫΛ෼ੳ w ສ࣌ؒεςοϓͷສݸͷ࣌ܥྻؒͷશମશͰ$$.ҼՌ෼ੳΛߦ͏ͱɼ   ԯ࣌ܥྻରͷ$$.ܭࢉ͕ඞཁ 7
θϒϥϑΟογϡ༮ੜ ޫγʔτܬޫݦඍڸ ਆܦ׆ಈ ਆܦࡉ๔ؒͷҼՌؔ܎

Dataset # of time series # of time steps cppEDM
  512 nodes mpEDM   1 node mpEDM   512 nodes Fish1_Normo 1,450 53,053 8.5h 1,973s 20s Subject6 3,780 92,538 176h* 13,953s 101s Subject11 8,528 101,729 1,081h* 39,572s 199s ։ൃͨ͠ฒྻ෼ࢄ$$.ͷੑೳ 8 1,530x ߴ଎Խ 7,941x ࢿݯ࡟ݮ   (8,704 USD→1.1 USD) 'JTI@/PSNPσʔληοτͷॲཧ࣌ؒΑΓ֎ૠ [1] Wassapon Watanakeesuntorn et al., “Massively Parallel Causal Inference of Whole Brain Dynamics at Single Neuron Resolution,”   26th International Conference on Parallel and Distributed Systems (ICPADS 2020), Dec. 2020. ۙ๣୳ࡧΛ(16্ʹΦϑϩʔυ͠ɼෳ਺ϊʔυɾ(16ʹରԠͨ͠$$.࣮૷Λ։ൃ<>   ࢈૯ݚ"#$* 7ϊʔυ ͰੑೳධՁ

ຊ೥౓ͷݚڀ߲໨ w &%.ʹۙࣅLۙ๣୳ࡧΞϧΰϦζϜΛԠ༻͢Δ͜ͱʹΑΓɼ௒େن໛ͳσʔλ ηοτͷղੳΛ࣮ݱ͢Δ w Lۙ๣୳ࡧͷਫ਼౓͕&%.ͷ݁ՌʹͲͷΑ͏ʹӨڹΛ༩͑Δ͔໌Β͔ʹ͢Δ w 49"VSPSB546#"4"7FDUPS&OHJOF্ʹ͓͚Δ&%.ͷॳظੑೳධՁ w ϝϞϦ཯଎ͳܭࢉͳͷͰɼ7&্Ͱͷੑೳ͕ظ଴Ͱ͖Δ
w 7&্ʹ͓͚Δ5PQL୳ࡧͷੑೳΛ໌Β͔ʹ͢Δ 9

ۙࣅLۙ๣୳ࡧʹΑΔେن໛σʔλ΁ͷରԠ w ϕΫτϧ୳ࡧϥΠϒϥϦ'"*44Λ༻͍ɼҎԼͷۙࣅతLۙ๣୳ࡧΞϧΰϦζϜ Λ&%.ʹ࣮૷͠ɼൺֱධՁ w *OWFSUFE'JMF*OEFYσʔλ఺ΛΫϥελϦϯά͠ɼΫΤϦ఺ͷۙ๣ͷΫϥ ελ಺ͷσʔλ఺ͷΈ୳ࡧ w LE5SFFۭؒΛ࠶ؼతʹ௒ฏ໘Ͱ෼ׂ͠ɼΫΤϦ఺ؚ͕·ΕΔ෦෼ۭؒͷ पลͷΈ୳ࡧ
w )JFSBSDIJDBM/BWJHBCMF4NBMM8PSME )/48 σʔλ఺Λ௖఺ͱ͢Δάϥ ϑߏ଄Λߏங͠ɼΫΤϦ఺ʹྨࣅͨ͠௖఺Λᩦཉతʹ୳ࡧ w ςετσʔλͱͯ͠͸-PSFO[ํఔࣜͷղΛ࢖༻ 10

ۙࣅL//୳ࡧΛద༻ͨ͠ࡍͷ࣮ߦ࣌ؒ 11 1×10-1 1×100 1×101 1×102 1×103 1×104 1×104 1×105
1×106 Runtime [ms] L CPU Exact GPU Exact CPU IVF GPU IVF CPU HNSW CPU K-D Tree 1×10-1 1×100 1×101 1×102 1×103 1×104 1×104 1×105 1×106 Runtime [ms] L CPU Exact GPU Exact CPU IVF GPU IVF CPU HNSW CPU K-D Tree E = 1 E = 20 $16ຒΊࠐΈ࣍ݩ͕௿࣍ݩͷ৔߹͸LE5SFFɼߴ࣍ݩͷ৔߹͸)/48͕ߴ଎ (16࣌ܥྻ͕୹͍৔߹͸શ୳ࡧɼ௕͍৔߹͸*7'͕ߴ଎ ສεςοϓͰ΋   ඵະຬ Yߴ଎Խ Yߴ଎Խ

ۙࣅL//୳ࡧΛద༻ͨ͠ࡍͷ༧ଌਫ਼౓ 12 0.001 0.01 0.1 1 10 1×104 1×105 1×106
MAPE L Exact IVF HNSW K-D Tree 0.001 0.01 0.1 1 10 1×104 1×105 1×106 MAPE L Exact IVF HNSW K-D Tree 0 0.2 0.4 0.6 0.8 1 1×104 1×105 1×106 Recall L Exact IVF HNSW K-D Tree 0 0.2 0.4 0.6 0.8 1 1×104 1×105 1×106 Recall L Exact IVF HNSW K-D Tree Lۙ๣୳ࡧͷਫ਼౓ ۙ๣఺ͷ3FDBMMͰධՁ 4JNQMFY1SPKFDUJPOͷਫ਼౓ ."1& .FBO"CTPMVUF1FSDFOUBHF&SSPS ͰධՁ E = 1 E = 20 E = 1 E = 20 L//ͷ3FDBMM͕௿Լͯ͠΋ɼ4JNQMFYͷ."1&͸ఔ౓ͷ૿ՃˠۙࣅL//͕ޮՌత ۙࣅL//͸ݫີͳUPQLͰ͸ͳͯ͘΋ɼۙ๣ͷ఺Λฦ͢Մೳੑ͕ߴ͍͔Β͔   σʔλ͕ີʹ෼෍͍ͯ͠Δ͜ͱ͕લఏͳͷͰɼ࣮σʔλͰͷධՁ͕ඞཁ શ୳ࡧͱͷࠩ͸   3FDBMM͸௿Լ

7FDUPS&OHJOF্Ͱͷ&%. w શ୳ࡧʹΑΔLۙ๣୳ࡧॲཧ͸ɼڑ཭ߦྻͷܭࢉͱ5PQL୳ࡧʹΑͬͯߏ੒ w ڑ཭ߦྻͷܭࢉ͸ϕΫτϧԽ͕༰қͰ͋ΓɼϕΫτϧԽ཰ɾϕΫτϧ௕͸   े෼ʹߴ͍ w Ұํɼ7FDUPS&OHJOFͰͷ5PQL୳ࡧͷ࣮૷ɾੑೳධՁ͸গͳ͍ͷͰɼ  
ຊ೥౓͸5PQL୳ࡧͷࢼ࡞ɾੑೳධՁΛ࣮ࢪ 13

ج਺ιʔτ 14 330 100 010 212 030 323 021 102
002 121 330 100 010 030 212 102 002 323 021 121 -4%ج਺ιʔτ O E 100 102 002 010 212 021 121 323 330 030 002 010 021 030 323 330 100 102 121 212 330 100 010 212 030 323 021 102 002 121 010 030 021 002 100 102 121 212 330 323 002 010 021 030 100 102 121 100 102 .4%ج਺෦෼ιʔτ O E L 7&ɼ(16Ͱ͸ฒྻԽɾϕΫτϧԽ͕༰қͳج਺ιʔτͷ࢖༻͕Ұൠత   (16Ͱ͸ɼL͕খ͍͞৔߹ʹ͸ώʔϓιʔτΛ࢖༻

5PQL୳ࡧͷ࣮ߦ࣌ؒ 15 0.01 0.1 1 10 100 1000 1000 10000
100000 1x106 Runtime [ms] Length of vector STL full sort STL partial sort ASL full sort LSD radix full sort MSD radix partial sort L 0.001 0.01 0.1 1 10 100 1000 1×103 1×104 1×105 1×106 Runtime [ms] Array length (N) STL partial sort (VE) MSD radix partial sort (VE) STL full sort (Xeon) STL partial sort (Xeon) LSD radix full sort (Xeon) MSD radix partial sort (Xeon) w $ 45-ιʔτ TUETPSU ɼ$ 45-෦෼ιʔτ TUEQBSUJBM@TPSU ɼ/&$"4-ιʔτ BTM@TPSU@ ɼ-4%ج਺ιʔτ /&$ͷ044 ɼ.4%ج਺෦෼ιʔτ ಠ࣮ࣗ૷ Λൺֱ w 7FDUPS&OHJOF5ZQF#ͱ*OUFM9FPO4JMWFSͷίΞಉ࢜Λൺֱ 9FPOͰͷ   TUEQBSUJBM@TPSU͕   ৗʹ࠷଎

࣮σʔλͰͷ෼ੳ݁Ռ 16 /PSNPYJB ਖ਼ৗࢎૉೱ౓؀ڥ )ZQPYJB ௿ࢎૉೱ౓؀ڥ θϒϥϑΟογϡ   Χϧγ΢ϜΠϝʔδϯά  
ܭଌ݁Ռ ສχϡʔϩϯ ࣌ܥྻ Ͱग़ྗ͸(#ఔ౓

ݚڀۀ੷Ұཡ ֶज़࿦จ ࠪಡ͋Γ • Keichi Takahashi, Kohei Ichikawa, Joseph
Park, Gerald M. Pao, “Scalable Empirical Dynamic Modeling with Parallel Computing and Approximate k-NN Search,” IEEE Access. (in print) ࠃࡍձٞൃද ࠪಡͳ͠ • Keichi Takahashi and Gerald M. Pao, “Challenges in Scaling Empirical Dynamic Modeling,” 34th Workshop on Sustained Simulation Performance, October 24-25, 2022. ެ։ιϑτ΢ΣΞ • kEDM: https://github.com/keichi/kEDM ͦͷଞ • Keichi Takahashi, Kohei Ichikawa and Gerald M. Pao, “Toward Scalable Empirical Dynamic Modeling,” Sustained Simulation Performance 2022, Springer Cham, 2023. (in print) 17

·ͱΊͱࠓޙͷ՝୊ w ۙࣅLۙ๣୳ࡧ͸&%.ʹ༗ޮͰ͋Γɼ3FDBMM͕௿ͯ͘΋&%.ͷ݁Ռ΁ͷӨڹগ w σʔλͷಛੑʹґଘ͢ΔՄೳੑ͕͋ΔͷͰɼ࣮σʔλΛ༻͍ͯධՁ w 7FDUPS&OHJOF্Ͱͷ5PQL୳ࡧ͸9FPOΑΓੑೳ͕௿͍ w ڑ཭ܭࢉͱ5PQL୳ࡧΛͦΕͧΕ7&ͱ$16Ͱ෼୲͢Δ w
ۭؒॆరۂઢΛ༻͍ͨ7FDUPS&OHJOF্ͰͷۙࣅL//୳ࡧख๏<>ΛධՁ w ଟม਺ͷ4JNQMFY1SPKFDUJPOΛωοτϫʔΫ্ʹ૊Έ߹Θͤͨɼੜ੒తϞσϧ ͷߏஙͱݕূ<> 18 [1] খࣉխ࢘Β, “ۭؒॆరۂઢΛ༻͍ͨϕΫτϧϓϩηοαʹ͓͚Δkۙ๣๏ͷߴ଎Խ,” ୈ185ճHPCݚڀձ, 2022.   [2] Gerald M. Pao et al., “Experimentally testable whole brain manifolds that recapitulate behavior,” arXiv:2106.10627, 2021.

Implementation and Application of High-Performa...

Implementation and Application of High-Performance Empirical Dynamic Modeling

Keichi Takahashi

More Decks by Keichi Takahashi

Other Decks in Science

Featured

Transcript

౦๺େֶαΠόʔαΠΤϯεηϯλʔߴڮܛஐ Gerald M. Pao, Salk Institute for Biological Studies Implementation

&NQJSJDBM%ZOBNJD.PEFMJOH &%. w ඇઢܗ࣌ܥྻσʔλͷ෼ੳɾ༧ଌͷͨΊͷ৽͍͠ख๏܈ w ϞσϧϑϦʔͳख๏Ͱ͋Γɼ؍ଌσʔλͷΈʹΑΓσʔλۦಈతʹ෼ੳ w 5BLFOTͷຒΊࠐΈఆཧ< >ʹج͖ͮ஗Ԇ࠲ඪܥΛ༻͍ͯྗֶܥΛ෮ݩ 2

&%.ͷద༻ྫҨ఻ࢠൃݱྔͷ࣌ؒมԽ 3 300 250 200 150 100 50 0 180

4JNQMFY1SPKFDUJPOʹΑΔ୹ظ༧ଌ 4 ࣌ܥྻΛ࣌ؒ஗Ε࠲ඪܥΛ༻͍ͯঢ়ଶۭؒʹຒΊࠐΉ ݱࡏͷঢ়ଶͷۙ๣఺͕ɼ࣍ͷ࣌ؒεςοϓͰͲͷΑ͏ʹಈ͍͔ͨௐ΂Δ   ঢ়ଶ͕ྨࣅ͍ͯ͠Δ఺͸ɼ࣍ͷ࣌ؒεςοϓͰ΋ঢ়ଶ͕ྨࣅ͍ͯ͠Δͱߟ͑ΒΕΔ

4JNQMFY1SPKFDUJPOʹΑΔ୹ظ༧ଌ 5 ؍ଌ࣌ܥྻͷ࣌ؒ஗ΕΛ ࣍ݩͷঢ়ଶۭؒʹຒΊࠐΈ ঢ়ଶۭؒʹ͓͚Δ ͷ

࣌ܥྻ ͱ ΛͦΕͧΕঢ়ଶۭؒʹຒΊࠐΉ ঢ়ଶۭؒʹ͓͚Δ ͷۙ๣఺Λ୳ࡧ ͷۙ๣఺ͷ৘ใΛ༻͍ɼ

λʔήοτΞϓϦέʔγϣϯ w $$.ҼՌ෼ੳʹΑΓɼΧϧγ΢ϜΠϝʔδϯάʹΑΓܭଌͨ͠શ೴ͷ   ਆܦࡉ๔ؒͷҼՌؔ܎ωοτϫʔΫΛ෼ੳ w ສ࣌ؒεςοϓͷສݸͷ࣌ܥྻؒͷશମશͰ$$.ҼՌ෼ੳΛߦ͏ͱɼ   ԯ࣌ܥྻରͷ$$.ܭࢉ͕ඞཁ 7

Dataset # of time series # of time steps cppEDM

ۙࣅL//୳ࡧΛద༻ͨ͠ࡍͷ࣮ߦ࣌ؒ 11 1×10-1 1×100 1×101 1×102 1×103 1×104 1×104 1×105

ۙࣅL//୳ࡧΛద༻ͨ͠ࡍͷ༧ଌਫ਼౓ 12 0.001 0.01 0.1 1 10 1×104 1×105 1×106

7FDUPS&OHJOF্Ͱͷ&%. w શ୳ࡧʹΑΔLۙ๣୳ࡧॲཧ͸ɼڑ཭ߦྻͷܭࢉͱ5PQL୳ࡧʹΑͬͯߏ੒ w ڑ཭ߦྻͷܭࢉ͸ϕΫτϧԽ͕༰қͰ͋ΓɼϕΫτϧԽ཰ɾϕΫτϧ௕͸   े෼ʹߴ͍ w Ұํɼ7FDUPS&OHJOFͰͷ5PQL୳ࡧͷ࣮૷ɾੑೳධՁ͸গͳ͍ͷͰɼ

ج਺ιʔτ 14 330 100 010 212 030 323 021 102

5PQL୳ࡧͷ࣮ߦ࣌ؒ 15 0.01 0.1 1 10 100 1000 1000 10000

࣮σʔλͰͷ෼ੳ݁Ռ 16 /PSNPYJB ਖ਼ৗࢎૉೱ౓؀ڥ )ZQPYJB ௿ࢎૉೱ౓؀ڥ θϒϥϑΟογϡ   Χϧγ΢ϜΠϝʔδϯά

ݚڀۀ੷Ұཡ ֶज़࿦จ ࠪಡ͋Γ • Keichi Takahashi, Kohei Ichikawa, Joseph

·ͱΊͱࠓޙͷ՝୊ w ۙࣅLۙ๣୳ࡧ͸&%.ʹ༗ޮͰ͋Γɼ3FDBMM͕௿ͯ͘΋&%.ͷ݁Ռ΁ͷӨڹগ w σʔλͷಛੑʹґଘ͢ΔՄೳੑ͕͋ΔͷͰɼ࣮σʔλΛ༻͍ͯධՁ w 7FDUPS&OHJOF্Ͱͷ5PQL୳ࡧ͸9FPOΑΓੑೳ͕௿͍ w ڑ཭ܭࢉͱ5PQL୳ࡧΛͦΕͧΕ7&ͱ$16Ͱ෼୲͢Δ w