ReZero͕͋ΕେৎɻେਂͰͷߴऩଋ σΟʔϓωοτϫʔΫɺྖҬΛ͑ͯେ෯ͳੑೳ্ΛՄೳʹ͠·͕ͨ͠ɺଟ͘ͷ߹ɺফ ࣦ/രൃతͳޯʹ·͞Ε͍ͯ·͢ɻ͜ΕಛʹτϥϯεϑΥʔϚʔΞʔΩςΫνϟʹͯ ·Γɺେنͳσʔληοτܭࢉ༧ࢉ͕ͳ͍ͱ12Λ͑Δਂ͞ͷֶश͕ࠔͰ͢ɻҰൠత ʹɺඇޮͳ৴߸͕σΟʔϓωοτϫʔΫͷֶशΛ્͢Δ͜ͱ͕Θ͔͍ͬͯ·͢ɻτϥ ϯεͰɺϚϧνϔουͷࣗݾҙ͕͜ͷѱ͍৴߸ͷओͳݪҼͱͳ͍ͬͯ·͢ɻਂ৴߸ Λଅਐ͢ΔͨΊʹɺզʑReZeroΛఏҊ͠·͢ɻ͜ΕΞʔΩςΫνϟΛ؆୯ʹมߋͨ͠ ͷͰɺϨΠϠʔ͝ͱʹ1ͭͷՃֶशύϥϝʔλΛ༻ͯ͠ɺҙͷϨΠϠʔΛಉҰੑϚο ϓͱͯ͠ॳظԽ͢ΔͷͰ͢ɻզʑ͜ͷٕज़ΛݴޠϞσϦϯάʹద༻͠ɺ100Ҏ্ͷ ReZero-τϥϯεϑΥʔϚʔωοτϫʔΫΛ؆୯ʹ܇࿅Ͱ͖Δ͜ͱΛൃݟ͠·ͨ͠ɻ12ͷτ ϥϯεϑΥʔϚʔʹద༻͢Δͱɺenwiki8ͰReZero56%͘ऩଋ͠·͢ɻReZero TransformerΛ͑ͯଞͷࠩωοτϫʔΫʹద༻͞Εɺਂ͍શʹଓ͞ΕͨωοτϫʔΫ Ͱ1,500%͘ऩଋ͠ɺCIFAR 10Ͱ܇࿅͞ΕͨResNet-56Ͱ32%͘ऩଋ͠·͢ɻ https://arxiv.org/abs/2003.04887v1