Float

下記で取り扱った浮動小数点(Float型)についての記述を公開します。

・DeepLearningと量子化(Quantization)
https://lib-arts.booth.pm/items/6074054

IEEE 754に基づく小数の表現については大まかに抑えておくと良いのではないかと思います。

Transcript

1. A.2 BNN(1-bit ͰͷྔࢠԽ)* Deep Neural Networks) Ͱ͸ਤ A.2 ͷΑ͏ͳΞϧΰϦζϜʹجֶ͍ͮͯश ΍ਪ࿦͕ߦΘΕ·͢ɻ A.2 BNN(1-bit ͰͷྔࢠԽ)* BNN(Binarized Neural Networks) ͸ֶशύϥϝʔλ (weights) Λ +1 ͱ −1 ͷ 1-bit Ͱදݱͨ͠ BinaryConnect Λ֦ுͨ͠ݚڀͰ͢ɻBNN Ͱ͸ύ ϥϝʔλͱಉ༷ʹ׆ੑԽؔ਺ͷྔࢠԽ΋ߦ͍·͢ɻ ˛ਤ A.3 BNN(Binarized Neural Networks) ࿦จ https://arxiv.org/abs/1602.02830v3 A.3 ුಈখ਺఺ɾFloat ܕ ౰અ (A.3 અ) Ͱ͸ҎԼɺԼهͷ Wikipedia Λࢀߟʹුಈখ਺఺ʹ͍ͭͯ ·ͱΊ·͢ɻ 41
2. ෇࿥ A Appendix https://ja.wikipedia.org/wiki/ුಈখ਺఺਺ A.3.1 ුಈখ਺఺ͷ࢓૊Έ ුಈখ਺఺ (floating-point) ͸࣮਺ΛίϯϐϡʔλͰॲཧ͢Δʹ͋ͨͬͯ ༗ݶݸͷখ਺ͰऔΓѻ͏ํࣜͰ͢ɻFloat ܕ͸͜ͷΑ͏ͳํࣜʹج͍ͮͯ ਺͕ද͞Ε·͢ɻҎԼɺුಈখ਺఺ʹ͓͚Δ਺ͷදݱͷ࢓૊Έʹ͍ͭͯ֬ೝ ͠·͢ɻ ·ͣɺ10 ਐ਺Ͱ − 1 3 Λදݱ͢Δ৔߹ΛԾఆ͠·͢ɻ− 1 3 ͸ࣜ A.1 ͷΑ͏ ʹද͢͜ͱ͕Ͱ͖·͢ɻ ࣜ A.1: − 1 3 ͷۙࣅ (10 ਐ਺) − 1 3 = −1 × 0.33333 · · · ≃ −1 × 0.3333 × 100 = −1 × 3.333 × 10−1 ࣜ A.1 Ͱ͸ 0.33333 · · · Λ 4 ܻͷখ਺Ͱۙࣅ͠·ͨ͠ɻԼͷ 2 ͭͷࣜ ͷදهΛՊֶه਺๏ (scientific notation) ͱ͍͍·͢ɻࣜ A.1 ͷҰ൪Լͷ −1 × 3.333 × 10−1 ͷΑ͏ͳදهΛਖ਼نԽ਺ (normalized number) ͱݺͼ ·͢ɻ ਖ਼نԽ਺ (normalized number) ͸ɺ్தʹग़ͯ͘Δখ਺ͷ࠷্Ґͷܻ͕ 0 Ҏ֎ͷ਺ࣈΛऔΔ਺ࣈΛݴ͍·͢ɻ·ͨɺࣜ A.1 ͷ 10 ਐ਺ͷ৔߹ͱಉ༷ ʹ − 1 3 ͸ࣜ A.2 ͷΑ͏ʹ 2 ਐ਺Ͱۙࣅ͢Δ͜ͱ΋Ͱ͖·͢ɻ 42
3. A.3 ුಈখ਺఺ɾFloat ܕ ࣜ A.2: − 1 3 ͷۙࣅ (2 ਐ਺) ᶃ ɹ − 1 3 = −1 × 0.01010101010101010101010101010101 · · · ≃ −1 × 0.0101010101010101 = −1 × 2−2 + 2−4 + 2−6 + 2−8 + 2−10 + 2−12 + 2−14 + 2−16 = −1 × 1 4 + 1 16 + 1 64 + 1 256 + 1 1024 + 1 4096 + 1 16834 + 1 65536 = −0.333328247 (ۙࣅ஋ͷ 10 ਐ਺දه) ࣜ A.2 ͷΑ͏ͳ 2 ਐ਺ͷදه΋ࣜ A.3 ͷΑ͏ʹՊֶه਺๏ (scientific notation) Ͱද͢͜ͱ͕Ͱ͖·͢ɻ ࣜ A.3: − 1 3 ͷۙࣅ (2 ਐ਺) ᶄ − 1 3 ≃ −1 × 0.0101010101010101 × 20 = −1 × 0.101010101010101 × 2−1 = −1 × 1.01010101010101 × 2−2 ࣜ A.3 ͷҰ൪Լͷ͕ࣜਖ਼نԽ਺ʹରԠ͠·͢ɻਖ਼نԽ਺ (normalized number) ͷࣜ͸ූ߸ (sign) ͷ SɺԾ਺ (significand) ͷ Fɺࢦ਺ (exponent) ͷ E Λݩʹࣜ A.4 ͷΑ͏ʹද͢͜ͱ͕Ͱ͖·͢ɻ ࣜ A.4: ਖ਼نԽ਺ (normalized number) ͷࣜ (−1)S × F × 2E ࣜ A.4 ͷΑ͏ͳਖ਼نԽ਺ͷࣜΛݩʹ IEEE ํࣜ (IEEE 754 ํࣜ) ΍ IBM ํ͕ࣜఆٛ͞Ε·͢ɻҎԼɺ࠷΋޿͘࠾༻͞Ε͍ͯΔඪ४ن֨Ͱ͋Δ IEEE 754 ํࣜʹ͍ͭͯ֬ೝ͠·͢ɻIEEE 754 Ͱ͸ද A.1 ͷΑ͏ʹʮ൒ਫ਼౓ʯ ɺ ʮ୯ਫ਼౓ʯ ɺ ʮഒਫ਼౓ʯ͕ఆٛ͞Ε·͢ɻ 43
4. ෇࿥ A Appendix ˝ද A.1 ຊॻͰऔΓѻͬͨ૊ΈࠐΈؔ਺ͷҰཡ ਫ਼౓ ූ߸෦ ࢦ਺෦ Ծ਺෦ ൒ਫ਼౓ (16 Ϗοτ) 1 Ϗοτ 5 Ϗοτ 10 Ϗοτ ୯ਫ਼౓ (32 Ϗοτ) 1 Ϗοτ 8 Ϗοτ 23 Ϗοτ ഒਫ਼౓ (64 Ϗοτ) 1 Ϗοτ 11 Ϗοτ 52 Ϗοτ ූ߸෦͸ਖ਼ෛͷූ߸ɺࢦ਺෦͸ 2 ͷྦྷ৐ɺԾ਺෦͸খ਺෦෼ʹͦΕͧΕର Ԡ͠·͢ɻͨͱ͑͹൒ਫ਼౓ͷ৔߹ɺූ߸෦Λ sɺԾ਺෦ (੔਺෦෼͕ 1 Ͱ͋ Δ 2 ਐখ਺ͷখ਺෦෼) Λ fɺࢦ਺෦Λ e ͱ͓͘ͱࣜ A.5 ͷΑ͏ʹখ਺Λද ݱ͠·͢ɻ ࣜ A.5: ൒ਫ਼౓ (IEEE 754) ͷখ਺ͷදݱ (−1)s × 2e−15 × (1 + f) ࣜ A.5 Ͱ͸ 0 ≤ e ≤ 25 − 1 ΑΓɺ−15 ≤ e − 15 ≤ 16 ͕੒ཱ͠·͢ɻ· ͨɺ210 = 1024 ΑΓɺf ͸ 1 1024 ࠁΈͰදݱ͞Ε·͢ɻ ಉ༷ʹ ʮ୯ਫ਼౓ʯ ͸ 0 ≤ e ≤ 28 −1 ΑΓࣜ A.6ɺ ʮഒਫ਼౓ʯ ͸ 0 ≤ e ≤ 211 −1 ΑΓࣜ A.7 ͷΑ͏ͳ਺ࣜͰͦΕͧΕදݱ͢Δ͜ͱ͕Ͱ͖·͢ɻ ࣜ A.6: ୯ਫ਼౓ (IEEE 754) ͷখ਺ͷදݱ (−1)s × 2e−127 × (1 + f) ࣜ A.7: ഒਫ਼౓ (IEEE 754) ͷখ਺ͷදݱ (−1)s × 2e−1023 × (1 + f) A.3.2 IEEE 754 Ͱුಈখ਺఺Λදݱ͢Δ·ͰͷྲྀΕ ᶃ ࢦ਺ͷج਺Λ 2 ͱ͢Δ ᶄ Ծ਺Λਖ਼نԽ͢Δ 44
5. A.3 ුಈখ਺఺ɾFloat ܕ ᶅ ූ߸෦ɺࢦ਺෦ɺԾ਺෦Λ 2 ਐ਺ͷϏοτͰදݱ͠ɺ࠷্ҐϏοτ ͔Βූ߸෦ɺࢦ਺෦ɺԾ਺෦ͷॱʹهࡌ جຊతʹ͸্هͷྲྀΕʹجͮ͘͜ͱͰ 10 ਐ਺ͷখ਺Λුಈখ਺఺ͷදه Ͱද͢͜ͱ͕Ͱ͖·͢ɻҎԼɺ2.5 ͷ୯ਫ਼౓Ͱͷුಈখ਺఺ԽΛྫʹྲྀΕͷ ֬ೝΛߦ͍·͢ɻ ·ͣɺᶃͰ͸ 2.5 Λ 2.5 = (−1)0 × 2.5 × 20 ͷΑ͏ʹද͠·͢ɻ࣍ʹᶄͰ ͸ (−1)0 × 2.5 × 20 Λ (−1)0 × 2.5 × 20 = (−1)0 × (1 + 0.25) × 2128−127 ͷΑ͏ʹม׵͠·͢ɻ ଓ͘ᶅͰ͸ූ߸෦ͷ 0 Λ 0ɺԾ਺෦ͷ 0.25 Λ 010 0000 0000 0000 000 0 0000ɺࢦ਺෦ͷ 128 Λ 1000 0000 Ͱද͠·͢ɻ͜ΕΒʹج͍ͮͯɺ010 0 0000 0010 0000 0000 0000 0000 0000 ͕ಘΒΕ·͢ɻ 45