Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Using Thanos as a long-term storage for your Pr...

Using Thanos as a long-term storage for your Prometheus metrics

Moto Ishizawa

January 15, 2020
Tweet

More Decks by Moto Ishizawa

Other Decks in Technology

Transcript

  1. Prometheus ಠࣗΞμϓλʔ ؅ཧऀ ΦϒδΣΫτετϨʔδ Remote Read/Write API ϝτϦΫεΛࢀর ϝτϦΫεΛసૹ ࣮૷ҊಠࣗΞμϓλʔͷ։ൃ

    w1SPNFUIFVTͱΦϒδΣΫτετϨʔδΛܨ͙͚ͩͷ୯७ͳΞμϓλʔΛ։ൃ w1SPNFUIFVTͷ3FNPUF3FBE8SJUF"1*ͱ࣮ͯ͠૷ w1SPNFUIFVTͷ54%#ΛGPSLͯ͠ɺΦϒδΣΫτετϨʔδʹDIVOLΛอଘ͢ΔΑ͏मਖ਼ w؅ཧऀ΍։ൃऀ͔Β͸1SPNFUIFVTͷετϨʔδ͚͕ͩେ͖͘ͳͬͨΑ͏ʹݟ͑Δ wϝτϦΫεͷࢀরʹඞཁͳͷͰ1SPNFUIFVTʹ͸े෼ͳ3".͕·ͩඞཁ
  2. 5IBOPTͷίϯϙʔωϯτ 4JEFDBS 1SPNFUIFVTͷσʔλσΟϨΫτϦΛ؂ࢹͯ͠ɺΦϒδΣΫτετϨʔδʹϝτϦΫεΛΞοϓϩʔυ͢ Δɻ1SPNFUIFVTͱͷؔ܎ͰσϓϩΠ͢Δɻ 4UPSF ΦϒδΣΫτετϨʔδʹอଘ͞ΕͨϝτϦΫεʹΞΫηε͢ΔͨΊͷ"1*ɻ௨ৗ͸2VFSZͱ૊Έ߹Θͤ ͯ࢖͏ɻ 2VFSZ 1SPN2-Λ࢖༻ͯ͠ϝτϦΫεΛूܭ͢Δɻ1SPNFUIFVTޓ׵ͷ6*ͱ"1*Λඋ͍͑ͯΔͨΊɺ 1SPNFUIFVTͱಉ͡Α͏ʹ࢖͑Δɻ

    3VMF 1SPNFUIFVTͷϧʔϧϑΝΠϧΛಡΈࠐΈɺΞϥʔτΛੜ੒͢ΔɻΦϒδΣΫτετϨʔδʹอଘ͞Εͨ ϝτϦΫεશମʹϧʔϧΛద༻͍ͨ͠৔߹ͳͲʹ࢖༻ɻΞϥʔτͷૹ৴͸"MFSUNBOBHFSΛ࢖͏ɻ $PNQBDU ΦϒδΣΫτετϨʔδʹอଘ͞ΕͨϝτϦΫεΛఆظతʹू໿ͨ͠Γμ΢ϯαϯϓϦϯά͢Δɻϲ݄ܦ աͨ͠ϝτϦΫεΛ෼·ͨ͸࣌ؒ͝ͱʹαϯϓϦϯά͠ͳ͓͢ɺͱ͍ͬͨ͜ͱ͕Ͱ͖Δɻ 3FDFJWF 3FNPUF8SJUF"1*Λඋ͓͑ͯΓɺ֎෦͔ΒϝτϦΫεΛड͚෇͚ΔɻૹΒΕ͖ͯͨϝτϦΫε͸ 4JEFDBSͱಉ༷ʹΦϒδΣΫτετϨʔδʹอଘ͢Δɻ·࣮ͩݧతͳѻ͍ɻ
  3. 5IBOPT3FDFJWFͷ࣮ߦ $ thanos receive \ --tsdb.path /var/lib/thanos \ # TSDB

    ϑΝΠϧͷύε --tsdb.retention 3h \ # TSDB ϑΝΠϧͷอ࣋ظݶ --objstore.config-file /etc/thanos/bucket_config.yaml \ # ΦϒδΣΫτετϨʔδͷઃఆ --receive.local-endpoint http://thanos1.local:19291/api/v1/receive \ # ࣗ਎ͷΤϯυϙΠϯτ໊ --receive.hashrings-file /etc/thanos/hashring.json \ # ৑௕ԽͷͨΊͷϋογϡϦϯάͷઃఆ --receive.replication-factor 3 # Replication Factor ͷઃఆ --label "thanos='1'" ɹɹɹɹɹɹ# Thanos ͷ External Label [ { "hashring": "cluster", "endpoints": [ "http://thanos1.local:19291/api/v1/receive", "http://thanos2.local:19291/api/v1/receive", "http://thanos3.local:19291/api/v1/receive" ] } ] /etc/thanos/hashring.json