Upgrade to Pro — share decks privately, control downloads, hide ads and more …

よく聞くけど使ったことないソフトウェアNo.1 KafkaとSnowflake

Avatar for numa numa
April 22, 2024

よく聞くけど使ったことないソフトウェアNo.1 KafkaとSnowflake

2024年4月22日に行われたPyData Okinawaで行なったLTです
https://pydataokinawa.connpass.com/event/314900/

言及してるQuickStartはこれ
Getting Started with Snowpipe Streaming and Amazon MSK
- https://quickstarts.snowflake.com/guide/getting_started_with_snowpipe_streaming_aws_msk/index.html#0

Getting Started with Snowflake and Amazon Data Firehose
- https://quickstarts.snowflake.com/guide/getting_started_with_snowflake_and_aws_kdf/index.html

Avatar for numa

numa

April 22, 2024
Tweet

More Decks by numa

Other Decks in Technology

Transcript

  1. ˜$IVSB%"5"JOD ,BGLBͷΞʔΩςΫνϟ #SPLFSDMVTUFS ;PP,FFQFSDMVTUFS ΞϓϦέʔγϣϯ 1VCMJTIFS ΞϓϦέʔγϣϯ 4VCTDSJCFS ,BGLB1SPEVDFS ˞ॻ͖ࠐΈϥΠϒϥϦ

    ,BGLB$POTVNFS ˞ಡΈࠐΈϥΠϒϥϦ #SPLFS #SPLFS #SPLFS 5PQJD 1BSUJUJPO 1BSUJUJPO 1BSUJUJPO 1BSUJUJPO 1BSUJUJPO 1BSUJUJPO ;PP,FFQFS ;PP,FFQFS ;PP,FFQFS • ,BGLB͸ෳ਺#SPLFSͰΫϥελߏ ੒ɺΫϥελ৘ใ͸;PP,FFQFS ͕࣋ͪɺ#SPLFSͷ୆͕Ϧʔμʔ ͱͳΓΫϥελ؅ཧ • ,BGLBͷϝοηʔδ͸ΩʔόϦϡʔ ܗࣜͷ3FDPSEͱݺ͹Εɺͭͷ 5PQJD͸ෳ਺ͷ#SPLFSʹ෼ࢄ͞Εͨ 1BSUJUJPOͰߏ੒ɺ5PQJD΁ͷ3FDPSE ͷಡΈॻ͖Λ1BSUJUJPO୯ҐͰ෼ࢄฒ ྻԽ͢Δ͜ͱͰߴ଎ͳॲཧΛ࣮ݱ
  2. ˜$IVSB%"5"JOD 4OPXGMBLFͷ,BGLB$POOFDUFS͸ํࣜ͋Δ ,BGLB5PQJD 1BSUJUJPO 1BSUJUJPO 1BSUJUJPO 1BSUJUJPO ,BGLB$POOFDUFS QPMM ˞IUUQTEPDTTOPXGMBLFDPNFOVTFSHVJEFEBUBMPBETOPXQJQFTUSFBNJOHLBGLBهࡌͷਤΛݩʹ୯७Խͯ͠࡞ਤ͠·ͨ͠

    4OPXGMBLF *OUFSOBM4UBHF 4 +40/'JMF 5BCMF 4OPXQJQF 4OPXQJQF 4OPXQJQF 4USFBNJOH ,BGLB5PQJD 1BSUJUJPO 1BSUJUJPO 1BSUJUJPO 1BSUJUJPO ,BGLB$POOFDUFS QPMM 4OPXGMBLF 5BCMF $IBOOOFM $IBOOOFM $IBOOOFM $IBOOOFM • 4OPXQJQF ্ ͸ɺ,BGDB$POOFDUFSͰσʔλΛड͚ͯɺ4΁ϑΝΠϧΛ165ɺͦͷޙ4OPXQJQFͱ͍͏ σʔλऔΓࠐΈϓϩάϥϜΛ3&45"1*Ͱىಈ͢ΔͷͰɺ෼஗Ε͘Β͍ͰऔΓࠐ·ΕΔɺ͢ͳΘͪϨΠς ϯγେ͖Ί • 4OPXQJQF4USFBNJOH Լ ͸ɺ,BGLB$POOFDUFSͰड͚ͯ1BSUJPOͱಉ͡਺ͷ$IBOOFMͰฒྻऔΓࠐΈΛߦ͏ ͷͰϨΠςϯγ͸ඵ୯ҐͰύϑΥʔϚϯε΋ߴ͍ʢ(#Tͱ͔ग़ΔΒ͍͠Αʣ • අ༻తʹ͸࣌ؒҐͰສߦྲྀ͠ࠐΜͰɺυϧ͘Β͍ͩΑ ˞1BSUJUJPOͱ$IBOOOFM͸ ͔͜͜Βࠨ͸ετϦʔϛϯά ͔͜͜Βӈ͸ϚΠΫϩόον