Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building a more Efficient Firestore Web App

Building a more Efficient Firestore Web App

Use Firestore in your Web App? Learn some tips and tricks to help you optimize Firestore billing.

Firebase Thailand

April 01, 2023
Tweet

More Decks by Firebase Thailand

Other Decks in Technology

Transcript

  1. Building a More Efficient Firestore Web App Google Developer Expert

    in Firebase Lim Shang Yi Firebase Dev Day 2023 GDG Bangkok Firebase Thailand Organized by
  2. • Store and sync data in real-time • NoSQL, document

    collections structure • Offline handling and auto scaling
  3. A common consideration With Firestore you are billed by the

    number of document reads and writes. So customers tend to be mindful of how often their data gets read from the server.
  4. Billed Reads Estimation 5,000,000 document reads 10,000 monthly visitors 20

    items on page 5 page refresh x x 5 pages loaded x
  5. Choosing the right tech for future scale As an engineer,

    I want to choose a technology that will help accomplish my near term goals without too much effort. But that same technology must also support my future ambitions, in terms of scalability, cost, compliance, governance, etc.
  6. I want to show data that doesn’t change often, such

    as the top 10 sales of the week. Which function should I use? getDocs onSnapshot
  7. Reading Data from Firestore 1. getDocs() Call to get the

    data once. 2. onSnapshot() Set a listener to receive data-change events.
  8. I want to show data that doesn’t change often, such

    as the top 10 sales of the week. Which function should I use? getDocs onSnapshot
  9. Billed Reads Estimation (getDocs) 10,000 monthly visitors 10 items on

    page 2 navigation between pages 2 page refresh x x x 400,000 document reads
  10. Billed Reads Estimation (onSnapshot) 10,000 monthly visitors 10 items on

    page 2 navigation between pages 2 page refresh x x x 200,000 document reads ( ↓200,000 reads)
  11. Reading documents with onSnapshot App onSnapshot Cloud Firestore Firebase SDK

    Cache onSnapshot Snapshot listener preserved Only fresh reads (such as a reload)
  12. Billed Reads Estimation with onSnapshot() 10,000 daily visitors 20 items

    on page 20 page refresh x x 4,000,000 document reads
  13. Reading documents with getFromCache App get Cloud Firestore Firebase SDK

    Cache getFromCache Only fresh reads (such as a reload)
  14. const db = getFirestore(); // remember to enable persistence enableIndexedDbPersistence(db);

    try { const querySnapshot = await getDocsFromCache(collection(db, "products")); let docs = []; if (querySnapshot.empty) { throw new Error("cache empty, refetch"); } querySnapshot.forEach((doc) => { docs.push(doc.data()); }); // do something } catch (err) { await getDocs(collection(db, "products")); // do something }
  15. Billed Reads Estimation 10,000 daily visitors 20 items on page

    20 1 page refresh x x 200,000 document reads (↓3,800,000 reads)
  16. Problem: Still querying the server daily for each of the

    10,000 visitors, even if content are the same
  17. What are Firestore Bundles? • Packaged results of common queries

    • Allow apps to load data quickly without querying backend • Save billed reads by loading from cache
  18. Products Bundle: Build it Cloud Firestore Firebase Server SDK await

    getDocs db.bundle() .add(‘products’, docs).build() Bundle
  19. var bundleId = "latest-products"; var bundle = firestore.bundle(bundleId); var querySnapshot

    = await firestore.collection('product').get(); // Build the bundle // Note how querySnapshot is named "latest-stories-query" var bundleBuffer = bundle .add('latest-products-query', querySnapshot) // Add a named query. .build()
  20. Reading documents with bundles App Firebase Server SDK Remote storage

    Bundle Create bundle Fetch & read bundle Store
  21. // Fetch the bundle from Firebase Hosting, if the CDN

    cache is hit the 'X-Cache' // response header will be set to 'HIT' const resp = await fetch('/createBundle'); // Load the bundle contents into the Firestore SDK await db.loadBundle(resp.body); // Query the results from the cache // Note: omitting "source: cache" will query the Firestore backend. const query = await db.namedQuery('latest-products-query'); const storiesSnap = await query.get({ source: 'cache' });
  22. Billed Reads Estimation 10,000 daily visitors 20 items on page

    1 page refresh x x 20 document reads (↓199,980)
  23. What is Client-Side Indexing? • Brings indexing to the Firestore

    Client SDK • Indexing always on server, new to client • Indexing enables very efficient queries
  24. // You will normally read this from a file asset

    or cloud storage. let indexConfigJson = { indexes: [ ... ], fieldOverrides: [ ... ] } // Apply the configuration. Firestore.firestore().setIndexConfiguration(indexConfigJson)
  25. When bundling makes sense: • Config data read in upon

    startup. • Data that doesn’t change frequently e.g top 10 stories every day. • Show standing data when user is not logged in
  26. When bundling does not makes sense: • Data that changes

    very frequently • Any data that contains private information
  27. Reading common or less frequently changed data App onSnapshot(‘cart’) Cloud

    Firestore onValue(‘promotions’) Realtime Database
  28. Counting documents via Cloud Functions App Cloud Functions Firestore Trigger

    getDoc(‘orders’) Cloud Firestore updateDoc(‘orders’, { count : count +1 })
  29. 10,000 document reads 100 document writes (update counter) 10,000 monthly

    visitors 1 post x 100 orders x Billed Reads Estimation (Counter)
  30. Billed Reads Estimation (Aggregate Queries) 11,000 document reads (10,000 listing

    read + 1000 reads from count) ( 1 document read / 1,000 indexes) 10,000 monthly visitors 1 listing x 100 orders x
  31. In summary • Understand when to use getDocs vs onSnapshot

    • Use bundles to save billing reads • Take advantage of caches for different scenarios, client side indexing to speed it up • Consider using Realtime Database as alternative for standing data • Use aggregate queries instead of counting manually to reduce billing reads