Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Measure the whole SERP at any scale #brightonSEO

Measure the whole SERP at any scale #brightonSEO

Slides from my brightonSEO talk on November 11, 2023 in San Diego, CA.

The era of traditional rank tracking is over.

In this era of SEO, we need to understand the value that can be derived from, instead, monitoring and measuring the entire SERP.

The shape of the modern SERP is evolving quickly.

It is critical to understand both the ""the how and the why"" of this data, which can be leveraged not only by SEOs but also by Paid, Content, and Product teams.

There are numerous challenges involved in collecting accurate data at scale and even more challenges in turning this data into signals that are relevant for business teams.

Scale is not only about technical capture but also ensuring maximum impact and we discuss how to ensure that both criteria are met.

We also introduce some key KPIs and metrics that can be used to measure the whole SERP, including visual ranking, pixel depth, pixel height, fold visibility, and more.

Ray Grieselhuber

November 11, 2023

More Decks by Ray Grieselhuber

Other Decks in Marketing & SEO


  1. Measuring the whole SERP at any scale Ray Grieselhuber DemandSphere

    Speakerdeck.com/raygrieselhuber @raygrieselhuber
  2. But the shape of the SERP is changing faster than

    ever Speakerdeck.com/raygrieselhuber @raygrieselhuber
  3. Impacts: • Fold visibility • CTR • Clicks / Sessions

    • Conversions • Revenue 600+ distinct elements across the SERPs Paid Content • Pixel depth: 120px • Pixel height: 320px • Visual position: 1 • Nested position: 2 Organic Result • Pixel depth: 440px • Pixel height: 330px • Visual position: 3 • Title changed
  4. Daily layout changes on Google SERPs Paid Content • Position

    (order of items) • Product Name • Price • URL • Pixel Data Organic Results • Organic Position • Visual Position • Title • Meta Description • Product URL • Pixel Data Popular Products • Position (order of items) • Product Name • Price • URL • Pixel Data
  5. Speakerdeck.com/raygrieselhuber @raygrieselhuber Scale driver #2: The cost of money is

    affecting investment in ads Money supply SPIKED but now is decreasing Money is not as cheap anymore
  6. Even better answer: exploding use cases Speakerdeck.com/raygrieselhuber @raygrieselhuber Complexity increases

    Developer • APIs • Data lakes SEO Analyst • SERP data • Crawl data • GSC • GA • Exploration Content Team • Content & topics • Content performance • Visibility trends Product • Technical • SERP rankings • Indexation • PAA + Knowledge Graph Executive / Reporting • Permissions • Dashboards • Speed of answers • Forecasting • Predictive Performance Team • Ad data • Hotel Ads • Shopping • Google for Jobs • etc.
  7. Typical data pipelines Speakerdeck.com/raygrieselhuber @raygrieselhuber Fetching (HTML) ETL (JSON &

    DB) Analytics (DB) Dashboards, etc. SERPs (HTML) GSC GA4 Build? Buy? Build? Buy? Keywords
  8. Where do development bottlenecks occur? Speakerdeck.com/raygrieselhuber @raygrieselhuber Fetching (HTML) ETL

    (JSON & DB) Analytics (DB) Dashboards, etc. SERPs (HTML) GSC GA4 Build? Buy? Build? Buy? Keywords
  9. Similar use cases Speakerdeck.com/raygrieselhuber @raygrieselhuber Data Operations Observability (log files

    + search index / database) Problem domains Asset / Technology Parallels to SEO • Log files: SERPs • Parsed log files: Parsed SERPS (JSON, etc.) • Index: ??? Data Science BI Datasets ML Models Dashboards & Visualizations • SERPs • GSC • GA4 • Keyword sets, etc. • Looker Studio • Superset • DOMO • etc.
  10. Tools that can help accelerate and support many use cases

    Speakerdeck.com/raygrieselhuber @raygrieselhuber Data Lakes BI / Dashboards / Data apps Cloud Data Warehouse • S3-compatible object storage • BigQuery • Snowflake • Redshift • Looker Studio • Superset • Jupyter & other notebooks • Python data apps (Dash, Streamlit)
  11. SQL is a decent lingua franca for turning SERP data

    into BI Speakerdeck.com/raygrieselhuber @raygrieselhuber select search_domain, url, title, desc, position, people_also_asked_snippets from results.organic_results where fetch_date = “2023-11-08” limit 100000
  12. We mapped every single element on the SERP and put

    it into a data warehouse Speakerdeck.com/raygrieselhuber @raygrieselhuber
  13. As data science matures, we’re seeing many Python-based tools become

    available Speakerdeck.com/raygrieselhuber @raygrieselhuber • Pandas / Dask / Pola.rs • Data Science notebooks • Numpy • PyTorch • Scrapy • Advertools • Plotly • spaCy • NLTK
  14. Simplified access to SERP data Speakerdeck.com/raygrieselhuber @raygrieselhuber Developers SEO Team

    Content Team Product Team Executive / Reporting Performance Team APIs Data Warehouse Data Lakes Data Warehouse Data Warehouse SEO & Content Tools BI
  15. Let’s talk! Speakerdeck.com/raygrieselhuber @raygrieselhuber • Scan the QR code to

    interact with these slides on Slido • Follow us at x.com/demandsphere • Connect with me at linkedin.com/in/raygrieselhuber