Data_strategy___engineering_for_agentic_workflows.pdf

Transcript

1. Data strategy & engineering for agentic workflows Ray Grieselhuber Founder & CEO, DemandSphere @MKE DMC April 15, 2026
2. Agenda • Intro • Background and motivations for this topic • Vibe coding vs. AI-driven engineering • Tools & skills • Testing & security • Project ideas • Open weight models vs. closed models • Final thoughts
3. Introduction • BA, Japanese & B.S. Computer Science • Software engineering with focus on early ML, agents, big data, distributed systems, etc. • SEO since 2006, built first enterprise platform on the market • Founded DemandSphere (GinzaMetrics) in 2009, Tokyo • Moved to Silicon Valley in 2010 for YC • Work with clients all over the world
4. Quick description of our platform for context
5. Today we’re building a map, don’t worry too much
6. Goal is to promote more fluency in building with AI
7. Manager mode vs. Founder mode
8. We are all in Founder mode now
9. Founder mode means: • Staying close to the user (even if it’s yourself at first) • Build what people want • Iterate quickly • Keep your sleeves rolled up
10. Software as a Service Or Service as a Software?
11. Experience at Functionize taught me about the new SaaS
12. The new SaaS (Service as a Software): • AI is empowered by human attention • Businesses pay for results, they don’t care about your cool AI systems if they don’t work • Humans have to backstop when AI systems fail
13. The dirty little secret in SaaS
14. Silicon Valley is finally starting to figure this out
15. Vibe coding vs. AI-driven engineering
16. Don’t look down on vibe coding
17. Vibe coding is good for building prototypes and testing ideas quickly
18. AI-driven engineering is for building products
19. Vibe coding should be the starting point
20. Deployments and operations are what separate vibe coding and AI-driven engineering
21. People and processes (manual & automated) support actual products
22. Simple example: AI frontier model tracker
23. Constant updates + QA (human & machine)
24. Cambrian explosion of tools and ideas
25. Need to anchor to business goals
26. But the cost of experimentation is cheaper
27. Let’s talk about some basics
28. Tokens are the currency of AI and heavy context spends more tokens
29. Tokens = cost + latency + attention budget
30. The best AI development workflows: • Compress • Summarize • Deduplicate • Retrieve only what is needed
31. It’s about curation of the context window
32. (This also applies to AI search & SEO btw)
33. Tools you should use & know
34. Skills (also an open standard)
35. Be very careful with skills you did not build (supply chain attack)
36. Claude Code
37. Working in a team environment
38. Context files
39. CLAUDE.md
40. CLAUDE.md files can be hierarchical
41. Per branch
42. Per subfolder
43. Have Claude Code generate code maps for context
44. Have Claude Code generate ERD diagrams for databases
45. Mermaid and Mermaid.live are great for diagramming
46. The point is that maintaining context for humans is just as important as maintaining it for agents
47. Simple text “standards” are always the thing to bet on
48. HTML Vanilla JS JSON CSV Prompts Git (based on hashes) Markdown (.md) Mermaid
49. Monitor your /context and /compact when needed
50. A basic vibe coding to AI-driven engineering workflow Start in Claude.ai Vanilla prototype Claude.ai to build CLAUDE.md Move to local filesystem claude init Create repo Enforce testing policy Configure deployment environment Automate deployments Deploy regularly
51. Let’s talk about building MCP servers
52. MCP servers start off as local and “self-hosted” until you deploy them
53. Remember the difference between vibe coding and AI-driven engineering? Deployment!
54. Security & good tool construction are the hardest parts
55. Be sure to create an automated test suite
56. Security risks with MCPs: • Client side ◦ Giving too much access to the MCPs ◦ Prompt injection ◦ Data / key leakage • Server side ◦ Validating tool inputs ◦ Sanitizing output ◦ Rate limits ◦ Access controls
57. Let’s get into solving some real problems
58. In search, we are trying to get into the context windows of our audience’s minds
59. Context windows are everywhere
60. This means we need lots of data about their behavior and motivations
61. GSC Analytics data Search volume SERP data Log files LLM data Query fanouts (the list is always growing)
62. The first step is deciding where this data is going to live
63. Then how to get it
64. APIs by themselves have challenges
65. MCPs can help to make tool usage of APIs more efficient
66. But MCPs are transient
67. API direct to dashboards introduces latency
68. API direct to agents is an improvement but granular
69. You can solve a lot of problems with data warehouses
70. BigQuery, Clickhouse, and Snowflake
71. BigQuery is probably the best default choice
72. Using BQ as a data buffer between systems
73. Knowledge bases for agentic workflows
74. Context files vs. LLM wikis vs. RAG
75. Entity canonicalization is a really hard problem
76. Final thoughts
77. AI feudalism is coming
78. Make the investment to own your own compute capabilities
79. Learn how to work with open weight models
80. Going to SEO Week? https://luma.com/p7r3nbvk
81. Join us at FOUND Tokyo 2027
82. Thank you!