Big Opportunities in Small Data

Citus Con 2023 - video here: https://www.youtube.com/watch?v=rsE0XhlPnug

Transcript

1. Big Opportunities in Small Data Simon Willison, Citus Con 2023
2. Small Data?
3. Data Journalism
4. What’s the best possible way of publishing structured data?
6. Demo: SF City Facilities
7. Read-only SQL queries via an API
14. SQLite
15. It’s installed already (even on my phone) A SQLite database is a single file Very stable file format
16. Datasette Python web app SQLite database fi les (read-only data) One deployable container Docker, K8s, Vercel, AWS Lambda, Google Cloud Run, Fly… Baked Data
17. HTTP load balancer Datasette Python web app SQLite database fi les Datasette Python web app SQLite database fi les Datasette Python web app SQLite database fi les
18. SQL plus HTTP is a fantastic integration tool
23. Baked Data with PostgreSQL? Deploy a container with a full read-only database
24. Datasette in WebAssembly in your browser
25. Demo: Datasette Lite
26. PostgreSQL WebAssembly
27. SQLite is serverless (before serverless was cool)
28. SQLite, DuckDB… PostgreSQL?
29. SQLite, DuckDB… PostgreSQL?
30. PostgreSQL as a library? $ pip install postgreslib (or npm, etc) import postgreslib db = postgreslib.connect( "/mnt/data.postgresql" ) db.execute("create table …")
31. Ideas to take away • Small Data deserves more tooling • Scale read-only data up and down with the Baked Data pattern • Read-only SQL APIs are a really good idea! • Especially since you can reformat data with a SQL query • You can run databases in the browser now • Please build me PostgreSQL as a library!
32. Questions