Keeping LLMs in Their Lane: Focused AI for Data Science and Research

R+AI, 2025-11-12

Transcript

1. Joe Cheng, CTO of Posit
 R+AI, 2025-11-12 Keeping LLMs in Their Lane: Focused AI for Data Science and Research
2. ChatGPT Copilot ✨Custom Agents✨
4. Any data scientist can harness LLMs to create advanced agents. But should they?
5. “Our mission is to create open-source software for data science, scienti fi c research, and technical communication.” Posit Software, PBC “Public Bene fi t Corp”
6. Those who receive the results of modern data analysis have limited opportunity to verify the results by direct observation. Users of the analysis have no option but to trust the analysis, and by extension the software that produced it. Both the data analyst and the software provider therefore have a strong responsibility to produce a result that is trustworthy, and, if possible, one that can be shown to be trustworthy. This obligation I label the Prime Directive.
7. Emil Hvitfeldt, Software Engineer at Posit “I’m aware that if I make a mistake, bad things happen—death, and… other things.”
8. Ful fi lling the Prime Directive ✅ Correctness: (Obviously) ✅ Transparency: The methods of the analysis can be inspected ✅ Reproducibility: The analysis can be repeated on the same data, hopefully producing the same results
9. LLMs + (data) science A seemingly terrible idea! ❌ Correctness: LLMs are infamous for giving convincing but wrong answers ❌ Transparency: Nobody understands (yet) how/why LLMs do what they do ❌ Reproducibility: LLMs are nondeterministic black boxes
10. Source: GPT-4o Mini, 2025-02-21
11. Source: GPT-4o Mini, 2025-02-21
12. Source: GPT-4o Mini, 2025-02-21
13. LLMs aren’t bad, they’re jagged Coding The simplest
 data tasks ❌
14. How bad are LLMs with data? Example: length() # Make an array of random numbers, of length n values <- runif(n) client <- ellmer::chat("openai/gpt-4.1") client$chat("How long is this array?", jsonlite::toJSON(values)) • n=10, LLM says: 10 ✅ • n=100, LLM says: 100 ✅ • n=1000, LLM says: 1000 ✅ • n=10,000, LLM says: 1000 ❌ • n=103, LLM says: 100 ❌
15. What does responsible use of LLMs for data science look like?
16. Approach 1: Constrain
17. Approach 1: Constrain • Identify useful abilities that are fi rmly inside the LLM’s capability frontier • Augment the LLM with (safe, deterministic) tools to increase its usefulness • Instruct the LLM to stick to the prescribed task • Resist the urge to feature-creep to the edge of the capability frontier • Example: LLM -> SQL -> Dashboard
18. What is my purpose?
19. Pass the butter.
21. What is my purpose?
22. You pass butter.
23. Oh. My. God.
24. Is it responsible? • Correctness: Only generates SQL, and does it quite well • Transparency: Every SQL query is displayed to user • Reproducibility: The SQL is reproducible The “SQL chatbot applied to data dashboard” approach worked so well, we introduced an open-source package querychat to let anyone recreate the experience with their own data and visualizations
25. Approach 2: Micromanage
26. Approach 2: Micromanage • Very tight human-AI feedback loop • Outcomes that are pretty obviously right or wrong (or subjective) • Human micromanages the AI so closely that mistakes are all but guaranteed to be caught • Example: Plot tweaking tool
27. Let’s plot mtcars
29. The scatterplot points are a little small
31. The text annotations are all overlapping
32. The y axis needs to start at 0, obviously
33. Did you just use a diverging color palette for a categorical variable??
34. You’re absolutely right!
35. Is it responsible? • Correctness: Feels like it makes far fewer mistakes than a human does when fumbling through a visualization; mistakes are usually easy to catch • Transparency: The user is directing, and can see the R code at all times • Reproducibility: The R code is generally reproducible Somehow feels far lower stakes. Helps that a lot of aspects of data viz are subjective.
36. Approach 3: Deferred review
37. Approach 3: Deferred review • Stay in the loop with the AI, but with a looser hand • Be aware of what it’s doing and why, but don’t closely scrutinize its work for errors and hallucinations • Enjoy fast progress/exploration, while piling up “review debt” • Before “shipping” your work, stop and carefully review • Akin to working on a git branch and getting a code review before merging • Example: Databot
38. Is it responsible? • Correctness: Relies on human discipline (to take the time to review) and expertise (to spot problems in the analysis); or, rapidly improving models • Transparency: There’s R code, but it goes by pretty fast • Reproducibility: Databot will generate a reproducible report for you on demand High risk of misuse. But so incredibly useful...
39. • Constrain: “You pass butter” • Micromanage: “Not quite my tempo” • Deferred review: “YOLO now, pay later” • ___________: _________________
40. Likelihood of
 overlooking
 mistakes Likelihood of mistakes Querychat
 (SQL dashboard) Databot
 (EDA agent) ChatGPT’s
 Deep Research mode
 (5-20 min of research at a time) ggbot2
 (Plot tweaking)
41. Likelihood of
 overlooking
 mistakes Likelihood of mistakes Querychat
 (SQL dashboard) Databot
 (EDA agent) ChatGPT’s
 Deep Research mode
 (5-20 min of research at a time) ggbot2
 (Plot tweaking)
42. Likelihood of
 overlooking
 mistakes Likelihood of mistakes Querychat
 (SQL dashboard) Databot
 (EDA agent) ChatGPT’s
 Deep Research mode
 (5-20 min of research at a time) ggbot2
 (Plot tweaking)
43. Learn more • YouTube: “Harnessing LLMs for Data Analysis” • {ellmer}: Easily call LLMs from R • {querychat}: Enhance Shiny data dashboards with LLMs that speak SQL • Databot: Exploratory Data Analysis agent for Positron