Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Tidy evaluation: programming with ggplot2 and d...

Tidy evaluation: programming with ggplot2 and dplyr

Learn how to program with tidyverse functions that "automatically quote" their input

Hadley Wickham

March 08, 2018
Tweet

More Decks by Hadley Wickham

Other Decks in Education

Transcript

  1. (df$a - min(df$a)) / (max(df$a) - min(df$a)) (df$b - min(df$b))

    / (max(df$b) - min(df$b)) (df$c - min(df$c)) / (max(df$c) - min(df$c)) (df$d - min(df$d)) / (max(df$d) - min(df$d)) Rule of three: make a function if you’ve copy-pasted threes times
  2. (df$a - min(df$a)) / (max(df$a) - min(df$a)) (df$b - min(df$b))

    / (max(df$b) - min(df$b)) (df$c - min(df$c)) / (max(df$c) - min(df$c)) (df$d - min(df$d)) / (max(df$d) - min(df$d)) First, identify the parts that might change
  3. (df$a - min(df$a)) / (max(df$a) - min(df$a)) (df$b - min(df$b))

    / (max(df$b) - min(df$b)) (df$c - min(df$c)) / (max(df$c) - min(df$c)) (df$d - min(df$d)) / (max(df$d) - min(df$d)) Then give them names x x x x
  4. rescale01 <- function(x) { (df$a - min(df$a)) / (max(df$a) -

    min(df$a)) } Then copy in one example
  5. rescale01 <- function(x) { rng <- range(x) (x - rng[1])

    / (rng[2] - rng[1])) } And maybe refactor a little
  6. rescale01 <- function(x) { rng <- range(x, na.rm = TRUE,

    finite = TRUE) (x - rng[1]) / (rng[2] - rng[1])) } And handle more cases
  7. df %>% group_by(x1) %>% summarise(mean = mean(y1)) df %>% group_by(x2)

    %>% summarise(mean = mean(y2)) df %>% group_by(x3) %>% summarise(mean = mean(y3)) df %>% group_by(x4) %>% summarise(mean = mean(y4)) Let’s try with some dplyr code
  8. df %>% group_by(x1) %>% summarise(mean = mean(y1)) df %>% group_by(x2)

    %>% summarise(mean = mean(y2)) df %>% group_by(x3) %>% summarise(mean = mean(y3)) df %>% group_by(x4) %>% summarise(mean = mean(y4)) First identify the parts that change
  9. df %>% group_by(x1) %>% summarise(mean = mean(y1)) df %>% group_by(x2)

    %>% summarise(mean = mean(y2)) df %>% group_by(x3) %>% summarise(mean = mean(y3)) df %>% group_by(x4) %>% summarise(mean = mean(y4)) Then give them names summary_var group_var df
  10. grouped_mean <- function(df, group_var, summary_var) { df %>% group_by(group_var) %>%

    summarise(mean = mean(summary_var)) } Now make a function
  11. grouped_mean <- function(df, group_var, summary_var) { df %>% group_by(group_var) %>%

    summarise(mean = mean(summary_var)) } grouped_mean(mtcars, cyl, mpg) #> Error: Column `group_var` is unknown It doesn’t work
  12. (x - min(x)) / (max(x) - min(x)) mtcars %>% group_by(cyl)

    %>% summarise(mean = mean(mpg)) We need some new vocabulary Evaluated using usual R rules Automatically quoted and evaluated in a “non-standard” way
  13. df <- data.frame( y = 1, var = 2 )

    df$y var <- "y" df$var You’re already familiar with this idea Predict the output!
  14. df <- data.frame( y = 1, var = 2 )

    df$y #> [1] 1 var <- "y" df$var #> [1] 2 $ automatically quotes the variable name
  15. df <- data.frame( y = 1, var = 2 )

    var <- "y" df[[var]] #> [1] 1 If you want refer indirectly, must use [[ instead
  16. library(MASS) #> Works MASS #> Error: object 'MASS' not found

    # -> The 1st argument of library() is quoted Can’t tell? Try running the code
  17. subset(mtcars, cyl == 4) #> Works cyl == 4 #>

    Error: object 'cyl' not found # -> The 2nd argument of subset() is quoted Can’t tell? Try running the code
  18. Base R has 3 primary ways to “unquote” Quoted/Direct Evaluated/Indirect

    df$y x <- "y"
 df[[x]] library(MASS) x <- "MASS"
 library(x, character.only = TRUE) rm(mtcars) x <- "mtcars"
 rm(list = x)
  19. library(tidyverse) mtcars %>% pull(am) by_cyl <- mtcars %>% group_by(cyl) %>%

    summarise(mean = mean(mpg)) ggplot(by_cyl, aes(cyl, mpg)) + geom_point() Identify which arguments are auto-quoted
  20. library(tidyverse) mtcars %>% pull(am) by_cyl <- mtcars %>% group_by(cyl) %>%

    summarise(mean = mean(mpg)) ggplot(by_cyl, aes(cyl, mpg)) + geom_point() Identify which arguments are auto-quoted
  21. Quoted Evaluated Tidy Direct df$y df[["y"]] pull(df, y) Indirect var

    <- "y"
 df[[var]] var <- quo(y)
 pull(df, !!var)
  22. x_var <- quo(cyl) y_var <- quo(mpg) by_cyl <- mtcars %>%

    group_by(!!x_var) %>% summarise(mean = mean(!!y_var)) ggplot(by_cyl, aes(!!x_var, !!y_var)) + geom_point() Everywhere in the tidyverse uses !! to unquote Pronounced bang-bang
  23. df %>% group_by(x1) %>% summarise(mean = mean(y1)) df %>% group_by(x2)

    %>% summarise(mean = mean(y2)) df %>% group_by(x3) %>% summarise(mean = mean(y3)) df %>% group_by(x4) %>% summarise(mean = mean(y4)) New: Identify quoted vs. evaluated arguments
  24. df %>% group_by(x1) %>% summarise(mean = mean(y1)) df %>% group_by(x2)

    %>% summarise(mean = mean(y2)) df %>% group_by(x3) %>% summarise(mean = mean(y3)) df %>% group_by(x4) %>% summarise(mean = mean(y4)) New: Identify quoted vs. evaluated arguments
  25. df %>% group_by(x1) %>% summarise(mean = mean(y1)) df %>% group_by(x2)

    %>% summarise(mean = mean(y2)) df %>% group_by(x3) %>% summarise(mean = mean(y3)) df %>% group_by(x4) %>% summarise(mean = mean(y4)) Then identify the parts that could change
  26. df %>% group_by(x1) %>% summarise(mean = mean(y1)) df %>% group_by(x2)

    %>% summarise(mean = mean(y2)) df %>% group_by(x3) %>% summarise(mean = mean(y3)) df %>% group_by(x4) %>% summarise(mean = mean(y4)) These become the function arguments summary_var group_var df
  27. grouped_mean <- function(df, group_var, summary_var) { data %>% group_by(group_var) %>%

    summarise(mean = mean(summary_var)) } Next write the function template & identify quoted arguments
  28. grouped_mean <- function(df, group_var, summary_var) { group_var <- enquo(group_var) summary_var

    <- enquo(summary_var) data %>% group_by(group_var) %>% summarise(mean = mean(summary_var)) } New: Wrap every quoted argument in enquo()
  29. grouped_mean <- function(df, group_var, summary_var) { group_var <- enquo(group_var) summary_var

    <- enquo(summary_var) data %>% group_by(!!group_var) %>% summarise(mean = mean(!!summary_var)) } New: And then unquote with !!
  30. filter(diamonds, x > 0 & y > 0 & z

    > 0) # vs diamonds[ diamonds$x > 0 & diamonds$y > 0 & diamonds$z > 0, ] It saves a lot of typing
  31. filter(diamonds, x > 0 & y > 0 & z

    > 0) # vs diamonds[ diamonds[["x"]] > 0 & diamonds[["y"]] > 0 & diamonds[["z"]] > 0, ] It saves a lot of typing
  32. mtcars_db %>% filter(cyl > 2) %>% select(mpg:hp) %>% head(10) %>%

    show_query() #> SELECT `mpg`, `cyl`, `disp`, `hp` #> FROM `mtcars` #> WHERE (`cyl` > 2.0) #> LIMIT 10 And makes it possible to translate to other languages
  33. 1. R code is a tree 2. Unquoting builds trees

    3. Environments map 
 names to values Now for some theory
  34. f x "y" 1 A function call First child =

    function Other children = arguments
  35. > lobstr::ast(if(x > 5) y + 1) █#`if` $#█#`>` %

    $#x % &#5 &#█#`+` $#y &#1 You can see this yourself with lobstr::ast()
  36. x1 <- expr(a + b) expr(f(!!x1, z)) #> f(a +

    b, z) # !! is called the unquoting operator # And is pronounced bang-bang Unquoting allows you to build your own trees
  37. + a b x1 <- expr(a + b) f z

    expr(f(!!x1, z)) x1
  38. ex1 <- expr(x + y) ex2 <- expr(!!ex1 + z)

    ex3 <- expr(1 / !!ex1) Predict what this code will return
  39. ex1 <- expr(x + y) # x + y ex2

    <- expr(!!ex1 + z) ex3 <- expr(1 / !!ex1) Predict what this code will return
  40. ex1 <- expr(x + y) # x + y ex2

    <- expr(!!ex1 + z) # x + y + z ex3 <- expr(1 / !!ex1) Predict what this code will return
  41. ex1 <- expr(x + y) # x + y ex2

    <- expr(!!ex1 + z) # x + y + z ex3 <- expr(1 / !!ex1) # 1 / (x + y) # Not 1 / x + y Predict what this code will return
  42. # expr() quotes your expression f1 <- function(z) expr(z) f1(a

    + b) #> z # enexpr() quotes user’s expression f2 <- function(z) enexpr(z) f2(x + y) #> x + y enexpr() lets you capture user expressions
  43. my_mutate <- function(df, var) { n <- 10 var <-

    enexpr(var) mutate(df, y = !!var) } df <- tibble(x = 1) n <- 100 my_mutate(df, x + n) #> x y #> 1 1.00 11 Capturing just expression isn’t enough
  44. my_mutate <- function(df, var) { n <- 10 var <-

    enexpr(var) mutate(df, y = !!var) } df <- tibble(x = 1) n <- 100 my_mutate(df, x + n) #> x y #> 1 1.00 11
  45. # quo() quotes your expression f1 <- function(z) quo(z) f1(a

    + b) #> <quosure> #> expr: ^z #> env: 0x10d3b9308 # enquo() quotes user’s expression f2 <- function(z) enquo(z) f2(x + y) #> <quosure> #> expr: ^x + y #> env: 0x10d3b9309 quo() captures expression and environment
  46. my_mutate <- function(df, var) { n <- 10 var <-

    enquo(var) mutate(df, y = !!var) } df <- tibble(x = 1) n <- 100 my_mutate(df, x + n) #> x y #> 1 1.00 101
  47. my_mutate <- function(df, var) { n <- 10 var <-

    enquo(var) mutate(df, y = !!var) } df <- tibble(x = 1) n <- 100 my_mutate(df, x + n) #> x y #> 1 1.00 101
  48. df <- data.frame(x = 1:5, y = 5:1) filter(df, abs(x)

    > 1e-3) filter(df, abs(y) > 1e-3) filter(df, abs(z) > 1e-3) my_filter <- function(df, var) { var <- enquo(var) filter(df, abs(!!var) > 1e-3) } my_filter(df, x) Key pattern is to quote and unquote Quote Unquote
  49. df1 %>% group_by(g1) %>% summarise(mean = mean(a)) df2 %>% group_by(g2)

    %>% summarise(mean = mean(b)) df3 %>% group_by(g3) %>% summarise(mean = mean(c)) df4 %>% group_by(g4) %>% summarise(mean = mean(d)) Tidy eval lets you reduce duplication df1 %>% grouped_mean(g1, a) df2 %>% grouped_mean(g2, b) df3 %>% grouped_mean(g3, c) df4 %>% grouped_mean(g4, d)
  50. Code is a tree f y !!x `-` 1 Build

    trees with unquoting Quote to capture code + env enquo() Learn more https://adv-r.hadley.nz/expressions.html https://adv-r.hadley.nz/quasiquotation.html https://adv-r.hadley.nz/evaluation.html WIP 2nd ed
  51. This work is licensed as Creative Commons
 Attribution-ShareAlike 4.0 


    International To view a copy of this license, visit 
 https://creativecommons.org/licenses/by-sa/4.0/