Share
Embed
Interactivity and Programming in the tidyverse
Lionel Henry
January 30, 2020
Transcript
31 slides
-
• Idea of blending data with the workspace • Helps
"turning ideas into software" (John Chambers) but hinders code reuse • Progress in tooling and teaching tidy eval made easy?? Data-masking in R -
1988 — The New S Language (Bell labs) attach(starwars) mean(height,
na.rm = TRUE) #> [1] 174.358 Data-masking in R -
1993 — Statistical Models in S lm( birth_year ~ mass
+ height, starwars ) Data-masking in R -
1997 — frametools (Peter Dalgaard, R core) aq <- airquality[1:10,]
subset.frame(aq, Ozone > 20) select.frame(aq, Ozone:Temp) modify.frame(aq, ratio = Ozone / Temp) Data-masking in R -
select.frame(aq, Ozone:Temp) First apparition of selections 1997 — frametools (Peter
Dalgaard, R core) Data-masking in R -
subset.frame(aq, Ozone > 20) select.frame(aq, Ozone:Temp) 0.62 subset(aq, Ozone >
20, select = Ozone:Temp) modify.frame(aq, ratio = Ozone / Temp) transform(aq, ratio = Ozone / Temp) -
2001 — Luke Tierney bmi <- with( starwars, mass /
(height / 100)^2 ) starwars <- within( starwars, bmi <- mass / (height / 100)^2 ) 2007 — Peter Dalgaard Few developments after inclusion of frametools Data-masking in R -
2006 — data.table starwars[ mass > 150, name:mass ] dt[i,
j] • Data-masking in i • Selections in j Data-masking in R Most new developments in package space -
Data-masking in R 2014 — dplyr Most new developments in
package space airquality %>% filter(Ozone > 20) %>% select(Ozone:Temp) %>% mutate(ratio = Ozone / Temp) -
Trouble in data-masking town This is a convenience function intended
for use interactively [...] The non-standard evaluation [...] can have unanticipated consequences. ?subset ‟ ” ?transform -
Trouble in data-masking town 1. Unexpected masking by data-variables 2.
Data-variables can't get through arguments The tidyverse offers solutions for both issues Ambiguity between data-variables and environment-variables (workspace) -
1. Unexpected masking n <- 100 data.frame(x = 1) %>%
mutate(y = x / n) %>% pull(y) #> [1] 0.01 -
n <- 100 data.frame(x = 1, n = 2) %>%
mutate(y = x / n) %>% pull(y) #> [1] 0.5 data.frame(x = 1) %>% mutate(y = x / n) %>% pull(y) #> [1] 0.01 Data frame is a moving part 1. Unexpected masking -
n <- 100 data <- data.frame(x = 1, n =
2) data %>% mutate(y = .data$x / .env$n) • Use the .env pronoun to refer to the workspace • Use the .data pronoun to refer to the data frame Solution: Be explicit in production code 1. Unexpected masking -
iris %>% mean_by(Species, Sepal.Width) #> Error: Column `by` is unknown
mean_by <- function(data, by, var) { data %>% group_by(by) %>% summarise(avg = mean(var)) } 2. Data-variables through arguments -
iris %>% mean_by(Species, Sepal.Width) #> Error: Column `by` is unknown
mean_by <- function(data, by, var) { data %>% group_by(by) %>% summarise(avg = mean(var)) } • env-variable by • data-variable Species 2. Data-variables through arguments -
iris %>% my_function(Species, Sepal.Width) #> Species avg #> <fct> <dbl>
#> 1 setosa 3.43 #> 2 versicolor 2.77 #> 3 virginica 2.97 mean_by <- function(data, by, var) { data %>% group_by({{ by }}) %>% summarise(avg = mean({{ var }})) } Tunnel the data-variable through the env-variable with the {{ }} operator 2. Data-variables through arguments -
mean_by <- function(data, by, var) { data %>% group_by({{ by
}}) %>% summarise(avg = mean({{ var }})) } iris %>% my_function(Species, Sepal.Width) #> Species avg #> <fct> <dbl> #> 1 setosa 3.43 #> 2 versicolor 2.77 #> 3 virginica 2.97 Tunnel the data-variable through the env-variable with the {{ }} operator Hard-coded result name? 2. Data-variables through arguments -
iris %>% my_function(Species, Sepal.Width) #> Species Sepal.Width #> <fct> <dbl>
#> 1 setosa 3.43 #> 2 versicolor 2.77 #> 3 virginica 2.97 mean_by <- function(data, by, var) { data %>% group_by({{ by }}) %>% summarise("{{ var }}" := mean({{ var }})) } Tunnel data-variable inside strings! Hard-coded result name? Variant of glue syntax 2. Data-variables through arguments -
Tunnelling causes data-masking to propagate iris %>% my_function(Species, Sepal.Width) iris
%>% my_function(.data$Species, .data$Sepal.Width) Can we wrap tidyverse pipelines without data-masking contagion? 2. Data-variables through arguments -
2. Hard to reuse code in functions iris %>% group_by(.data$Species)
%>% summarise(avg = mean(.data$Sepal.Width)) -
2. Hard to reuse code in functions data %>% group_by(.data[[by]])
%>% summarise(avg = mean(.data[[var]])) Subset .data with [[ -
2. Hard to reuse code in functions mean_by <- function(data,
by, var) { data %>% group_by(.data[[by]]) %>% summarise(avg = mean(.data[[var]])) } iris %>% my_function("Species", "Sepal.Width") #> Species avg #> <fct> <dbl> #> 1 setosa 3.43 #> 2 versicolor 2.77 #> 3 virginica 2.97 Subset .data with [[ -
2. Hard to reuse code in functions iris %>% my_function("Species",
"Sepal.Width") #> Species Sepal.Width #> <fct> <dbl> #> 1 setosa 3.43 #> 2 versicolor 2.77 #> 3 virginica 2.97 mean_by <- function(data, by, var) { data %>% group_by(.data[[by]]) %>% summarise("{var}" := mean(.data[[var]], na.rm = TRUE)) } Use single { to glue the string -
Trouble in data-masking town 1. Unexpected masking by data-variables •
Use .data and .env to disambiguate 2. Data-variables can't get through arguments • Tunnel data-variables with {{ }} • Subset .data with [[ -
What about selections? Selections are a separate sublanguage starwars %>%
select(name:mass) starwars %>% select(c(name, mass)) starwars %>% select(1:3) starwars %>% select(c(1, 3)) ⟺ • Data-variables represent locations • Ambiguity much less an issue -
What about selections? Use all_of() to disambiguate name <- c("mass",
"height") starwars %>% select(name) Data-variable Env-variable starwars %>% select(all_of(name)) -
x <- c("Sepal.Length", "Petal.Length") iris %>% averages(x) #> Sepal.Length Sepal.Width
Petal.Length Petal.Width #> 5.843333 3.057333 3.758000 1.199333 Take character vectors with all_of() averages <- function(data, vars) { data %>% select(all_of(vars)) %>% map_dbl(mean, na.rm = TRUE) } -
iris %>% averages(starts_with("Sepal")) #> Sepal.Length Sepal.Width #> 5.843333 3.057333 Tunnel
selections with {{ }} averages <- function(data, vars) { data %>% select({{ vars }}) %>% map_dbl(mean, na.rm = TRUE) } -
1. Use .data / .env or all_of() to disambiguate 2.
Tunnel data-variables and selections with {{ }}