Lab 29: Shiny Crude Oil Forecast (Multivariate ARIMA) App

Time Series Fans - This one is for you! In Lab 29, you learn a workflow for Multivariate ARIMA Forecasting using Lagged Predictors, one of the most critical techniques in time series forecasting.

You learn to forecast using the new Fable library, part of the tidyverts ecosystem of time series & forecasting tools. We connect to the Quandl API to collect Energy Data & package the analysis in a Shiny Web Application for automated forecasting.

Transcript

1. With the Quandl API Matt Dancho & David Curry Business Science Learning Lab Shiny Crude Oil App (ARIMA)
2. Learning Lab Structure • Presentation (20 min) • Demo’s (30 min) • Pro-Tips (15 mins)
3. Shiny API Series • Lab 28 - Shiny Real Estate App ◦ Zillow API • Lab 29 - Shiny Oil & Gas App ◦ Quandl API • Lab 30 - Shiny Finance App ◦ Tidyquant API • Lab 31 - Shiny Marketing App ◦ Google Analytics API
4. Learning Labs PRO Every 2-Weeks 1-Hour Course Recordings + Code + Slack $19/month university.business-science.io Lab 29 - Shiny API Series, Pt 2 Shiny + Quandl + ARIMA for Energy Forecasting Lab 28 - Shiny API Series, Pt 1 Shiny + Zillow for Real Estate Lab 27 - Marketing Series, Pt 4 Google Trends Automation with Shiny Lab 26 - Marketing Series, Pt 3 Customer Journey with Machine Learning Lab 25 - Marketing Series, Pt 2 Attribution with ChannelAttribution Lab 24 - Marketing Series, Pt 1 A/B Testing with Infer Lab 23 - SQL Series SQL with BigQuery & Conversion Funnel Lab 22 - SQL Series SQL for Time Series Continuous Learning Advanced Topics
5. Agenda • Demo ◦ Shiny App that Forecasts Crude Oil (WTI) Prices • Why a Crude Oil ARIMA App? ◦ Helping businesses predict future impacts • Data Science Workflow Concepts ◦ What Impacts Prices? ◦ TS Feat. Engineering ◦ Multivariate ARIMA ◦ Deploying Models into Production • 30-Min Demo ◦ Quandl API ◦ Recipes Prep. ◦ ARIMA Workflow ◦ Shiny 5-Model ARIMA App [LL PRO] • Pro-Tips & Learning Guide
6. Demo
7. Why make a WTI Forecast App? Energy
8. Real Story I used to work for a valve & fitting manufacturer Main consumers were Oil & Gas. Product Sales were tied to oil prices. More correctly, sales were tied to Lags in Oil Prices. Since we knew which way oil prices were going, we could predict revenue & product demand.
9. API + ARIMA + Shiny How an app can help Quandl has a vast amount of energy & financial data. • Historical Stock Prices • Energy Data (EIA) • Economic Data (FRED) We can use this data to forecast the WTI Crude Oil Price. Forecasting (NEW)
10. Benefits Oil & Gas Companies can better forecast revenue
11. Benefits Product Suppliers can anticipate demand
12. Benefits Transportation can better estimate fuel costs
13. How to Forecast WTI Challenging Problem
14. EIA & Academia https://arxiv.org/pdf/1811.08963.pdf https://www.eia.gov/
15. EIA & Academia https://arxiv.org/pdf/1811.08963.pdf https://www.eia.gov/finance/markets/crudeoil/ EIA G. Natarajan and A. Ashok
16. Time Series Analysis In 3 minutes
17. Concept 1: Lag Lags Offsets (shifts) the value 1 or more time units.
18. Concept 2: Lags Plots & Correlation Lag Plots Compare Autocorrelation WTI vs Lag(WTI) Cross Correlation WTI vs Lag(Predictor) WTI vs Lag(WTI, n = 1:12)
19. Concept 3: Predicting with Lags
20. Crude Oil Forecasting Concepts & Workflow
21. Workflow Step-By-Step Start Finish 1 2 3 Quandl API Connect to Quandl Preprocess & Visualize Data ARIMA Create models that forecast using lagged features Shiny App that productionizes the models
22. Forecast App
23. 30-Min Demo Let’s do this!
24. PRO-TIPS Yeahhhhhh!
25. Pro-Tip #1: Give Businesses Apps Businesses Need Apps Businesses can’t scale reports Businesses can scale apps Apps help decision making
26. Pro-Tip #2: Leverage Shiny by integrating your analysis Shiny is production for your analysis Shiny packages up: Machine Learning Visualizations Interactivity Use Shiny to help decision making
27. Pro-Tip #3: Say no to reports No https://arxiv.org/pdf/1811.08963.pdf
28. Pro-Tip #3: ...and say YES to apps. Yes apps.business-science.io
29. Advanced Visualization Advanced Data Wrangling Advanced Functional Programming & Modeling Advanced Machine Learning Visualization Data Cleaning & Manipulation Data Science Algorithms & Iteration Business Reporting Business Analysis with R (DS4B 101-R) Data Science For Business with R (DS4B 201-R) Web Apps & Shiny Developer (DS4B 102-R + DS4B 202A-R) Web Apps Data Science Foundations 7 Weeks Machine Learning & Business Consulting 10 Weeks Web Application Development 12 Weeks -TRACK Project-Based Courses with Business Application Business Science University R-Track 4-Course R-Track System
30. Key Benefits - Fundamentals - Weeks 1-5 (25 hours of Video Lessons) - Data Manipulation (dplyr) - Time series (lubridate) - Text (stringr) - Categorical (forcats) - Visualization (ggplot2) - Programming & Iteration (purrr) - 3 Challenges - Machine Learning - Week 6 (8 hours of Video Lessons) - Clustering (3 hours) - Regression (5 hours) - 2 Challenges - Learn Business Reporting - Week 7 - RMarkdown & plotly - 2 Project Reports: 1. Product Pricing Algo 2. Customer Segmentation Visualization Data Cleaning & Manipulation Business Reporting Business Analysis with R (DS4B 101-R) Data Science Foundations 7 Weeks Data Science Algorithms & Iteration
31. Key Benefits Understanding the Problem & Preparing Data - Weeks 1-4 - Project Setup & Framework - Business Understanding / Sizing Problem - Tidy Evaluation - rlang - EDA - Exploring Data -GGally, skimr - Data Preparation - recipes - Correlation Analysis - 3 Challenges Machine Learning - Weeks 5, 6, 7 - H2O AutoML - Modeling Churn - ML Performance - LIME Feature Explanation Return-On-Investment - Weeks 7, 8, 9 - Expected Value Framework - Threshold Optimization - Sensitivity Analysis - Recommendation Algorithm Data Science For Business (DS4B 201-R) Machine Learning & Business Consulting 10 Weeks Advanced Visualization Advanced Data Wrangling Advanced Functional Programming & Modeling Advanced Data Science End-to-End Churn Project
32. Key Benefits Learn Shiny & Flexdashboard - Build Applications - Learn Reactive Programming - Integrate Machine Learning App #1: Predictive Pricing App - Model Product Portfolio - XGBoost Pricing Prediction - Generate new products instantly App #2: Sales Dashboard with Demand Forecasting - Model Demand History - Segment Forecasts by Product & Customer - XGBoost Time Series Forecast - Generate new forecasts instantly Shiny Apps for Business (DS4B 102-R) Web Application Development 4 Weeks Web Apps Machine Learning
33. Key Benefits Frontend + Backend + Production Deployment Frontend for Shiny - Bootstrap Backend for Shiny - MongoDB - Dynamic UI - User Authentication - Store & Write User Data Production Deployment - AWS - EC2 Server - VPC Connection - URL Routing Shiny Apps for Business (DS4B 202A-R) Web Application Development 6 Weeks
34. -TRACK BUNDLE 15% OFF PROMO Code: learninglabs $127/mo Limited Time
35. Begin Learning Today university.business-science.io