Share
Embed
Democratize Deep Learning Models
A perspective on how to make deep learning models more accessible and usable for a wider spectrum of people. Using tensorflow.js
1. Educate: Explorable Explanations
2. Create: Rapid Prototyping
3. Visualise: Model Visualisation
4. Intervene: Decision Making
5. Imagine: Generative Spaces
Amit Kapoor
June 30, 2020
Transcript
48 slides
-
Democratize Deep Learning Models Using TensorFlow.js Perspective & Views Amit
Kapoor Bargava Subramanian 1 -
Amit Kapoor Data | Visual | Story — Teaching Analytics
& Vis — Consulting & Mentoring — Building DataVis Tools amitkaps.com 2 -
Bargava Subramanian Data | AI & ML | Productivity —
Startup @Binaize — Data Scientist (ex @Cisco, @RedHat) — Consulting on ML products bargava.com 3 -
Deep Learning Challenges — Scale Up: Faster data & models
— Scale Out: Bigger data & models — Scale Wide: Entire data & model pipelines — Scale Across: Beyond data scientists & engineers How do we get more Human in the Loop approach to using Deep Learning? 4 -
Democratize Deep Learning (DL) 1. Educate: Why will DL work
for our use case? 2. Create: What can we do to build our DL model? 3. Visualise: So how do we improve our DL models? 4. Intervene: So what actions can we take using DL? 5. Imagine: So what else can we use DL for? 5 -
Perspective on Democratize DL 1. Educate: Explorable Explanations 2. Create:
Rapid Prototyping 3. Visualise: Model Visualisation 4. Intervene: Decision Making 5. Imagine: Generative Spaces 6 -
Structure for Each Perspective — Conceptual: Meaning & View —
In Real Life: Context & challenge — Our Approach: Our Solutions & Experiments — Exemplars: Open-source Examples & Libraries — Learnings: Lessons learned 7 -
Lens – Tensorflow.js — Javascript version of Tensorflow Deep Learning
Library — Core (Linear Algebra low level API) — Layers (Keras-type high level API) — Runs both on Client-side (JS, WebAssembly, WebGL) and on Server-side (Nodejs with C binary for CPU & GPU) — Can be used for both inference and training — Custom models optimised for Image, Text, Audio etc. — Can Import & use models build in Python version 8 -
1. Educate Explorable Explanations “For the things we have to
learn before we can do them, we learn by doing them.” ― Aristotle 9 -
In Real Life: Context Client: Leading US Network Service Provider
Scope: Threat detection using User & Entity Behaviour Analytics Status: Unsustainable Rule Based Policies Mandate: Evaluate shift to ML/DL Models Challenge: Domain experts skeptical of ML/DL paradigm 11 -
How to Educate? — Work with sample or open data
— Show the potential for ML/DL models — Need to showcase visually explorable insights — Need for interactive exploration across stakeholders (CISO, CTO) 12 -
Explorable Explanations — Showcased multiple models: K-Means, One Class SVM,
Autoencoders — Created models in Python & exported to Tensorflow.js — Integrated with an interactive dashboard (using AngularJS) — Built-in exploration capability on tweaking (and re- running) models Eventual API & implementation based on K-Means model 13 -
Learning on Educate — Discovery by Active Learning is powerful
— Allows the user to build intuition on DL — Not easy! Requires deep thinking on design and usability 16 -
2. Create Rapid Prototyping "The secret of getting started is
breaking your complex overwhelming tasks into smaller manageable tasks, and then starting on the first one" - Mark Twain 17 -
In Real Life: Context Client: Boutique US Software Consulting shop
Scope: Computer vision use-cases for an Automotive client Status: Full stack web & mobile expertise Mandate: Build & extend capabilities for ML&DL Challenge: No experience in python data stack 19 -
What to Prototype — Working with sample data — Needed
to build an image classification model — Show how to integrate that model as API — Create an interactive dashboard to demo to client 20 -
Create Rapid Prototypes — Built CNN models for tra ic
sign detection — Built models in tensorflow.js using pre-built models (transfer learning) — Deployed model as API using Node — Built a simple single-page application for user to upload image and get predictions 21 -
Learning on Create — Enabling software developers to build ML
unlocks huge potential — Easy to prototype without adding technical debt and additional technology costs (eg: Python data stack) — Not easy! How to build custom models. 23 -
3. Visualise Model Visualisation “The most powerful way to gain
insight into a system is by moving between levels of abstraction.” — Bret Victor 24 -
Model Visualisation [0] Visualise the data space [1] Visualise the
predictions in the data space [2] Visualise the errors in model fitting [3] Visualise with di erent model parameters [4] Visualise with di erent input datasets [5] Visualise the entire model space [6] Visualise the entire feature space [7] Visualise the many models together 26 -
ModelVis & ML/DL [0] DATA VIS: the data space [1]
PREDICTION: the predictions in the data space [2] VALIDATION: the errors in model fitting [3] TUNING: with di erent model parameters [4] BOOTSTRAP: with di erent input datasets [5] ENSEMBLE: the entire model space [6] FEATURE ENGG: the entire feature space [7] N-MODELS: the many models together 27 -
ModelVis Key Concept Use visualisation to aid the transition of
implicit knowledge about the data to explicit knowledge in the model. — Iterative, not linear — Up and Down, not lateral — Complementary, not exclusive 28 -
In Real Life: Context Client: Large Consumer Product Company in
India Scope: Network optimisation of secondary freight Status: Regional & fragmented network Mandate: Evaluate consolidation opportunities Challenge: Limited data on new options, Implicit domain knowledge 29 -
Applying Model Visualisation — Engage with Supply Chain domain experts
with limited ML understanding — Need to represent models in the same data visualisation space — Need to showcase visually explorable model options — Need to build interactive exploration for di erent stakeholders (CSO, Managers) 31 -
Learning on Model Vis — Model improvements require engaging with
domain through their lens — Visual analytics principles to model visualisation — Decouple model vis from model building compute & pipelines — Multivariate visualisation is more challenging than larger data! 35 -
4. Intervene Decision Making "All models are wrong, but some
are useful" — George Box 36 -
Context & Use Case Client: Small Regional Movies OTT App
in India Scope: Map movie ROI & acquisition focus Status: Popularity & heuristic-based decisions Mandate: Build RecSys for engagement & licensing Challenge: Limited user insight, no data team 38 -
Challenges — User and movie personas were hard-coded — No
real-time user insight to drive recommendations — Hard to link user behaviour to movie acquisition — Tool should work for business (content acquisition team), not only technology team 39 -
Decision Making Tool — Built multiple RecSys models to drive
app recommendations — Interactive dashboard of user & movie embeddings — Created "What-If" scenario explorer - to link changing user behaviour with movie acquisition Shifted from Heuristic-based to Data-driven decision 40 -
5. Imagine Generative Spaces "I don't know, what I don't
know" — The second level of ignorance 43 -
Generative Spaces — Moving beyond Supervised learning paradigms — Using
generative models to invent meaningful creative operations — Designing and exploring latent spaces (with constraints) — Applications across design, product development, service design 45 -
Examples & Experiments using tfjs — ML for the web:
ml5.js — Music & Art: magenta.js — Machine Learning for Artists: ml4a — AI Google Experiments: AI Collections 46 -
Democratize Deep Learning (DL) 1. Educate: Explorable Explanations 2. Create:
Rapid Prototyping 3. Visualise: Model Visualisation 4. Intervene: Decision Making 5. Imagine: Generative Spaces 47