Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
What is Data Engineering?
Search
Kan Ouivirach
February 27, 2019
Technology
1
150
What is Data Engineering?
Data Engineering helps make the world/organization a better place with "Open Data".
Kan Ouivirach
February 27, 2019
Tweet
Share
More Decks by Kan Ouivirach
See All by Kan Ouivirach
Getting to Know the Data Council
zkan
1
80
Other Decks in Technology
See All in Technology
DeNA での思い出 / Memories at DeNA
orgachem
PRO
3
1.7k
「AI2027」を紐解く ― AGI・ASI・シンギュラリティ
masayamoriofficial
0
110
Gaze-LLE: Gaze Target Estimation via Large-Scale Learned Encoders
kzykmyzw
0
340
7月のガバクラ利用料が高かったので調べてみた
techniczna
3
580
株式会社ARAV 採用案内
maqui
0
360
Devinを使ったモバイルアプリ開発 / Mobile app development with Devin
yanzm
0
190
【5分でわかる】セーフィー エンジニア向け会社紹介
safie_recruit
0
30k
新卒(ほぼ)専業Kagglerという選択肢
nocchi1
1
2.4k
アジャイルテストで高品質のスプリントレビューを
takesection
0
120
Yahoo!ニュースにおけるソフトウェア開発
lycorptech_jp
PRO
0
380
Claude Code x Androidアプリ 開発
kgmyshin
1
600
進捗
ydah
1
130
Featured
See All Featured
What's in a price? How to price your products and services
michaelherold
246
12k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
358
30k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
Done Done
chrislema
185
16k
GraphQLとの向き合い方2022年版
quramy
49
14k
KATA
mclloyd
32
14k
How GitHub (no longer) Works
holman
315
140k
GitHub's CSS Performance
jonrohan
1031
460k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
Building Adaptive Systems
keathley
43
2.7k
Balancing Empowerment & Direction
lara
2
590
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
656
61k
Transcript
K A N O U I V I R A
C H W H AT I S D ATA E N G I N E E R I N G ?
Kan Ouivirach Lead Software Architect
Photo by Frank Vex on Unsplash Companies use DATA to
answer these business questions: • What is our churn rate? • Are we doing better? How can we improve our services? • Any recommendation to generate more revenue from repeated customers?
J O I N T H E C L U
B . .
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R Data Engineering helps make the world/organization a better place with “Open Data”.
None
https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 T H E R I S E O F
T H E D ATA E N G I N E E R
Use programming languages to ensure clean, reliable, and performative access
to data and databases D ATA E N G I N E E R https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 The Rise of the Data Engineer
https://www.oreilly.com/ideas/data-engineers-vs-data-scientists
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T https://www.datacamp.com/community/blog/data-scientist-vs-data-engineer
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T Updated Feb 25, 2019
H O W M U C H D O E
S A D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
H O W M U C H D O E
S A S E N I O R D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
“In Data Engineering, if an expensive data scientist can’t work,
it is an emergency.” https://blog.datasyndrome.com/rules-for-crash-course-data-engineering-d19bed5f277
• Sanitize while processing data • Decouple components • Access
data at multiple stages • Promote transparency • Automate! http://prismalytics.io/datapipeline/ Data Pipeline
Data scientists doing data engineering?
http://mattturck.com/bigdata2018/
Data engineers doing data science?
https://analyticks.wordpress.com/2016/12/23/machine-learning-cheat-sheet-2/
Data Engineers Data Scientists Photo by rawpixel on Unsplash