Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
What is Data Engineering?
Search
Kan Ouivirach
February 27, 2019
Technology
1
150
What is Data Engineering?
Data Engineering helps make the world/organization a better place with "Open Data".
Kan Ouivirach
February 27, 2019
Tweet
Share
More Decks by Kan Ouivirach
See All by Kan Ouivirach
Getting to Know the Data Council
zkan
1
81
Other Decks in Technology
See All in Technology
現場の壁を乗り越えて、 「計装注入」が拓く オブザーバビリティ / Beyond the Field Barriers: Instrumentation Injection and the Future of Observability
aoto
PRO
1
610
知覚とデザイン
rinchoku
1
590
20251024_TROCCO/COMETAアップデート紹介といくつかデモもやります!_#p_UG 東京:データ活用が進む組織の作り方
soysoysoyb
0
110
組織全員で向き合うAI Readyなデータ利活用
gappy50
1
690
AI-Readyを目指した非構造化データのメダリオンアーキテクチャ
r_miura
1
330
IoTLT@ストラタシスジャパン_20251021
norioikedo
0
140
re:Inventに行くまでにやっておきたいこと
nagisa53
0
300
AIとともに歩んでいくデザイナーの役割の変化
lycorptech_jp
PRO
0
890
[re:Inent2025事前勉強会(有志で開催)] re:Inventで見つけた人生をちょっと変えるコツ
sh_fk2
0
180
パフォーマンスチューニングのために普段からできること/Performance Tuning: Daily Practices
fujiwara3
2
130
AI時代におけるデータの重要性 ~データマネジメントの第一歩~
ryoichi_ota
0
720
dbtとAIエージェントを組み合わせて見えたデータ調査の新しい形
10xinc
2
520
Featured
See All Featured
The Cult of Friendly URLs
andyhume
79
6.6k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
230
22k
What's in a price? How to price your products and services
michaelherold
246
12k
A Modern Web Designer's Workflow
chriscoyier
697
190k
jQuery: Nuts, Bolts and Bling
dougneiner
65
7.9k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
34
2.3k
Navigating Team Friction
lara
190
15k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
27k
4 Signs Your Business is Dying
shpigford
185
22k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
46
2.5k
Raft: Consensus for Rubyists
vanstee
140
7.2k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
34
2.5k
Transcript
K A N O U I V I R A
C H W H AT I S D ATA E N G I N E E R I N G ?
Kan Ouivirach Lead Software Architect
Photo by Frank Vex on Unsplash Companies use DATA to
answer these business questions: • What is our churn rate? • Are we doing better? How can we improve our services? • Any recommendation to generate more revenue from repeated customers?
J O I N T H E C L U
B . .
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R Data Engineering helps make the world/organization a better place with “Open Data”.
None
https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 T H E R I S E O F
T H E D ATA E N G I N E E R
Use programming languages to ensure clean, reliable, and performative access
to data and databases D ATA E N G I N E E R https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 The Rise of the Data Engineer
https://www.oreilly.com/ideas/data-engineers-vs-data-scientists
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T https://www.datacamp.com/community/blog/data-scientist-vs-data-engineer
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T Updated Feb 25, 2019
H O W M U C H D O E
S A D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
H O W M U C H D O E
S A S E N I O R D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
“In Data Engineering, if an expensive data scientist can’t work,
it is an emergency.” https://blog.datasyndrome.com/rules-for-crash-course-data-engineering-d19bed5f277
• Sanitize while processing data • Decouple components • Access
data at multiple stages • Promote transparency • Automate! http://prismalytics.io/datapipeline/ Data Pipeline
Data scientists doing data engineering?
http://mattturck.com/bigdata2018/
Data engineers doing data science?
https://analyticks.wordpress.com/2016/12/23/machine-learning-cheat-sheet-2/
Data Engineers Data Scientists Photo by rawpixel on Unsplash