Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
What is Data Engineering?
Search
Kan Ouivirach
February 27, 2019
Technology
1
110
What is Data Engineering?
Data Engineering helps make the world/organization a better place with "Open Data".
Kan Ouivirach
February 27, 2019
Tweet
Share
More Decks by Kan Ouivirach
See All by Kan Ouivirach
Getting to Know the Data Council
zkan
1
71
Other Decks in Technology
See All in Technology
AI活用したくてもできなかった不動産SaaSの今とこれから
nealle
0
260
ロボットアームを遠隔制御の話 & LLMをつかったIoTの話もしたい
soracom
PRO
1
270
Oracle Exadata Database Service(Dedicated Infrastructure):サービス概要のご紹介
oracle4engineer
PRO
0
9.5k
なにもしてないのにNew Relicのデータ転送量が増えていたときに確認したこと
tk3fftk
2
200
事前準備が肝!AI活用のための業務改革
layerx
PRO
1
240
四国クラウドお遍路 2024 in 高知 エンディング
yukataoka
0
190
Oracle Database Backup Service:サービス概要のご紹介
oracle4engineer
PRO
0
4.1k
どこよりも遅めなWinActor Ver.7.5.0 新機能紹介
tamai_63
0
100
Autonomous Database Serverless 技術詳細 / adb-s_technical_detail_jp
oracle4engineer
PRO
15
40k
AI でアップデートする既存テクノロジーと、クラウドエンジニアの生きる道
soracom
PRO
2
400
Mocking in Rust Applications
taiki45
1
380
効果的なオンコール対応と障害対応
ryuichi1208
5
2.7k
Featured
See All Featured
YesSQL, Process and Tooling at Scale
rocio
167
14k
BBQ
matthewcrist
83
9.1k
[RailsConf 2023] Rails as a piece of cake
palkan
48
4.6k
Speed Design
sergeychernyshev
22
420
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
29
2.6k
How to Ace a Technical Interview
jacobian
275
23k
Scaling GitHub
holman
458
140k
Web Components: a chance to create the future
zenorocha
308
42k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
157
15k
Agile that works and the tools we love
rasmusluckow
327
20k
Why You Should Never Use an ORM
jnunemaker
PRO
53
8.9k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
230
17k
Transcript
K A N O U I V I R A
C H W H AT I S D ATA E N G I N E E R I N G ?
Kan Ouivirach Lead Software Architect
Photo by Frank Vex on Unsplash Companies use DATA to
answer these business questions: • What is our churn rate? • Are we doing better? How can we improve our services? • Any recommendation to generate more revenue from repeated customers?
J O I N T H E C L U
B . .
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R Data Engineering helps make the world/organization a better place with “Open Data”.
None
https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 T H E R I S E O F
T H E D ATA E N G I N E E R
Use programming languages to ensure clean, reliable, and performative access
to data and databases D ATA E N G I N E E R https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 The Rise of the Data Engineer
https://www.oreilly.com/ideas/data-engineers-vs-data-scientists
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T https://www.datacamp.com/community/blog/data-scientist-vs-data-engineer
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T Updated Feb 25, 2019
H O W M U C H D O E
S A D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
H O W M U C H D O E
S A S E N I O R D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
“In Data Engineering, if an expensive data scientist can’t work,
it is an emergency.” https://blog.datasyndrome.com/rules-for-crash-course-data-engineering-d19bed5f277
• Sanitize while processing data • Decouple components • Access
data at multiple stages • Promote transparency • Automate! http://prismalytics.io/datapipeline/ Data Pipeline
Data scientists doing data engineering?
http://mattturck.com/bigdata2018/
Data engineers doing data science?
https://analyticks.wordpress.com/2016/12/23/machine-learning-cheat-sheet-2/
Data Engineers Data Scientists Photo by rawpixel on Unsplash