Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
What is Data Engineering?
Search
Kan Ouivirach
February 27, 2019
Technology
1
150
What is Data Engineering?
Data Engineering helps make the world/organization a better place with "Open Data".
Kan Ouivirach
February 27, 2019
Tweet
Share
More Decks by Kan Ouivirach
See All by Kan Ouivirach
Getting to Know the Data Council
zkan
1
80
Other Decks in Technology
See All in Technology
空間を設計する力を考える / 20251004 Naoki Takahashi
shift_evolve
PRO
3
330
Shirankedo NOCで見えてきたeduroam/OpenRoaming運用ノウハウと課題 - BAKUCHIKU BANBAN #2
marokiki
0
130
AIAgentの限界を超え、 現場を動かすWorkflowAgentの設計と実践
miyatakoji
0
130
o11yで育てる、強い内製開発組織
_awache
3
120
Goに育てられ開発者向けセキュリティ事業を立ち上げた僕が今向き合う、AI × セキュリティの最前線 / Go Conference 2025
flatt_security
0
350
生成AIを活用したZennの取り組み事例
ryosukeigarashi
0
200
リーダーになったら未来を語れるようになろう/Speak the Future
sanogemaru
0
280
GC25 Recap+: Advancing Go Garbage Collection with Green Tea
logica0419
1
400
Flaky Testへの現実解をGoのプロポーザルから考える | Go Conference 2025
upamune
1
420
AI ReadyなData PlatformとしてのAutonomous Databaseアップデート
oracle4engineer
PRO
0
170
KMP の Swift export
kokihirokawa
0
330
それでも私はContextに値を詰めたい | Go Conference 2025 / go conference 2025 fill context
budougumi0617
4
1.2k
Featured
See All Featured
Optimizing for Happiness
mojombo
379
70k
Building Adaptive Systems
keathley
43
2.8k
Fantastic passwords and where to find them - at NoRuKo
philnash
52
3.4k
Mobile First: as difficult as doing things right
swwweet
224
10k
Designing Experiences People Love
moore
142
24k
Art, The Web, and Tiny UX
lynnandtonic
303
21k
How STYLIGHT went responsive
nonsquared
100
5.8k
Measuring & Analyzing Core Web Vitals
bluesmoon
9
610
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
127
53k
How to Think Like a Performance Engineer
csswizardry
27
2k
Thoughts on Productivity
jonyablonski
70
4.9k
The Cult of Friendly URLs
andyhume
79
6.6k
Transcript
K A N O U I V I R A
C H W H AT I S D ATA E N G I N E E R I N G ?
Kan Ouivirach Lead Software Architect
Photo by Frank Vex on Unsplash Companies use DATA to
answer these business questions: • What is our churn rate? • Are we doing better? How can we improve our services? • Any recommendation to generate more revenue from repeated customers?
J O I N T H E C L U
B . .
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R Data Engineering helps make the world/organization a better place with “Open Data”.
None
https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 T H E R I S E O F
T H E D ATA E N G I N E E R
Use programming languages to ensure clean, reliable, and performative access
to data and databases D ATA E N G I N E E R https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 The Rise of the Data Engineer
https://www.oreilly.com/ideas/data-engineers-vs-data-scientists
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T https://www.datacamp.com/community/blog/data-scientist-vs-data-engineer
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T Updated Feb 25, 2019
H O W M U C H D O E
S A D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
H O W M U C H D O E
S A S E N I O R D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
“In Data Engineering, if an expensive data scientist can’t work,
it is an emergency.” https://blog.datasyndrome.com/rules-for-crash-course-data-engineering-d19bed5f277
• Sanitize while processing data • Decouple components • Access
data at multiple stages • Promote transparency • Automate! http://prismalytics.io/datapipeline/ Data Pipeline
Data scientists doing data engineering?
http://mattturck.com/bigdata2018/
Data engineers doing data science?
https://analyticks.wordpress.com/2016/12/23/machine-learning-cheat-sheet-2/
Data Engineers Data Scientists Photo by rawpixel on Unsplash