Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
What is Data Engineering?
Search
Kan Ouivirach
February 27, 2019
Technology
1
140
What is Data Engineering?
Data Engineering helps make the world/organization a better place with "Open Data".
Kan Ouivirach
February 27, 2019
Tweet
Share
More Decks by Kan Ouivirach
See All by Kan Ouivirach
Getting to Know the Data Council
zkan
1
80
Other Decks in Technology
See All in Technology
データモデリング通り #2オンライン勉強会 ~方法論の話をしよう~
datayokocho
0
100
Google Cloud で学ぶデータエンジニアリング入門 2025年版 #GoogleCloudNext / 20250805
kazaneya
PRO
12
2.8k
Kiroから考える AIコーディングツールの潮流
s4yuba
4
660
✨敗北解法コレクション✨〜Expertだった頃に足りなかった知識と技術〜
nanachi
1
480
生成AI導入の効果を最大化する データ活用戦略
ham0215
0
110
마라톤 끝의 단거리 스퍼트: 2025년의 AI
inureyes
PRO
1
680
Foundation Model × VisionKit で実現するローカル OCR
sansantech
PRO
0
300
LLMでAI-OCR、実際どうなの? / llm_ai_ocr_layerx_bet_ai_day_lt
sbrf248
0
430
S3 Glacier のデータを Athena からクエリしようとしたらどうなるのか/try-to-query-s3-glacier-from-athena
emiki
0
180
LTに影響を受けてテンプレリポジトリを作った話
hol1kgmg
0
290
【OptimizationNight】数理最適化のラストワンマイルとしてのUIUX
brainpadpr
1
250
2025新卒研修・HTML/CSS #弁護士ドットコム
bengo4com
3
13k
Featured
See All Featured
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
8
870
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.8k
Agile that works and the tools we love
rasmusluckow
329
21k
Art, The Web, and Tiny UX
lynnandtonic
301
21k
Designing Experiences People Love
moore
142
24k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
29
9.6k
Product Roadmaps are Hard
iamctodd
PRO
54
11k
The Pragmatic Product Professional
lauravandoore
36
6.8k
Imperfection Machines: The Place of Print at Facebook
scottboms
267
13k
How to train your dragon (web standard)
notwaldorf
96
6.1k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
161
15k
BBQ
matthewcrist
89
9.8k
Transcript
K A N O U I V I R A
C H W H AT I S D ATA E N G I N E E R I N G ?
Kan Ouivirach Lead Software Architect
Photo by Frank Vex on Unsplash Companies use DATA to
answer these business questions: • What is our churn rate? • Are we doing better? How can we improve our services? • Any recommendation to generate more revenue from repeated customers?
J O I N T H E C L U
B . .
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R Data Engineering helps make the world/organization a better place with “Open Data”.
None
https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 T H E R I S E O F
T H E D ATA E N G I N E E R
Use programming languages to ensure clean, reliable, and performative access
to data and databases D ATA E N G I N E E R https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 The Rise of the Data Engineer
https://www.oreilly.com/ideas/data-engineers-vs-data-scientists
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T https://www.datacamp.com/community/blog/data-scientist-vs-data-engineer
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T Updated Feb 25, 2019
H O W M U C H D O E
S A D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
H O W M U C H D O E
S A S E N I O R D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
“In Data Engineering, if an expensive data scientist can’t work,
it is an emergency.” https://blog.datasyndrome.com/rules-for-crash-course-data-engineering-d19bed5f277
• Sanitize while processing data • Decouple components • Access
data at multiple stages • Promote transparency • Automate! http://prismalytics.io/datapipeline/ Data Pipeline
Data scientists doing data engineering?
http://mattturck.com/bigdata2018/
Data engineers doing data science?
https://analyticks.wordpress.com/2016/12/23/machine-learning-cheat-sheet-2/
Data Engineers Data Scientists Photo by rawpixel on Unsplash