Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
What is Data Engineering?
Search
Kan Ouivirach
February 27, 2019
Technology
1
130
What is Data Engineering?
Data Engineering helps make the world/organization a better place with "Open Data".
Kan Ouivirach
February 27, 2019
Tweet
Share
More Decks by Kan Ouivirach
See All by Kan Ouivirach
Getting to Know the Data Council
zkan
1
79
Other Decks in Technology
See All in Technology
彩の国で始めよう。おっさんエンジニアから共有したい、当たり前のことを当たり前にする技術
otsuki
0
140
IVRyにおけるNLP活用と NLP2025の関連論文紹介
keisukeosone
0
190
大AI時代で輝くために今こそドメインにディープダイブしよう / Deep Dive into Domain in AI-Agent-Era
yuitosato
1
360
プロダクト開発におけるAI時代の開発生産性
shnjtk
2
230
watsonx.data上のベクトル・データベース Milvusを見てみよう/20250418-milvus-dojo
mayumihirano
0
110
いつも初心者向けの記事に助けられているので得意分野では初心者向けの記事を書きます
toru_kubota
2
310
AWSのマルチアカウント管理 ベストプラクティス最新版 2025 / Multi-Account management on AWS best practice 2025
ohmura
4
280
AIコーディングの最前線 〜活用のコツと課題〜
pharma_x_tech
1
870
AWSLambdaMCPServerを使ってツールとMCPサーバを分離する
tkikuchi
1
3k
フロントエンドも盛り上げたい!フロントエンドCBとAmplifyの軌跡
mkdev10
2
270
От ручной разметки к LLM: как мы создавали облако тегов в Lamoda. Анастасия Ангелова, Data Scientist, Lamoda Tech
lamodatech
0
710
“パスワードレス認証への道" ユーザー認証の変遷とパスキーの関係
ritou
1
570
Featured
See All Featured
The World Runs on Bad Software
bkeepers
PRO
67
11k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
A designer walks into a library…
pauljervisheath
205
24k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
26k
For a Future-Friendly Web
brad_frost
176
9.7k
Thoughts on Productivity
jonyablonski
69
4.6k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.5k
How GitHub (no longer) Works
holman
314
140k
Fashionably flexible responsive web design (full day workshop)
malarkey
407
66k
Site-Speed That Sticks
csswizardry
5
490
GraphQLとの向き合い方2022年版
quramy
46
14k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
34
2.9k
Transcript
K A N O U I V I R A
C H W H AT I S D ATA E N G I N E E R I N G ?
Kan Ouivirach Lead Software Architect
Photo by Frank Vex on Unsplash Companies use DATA to
answer these business questions: • What is our churn rate? • Are we doing better? How can we improve our services? • Any recommendation to generate more revenue from repeated customers?
J O I N T H E C L U
B . .
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R Data Engineering helps make the world/organization a better place with “Open Data”.
None
https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 T H E R I S E O F
T H E D ATA E N G I N E E R
Use programming languages to ensure clean, reliable, and performative access
to data and databases D ATA E N G I N E E R https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 The Rise of the Data Engineer
https://www.oreilly.com/ideas/data-engineers-vs-data-scientists
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T https://www.datacamp.com/community/blog/data-scientist-vs-data-engineer
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T Updated Feb 25, 2019
H O W M U C H D O E
S A D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
H O W M U C H D O E
S A S E N I O R D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
“In Data Engineering, if an expensive data scientist can’t work,
it is an emergency.” https://blog.datasyndrome.com/rules-for-crash-course-data-engineering-d19bed5f277
• Sanitize while processing data • Decouple components • Access
data at multiple stages • Promote transparency • Automate! http://prismalytics.io/datapipeline/ Data Pipeline
Data scientists doing data engineering?
http://mattturck.com/bigdata2018/
Data engineers doing data science?
https://analyticks.wordpress.com/2016/12/23/machine-learning-cheat-sheet-2/
Data Engineers Data Scientists Photo by rawpixel on Unsplash