Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
What is Data Engineering?
Search
Kan Ouivirach
February 27, 2019
Technology
1
130
What is Data Engineering?
Data Engineering helps make the world/organization a better place with "Open Data".
Kan Ouivirach
February 27, 2019
Tweet
Share
More Decks by Kan Ouivirach
See All by Kan Ouivirach
Getting to Know the Data Council
zkan
1
79
Other Decks in Technology
See All in Technology
コード品質向上で得られる効果と実践的取り組み
ham0215
2
200
Multitenant 23ai の全貌 - 機能・設計・実装・運用からマイクロサービスまで
oracle4engineer
PRO
2
110
ペアプログラミングにQAが加わった!職能を超えたモブプログラミングの事例と学び
tonionagauzzi
1
140
技術的負債を正しく理解し、正しく付き合う #phperkaigi / PHPerKaigi 2025
shogogg
7
1.7k
パスキー導入の課題と ベストプラクティス、今後の展望
ritou
7
1.2k
頻繁リリース × 高品質 = 無理ゲー? いや、できます!/20250306 Shoki Hyo
shift_evolve
0
150
AI・LLM事業部のSREとタスクの自動運転
shinyorke
PRO
0
300
AIエージェントキャッチアップと論文リサーチ
os1ma
6
1.1k
PHPでアクターモデルを活用したSagaパターンの実践法 / php-saga-pattern-with-actor-model
ytake
0
1k
チームの性質によって変わる ADR との向き合い方と、生成 AI 時代のこれから / How to deal with ADR depends on the characteristics of the team
mh4gf
4
330
銀行でDevOpsを進める理由と実践例 / 20250317 Masaki Iwama
shift_evolve
1
100
Amazon EKS Auto ModeでKubernetesの運用をシンプルにする
sshota0809
0
110
Featured
See All Featured
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
46
2.4k
How to Think Like a Performance Engineer
csswizardry
22
1.5k
How GitHub (no longer) Works
holman
314
140k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
160
15k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.1k
Building a Scalable Design System with Sketch
lauravandoore
462
33k
Documentation Writing (for coders)
carmenintech
69
4.7k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
12k
Done Done
chrislema
183
16k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
227
22k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
30
2.3k
Building Flexible Design Systems
yeseniaperezcruz
328
38k
Transcript
K A N O U I V I R A
C H W H AT I S D ATA E N G I N E E R I N G ?
Kan Ouivirach Lead Software Architect
Photo by Frank Vex on Unsplash Companies use DATA to
answer these business questions: • What is our churn rate? • Are we doing better? How can we improve our services? • Any recommendation to generate more revenue from repeated customers?
J O I N T H E C L U
B . .
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R Data Engineering helps make the world/organization a better place with “Open Data”.
None
https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 T H E R I S E O F
T H E D ATA E N G I N E E R
Use programming languages to ensure clean, reliable, and performative access
to data and databases D ATA E N G I N E E R https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 The Rise of the Data Engineer
https://www.oreilly.com/ideas/data-engineers-vs-data-scientists
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T https://www.datacamp.com/community/blog/data-scientist-vs-data-engineer
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T Updated Feb 25, 2019
H O W M U C H D O E
S A D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
H O W M U C H D O E
S A S E N I O R D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
“In Data Engineering, if an expensive data scientist can’t work,
it is an emergency.” https://blog.datasyndrome.com/rules-for-crash-course-data-engineering-d19bed5f277
• Sanitize while processing data • Decouple components • Access
data at multiple stages • Promote transparency • Automate! http://prismalytics.io/datapipeline/ Data Pipeline
Data scientists doing data engineering?
http://mattturck.com/bigdata2018/
Data engineers doing data science?
https://analyticks.wordpress.com/2016/12/23/machine-learning-cheat-sheet-2/
Data Engineers Data Scientists Photo by rawpixel on Unsplash