Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
What is Data Engineering?
Search
Kan Ouivirach
February 27, 2019
Technology
1
110
What is Data Engineering?
Data Engineering helps make the world/organization a better place with "Open Data".
Kan Ouivirach
February 27, 2019
Tweet
Share
More Decks by Kan Ouivirach
See All by Kan Ouivirach
Getting to Know the Data Council
zkan
1
72
Other Decks in Technology
See All in Technology
APIテスト自動化の勘所
yokawasa
7
4.1k
プロダクトチームへのSystem Risk Records導入・運用事例の紹介/Introduction and Case Studies on Implementing and Operating System Risk Records for Product Teams
taddy_919
1
160
Oracle Base Database Service 技術詳細
oracle4engineer
PRO
5
49k
急成長中のWINTICKETにおける品質と開発スピードと向き合ったQA戦略と今後の展望 / winticket-autify
cyberagentdevelopers
PRO
1
160
グローバル展開を見据えたサービスにおける機械翻訳プラクティス / dp-ai-translating
cyberagentdevelopers
PRO
1
150
omakaseしないための.rubocop.yml のつくりかた / How to Build Your .rubocop.yml to Avoid Omakase #kaigionrails
linkers_tech
3
710
30万人が利用するチャットをFirebase Realtime DatabaseからActionCableへ移行する方法
ryosk7
5
330
Shift-from-React-to-Vue
calm1205
3
1.2k
事業者間調整の行間を読む 調整の具体事例
sugiim
0
1.1k
生成AIの強みと弱みを理解して、生成AIがもたらすパワーをプロダクトの価値へ繋げるために実践したこと / advance-ai-generating
cyberagentdevelopers
PRO
1
180
来年もre:Invent2024 に行きたいあなたへ - “集中”と“つながり”で楽しむ -
ny7760
0
450
「 SharePoint 難しい」ってよく聞くけど、そんなに言うなら8歳の息子に試してもらった
taichinakamura
1
580
Featured
See All Featured
Visualization
eitanlees
144
15k
What's new in Ruby 2.0
geeforr
342
31k
4 Signs Your Business is Dying
shpigford
180
21k
Build The Right Thing And Hit Your Dates
maggiecrowley
32
2.4k
Speed Design
sergeychernyshev
24
570
Bootstrapping a Software Product
garrettdimon
PRO
305
110k
Ruby is Unlike a Banana
tanoku
96
11k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
664
120k
BBQ
matthewcrist
85
9.3k
Building an army of robots
kneath
302
42k
Fontdeck: Realign not Redesign
paulrobertlloyd
81
5.2k
Facilitating Awesome Meetings
lara
49
6k
Transcript
K A N O U I V I R A
C H W H AT I S D ATA E N G I N E E R I N G ?
Kan Ouivirach Lead Software Architect
Photo by Frank Vex on Unsplash Companies use DATA to
answer these business questions: • What is our churn rate? • Are we doing better? How can we improve our services? • Any recommendation to generate more revenue from repeated customers?
J O I N T H E C L U
B . .
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R
M A K I N G D ATA S C
I E N C E B E T T E R Data Engineering helps make the world/organization a better place with “Open Data”.
None
https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 T H E R I S E O F
T H E D ATA E N G I N E E R
Use programming languages to ensure clean, reliable, and performative access
to data and databases D ATA E N G I N E E R https://medium.freecodecamp.com/the-rise-of-the-data-engineer-91be18f1e603 The Rise of the Data Engineer
https://www.oreilly.com/ideas/data-engineers-vs-data-scientists
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T https://www.datacamp.com/community/blog/data-scientist-vs-data-engineer
D ATA E N G I N E E R
V S . D ATA S C I E N T I S T Updated Feb 25, 2019
H O W M U C H D O E
S A D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
H O W M U C H D O E
S A S E N I O R D ATA E N G I N E E R / S C I E N T I S T M A K E ? https://www.glassdoor.com/Salaries/index.htm
“In Data Engineering, if an expensive data scientist can’t work,
it is an emergency.” https://blog.datasyndrome.com/rules-for-crash-course-data-engineering-d19bed5f277
• Sanitize while processing data • Decouple components • Access
data at multiple stages • Promote transparency • Automate! http://prismalytics.io/datapipeline/ Data Pipeline
Data scientists doing data engineering?
http://mattturck.com/bigdata2018/
Data engineers doing data science?
https://analyticks.wordpress.com/2016/12/23/machine-learning-cheat-sheet-2/
Data Engineers Data Scientists Photo by rawpixel on Unsplash