Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
The Design Philosophy of Kuroko2
Search
Naoto Takai
January 18, 2017
Programming
3
2.1k
The Design Philosophy of Kuroko2
Internal lightning talks in Cookpad
Naoto Takai
January 18, 2017
Tweet
Share
More Decks by Naoto Takai
See All by Naoto Takai
Building the Perfect Custom Keyboard
takai
0
420
キースイッチ潤滑入門 / A Brief Introduction to Lube MX Switches
takai
2
760
みなさん、分かっていますか / What is a difference?
takai
1
850
Agile and DevOps with Ruby
takai
2
1k
Software Quality and Testing #1
takai
5
620
Microservices in Action
takai
37
5.2k
Ruby App on Netflix Microservices Stack
takai
6
1k
How We Use Jenkins?
takai
21
8.2k
COOKPAD and Test Automation
takai
19
2.7k
Other Decks in Programming
See All in Programming
ぼっちを避けて楽しむためのアノテコノテ / Various Tips and Tricks to Avoid Loneliness and Have Fun
nrslib
3
1.7k
しくじり先生 Image Matching Challenge 2024 編
goosehaaan
0
810
Xcode 16のPreviewModifierと@Previewableを活用した効率的なプレビュー方法の考察
ojun9
2
160
初心者がおさえておきたいAWS CDKのベストプラクティス 2024
konokenj
15
7.3k
Introduction to GitOps
hwchiu
0
110
CSC307 Lecture 10
javiergs
PRO
0
310
SRE チーム立ち上げ前に考えたこと・取り組んだこと / Considerations and Preparations Before Establishing an SRE Team
mackey0225
3
320
Clean Architecture by TypeScript & NestJS
ryounasso
0
150
CSC307 Lecture 07
javiergs
PRO
0
220
Prompt FlowによるLLMアプリケーション開発
yuto2000
1
1k
CSC307 Lecture 08
javiergs
PRO
0
330
Polarsの成長: v0.14からv1.0までの変遷と今後の展望
zerebom
1
350
Featured
See All Featured
ReactJS: Keep Simple. Everything can be a component!
pedronauck
662
120k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
26
2.1k
Imperfection Machines: The Place of Print at Facebook
scottboms
262
13k
The MySQL Ecosystem @ GitHub 2015
samlambert
248
12k
What's new in Ruby 2.0
geeforr
338
31k
Debugging Ruby Performance
tmm1
71
11k
Agile that works and the tools we love
rasmusluckow
325
20k
Reflections from 52 weeks, 52 projects
jeffersonlam
346
19k
How GitHub Uses GitHub to Build GitHub
holman
471
290k
Fontdeck: Realign not Redesign
paulrobertlloyd
79
5.1k
What’s in a name? Adding method to the madness
productmarketing
PRO
21
2.9k
How to Think Like a Performance Engineer
csswizardry
4
590
Transcript
The Design Philosophy of
• Web-based (not “enterprisy”) • Embedded workflow engine (ruote). •
Distributed architecture • with on demand EC2 provisioning. • communicates with message queue. Kuroko1
kuroko-console RabbitMQ kuoko-worker workflow engine worker process EC2 API launch
exec command web interface web workflow Kuroko1 Overview
kuroko-console RabbitMQ kuoko-worker workflow engine worker process EC2 API launch
exec command web interface web workflow Kuroko1 Pitfalls × × × × × × × × × ×
• It’s really hard to isolate the problem in a
distributed environment. • “Programmable” means programming is always needed. • Nothing is reliable except MySQL. Lessons from Kuroko1
There is data inconsistency between the actual state and the
stored state. Lessons from Kuroko1 (cont.)
State management is a key factor.
• Monolithic is the best. • Data oriented, not process
oriented. • Monitor as possible. Kuroko2
Monolithic Architecture kuroko-console kuoko-worker workflow engine worker process exec command
web interface DB
Kuroko2 Job State job_definitions job_instances tokens noop noop noop noop
/0-noop A job definition A launched job A job state
it do engine.process(token) expect(token.path).to eq '/0-noop' engine.process(token) expect(token.path).to eq '/1-noop'
engine.process(token) expect(token).to be_finished expect(Kuroko2::Token.all.count).to eq 0 end
Kuroko2 Execution State kuroko-worker kuroko-console executions workflow-processor command-executor exec command
loop do execution = Execution.unstarted.take() execution.touch(:started_at) execute_shell(execution.shell) execution.update(exit_status: exit_status, finished_at:
Time.current) execution.finish() end
kuroko-worker kuroko-console executions workflow-processor command-executor exec command Kuroko2 Pitfalls ×
× ×
Kuroko2 Worker Processes command-executor kill process monitor process shell process
exec command
def check_assignment_delay(exec_no_pid) if exec_no_pid.started_at < 1.minutes.ago Kuroko2::Notifications .not_assigned(exec_no_pid, @hostname).deliver_now exec_no_pid.touch(:mailed_at)
end end
def check_process_absence(execution) begin Process.kill(0, execution.pid) rescue Errno::EPERM true rescue Errno::ESRCH
if Execution.exists?(execution.id) notify_process_absence(execution) end end end
• /v1/stats/waiting_execution • /v1/stats/instance Kuroko2 Monitoring API
• State management is a key factor: • monolithic is
the best. • data oriented, not process oriented. • monitor as possible. Kuroko2
is open source product. We are waiting your contributions!