Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building an MMORPG - Here be monsters

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →

Building an MMORPG - Here be monsters

In this session, Yan will share some of his experiences of building a successful MMORPG for a social audience and insights into some of the technical challenges that his team has had to overcome along the way.

Yan Cui

May 01, 2014

Transcript

124 slides
  1. • 5,700,000 items bought • 24,000,000 items sold • 280,000

    hours of time potion used • 526,000,000+ km travelled • around the earth 13,000 times...
  2. • Bulk of the game developed by • 1 server

    developer • 2 flash developers • 1 producer and 2 game designers • Several artists
  3. • A lot of states to track on character •

    100s of items in backpack • 100s of quests/achievements • Players like to collect things ;-)
  4. • Many actions requires both states at once • Operations

    are expensive • Best players get the worst latency
  5. • Stateless server approach + Easy to scale up and

    down + Scaling can be dynamic based on load + Every layer can scale independently
  6. • Stateless server approach – Serialization/deserialization is heavy on CPU

    – Heavy load (1:1 read write ratio) on databases – High bandwidth usage – High latency
  7. • Traffic is bursty • Keep state on server during

    bursts = Less serialization & deserialization = Less load on database = Fewer server and database nodes =
  8. • Stateful server approach + 500% improvement on efficiency +

    60% reduction in average latency + Fewer game servers + Fewer database nodes
  9. • Server affinity – Need to ensure server affinity during

    one burst – Need to route calls to affined server
  10. • Server hogging – Session lengths vary greatly – Need

    to avoid players hogging a server – Need to be able to scale down!
  11. • Scaling down – Need to gracefully drain player states

    – Need to be able to stop accepting new players
  12. • Stateful server approach – Complexity + But worthwhile! +

    Others avoid scaling down to reduce complexity
  13. Level Up Quest Progress EXP Item Gold Caught a Gnome

    Quest Complete New Quest Achievement Progress
  14. Level Up Quest Progress EXP Item Gold Caught a Gnome

    Quest Complete New Quest Achievement Progress Achievement Complete
  15. Level Up Quest Progress EXP Item Gold Caught a Gnome

    Quest Complete New Quest Achievement Progress Achievement Complete
  16. Level Up Quest Progress EXP Item Gold Caught a Gnome

    Quest Complete New Quest Achievement Progress Achievement Complete
  17. • 100+ actions available in the game • Most can

    be tied to quests/achievements • Many yields rewards
  18. • Message broker pattern • Something happened, it’s a FACT

    • Caught a Gnome • Got some EXP • ...
  19. Caught Gnome Trapping Queue Levelling Quests Achievements Analytics Partner Reporting

    EXP Item Gold Process Ignore Ignore Ignore Process Ignore
  20. • OK for small number of DU cases • 100+

    actions available in the game
  21. • Need regular content releases • Game is heavily data

    driven + No developer involvement for content changes • Tech team needs to be enablers
  22. • Producers need ability to • Plan changes weeks in

    advance • Quickly see and validate changes before release
  23. • Game designers need ability to • Work on changes

    in isolation • Quickly iterate and test changes in isolation
  24. • QA need ability to • Quickly switch between different

    changes to test • Smoke-test changes in live environment without affecting real players
  25. • Server team needs ability to • Version data changes

    • Rollback/hotfix data changes with no downtime
  26. • Git + Git flow • Servers allow multiple data

    branches and switching between them
  27. • What do our players like to do? • Every

    action in the game is recorded
  28. • 150,000,000+ events recorded in April • Several GBs of

    analytics data a day • Fast growing!
  29. • Monitoring • Not the same as profiling • Should

    be always ON • Difficult to set up in large distributed environment
  30. • Method execution time • e.g. IO, CPU intensive •

    Method execution count • e.g. service entry points • Error count • ...
  31. • Metrics are aggregated at instance level • Metrics gatherer

    has high concurrency requirement • F# agents to the rescue!
  32. Here Be Monsters http://apps.facebook.com/herebemonsters AOP and pseudo real-time monitoring with

    CloudWatch http://bit.ly/11GL3SQ http://bit.ly/10fe98N Building social games with a .Net stack http://bit.ly/ZKtHWb http://bit.ly/10CtyY9 F# Agents http://bit.ly/ZmXuEP http://bit.ly/17kjbam F# Deep Dives http://bit.ly/XZe10E

More Decks by Yan Cui

Other Decks in Programming