Armeria: LINE's next generation RPC layer

The slides for my talk at LINE Developer Meetup in Fukuoka on February 16, 2016

Newer slides with up-to-date API and features:
https://speakerdeck.com/trustin/building-asynchronous-microservices-with-armeria

Transcript

1. Armeria : LINE ' s next generation RPC layer line .github .io / armeria 이희승 (@ trustin /イ ·ヒスン ) · kojilin 1
2. Agenda • What and why • Case study : Sticker · theme shop by kojilin • Learning more 2
3. What is Armeria ? • ( A ) synchronous RPC library • Java 8 • HTTP /1 · 2 • Thri • with smooth migration in mind • ... all these becomes ' why ?' as well . 3
4. Why asynchrony ? • Synchronous calls do not scale up . • maxConcurrentReq ≤ maxThreads • Ine fficient CPU usage due to contention and context switches • Fast · Priority requests blocked by others • Cascading fluctuation of thread count • We need real asynchrony to solve this fundamentally . 4
5. Is Armeria really asynchronous ? • Yes ! • maxConcurrentReq ⋙ maxThreads • Can serve · send multiple reqs simultaneously with only a few threads . • Slow requests do not block others . • Bonus : • Asynchronous DNS lookups • Asynchronous connection pool 5
6. Why HTTP /2? • Solves known performance issues with HTTP /1: • Serving multiple requests simultaneously in fewer connections • Reduced tra ffic overhead thanks to binary header compression • Retains flexibility and extensibility of HTTP /1 • Smooth migration path from HTTP /1 • High quality tools - cURL , web browsers , Wireshark 6
7. Is Armeria ' s HTTP /1 · 2 prod . ready ? • Built on Netty 4.1 and its HTTP /1 · 2 implementation • HTTP /1 • TechEmpower web framework benchmark all - time high - ranker • Apple , Twitter , ... • HTTP /2 • Google - gRPC • LINE - authentication , sticker · theme shop , ... 7
8. What is Thri ? 1. Write a . t h r i f t file that contains service definitions . 2. Generate your favorite language code using Thri compiler . 3. Fill the stub . i .e . business logic n a m e s p a c e j a v a c o m . l i n e c o r p . e x a m p l e . t h r i f t s e r v i c e H e l l o S e r v i c e { s t r i n g h e l l o ( 1 : s t r i n g n a m e ) } 0 1 . 0 2 . 0 3 . 0 4 . 8
9. What is Thri ? ( Cont ' d ) c l a s s M y H e l l o S e r v i c e i m p l e m e n t s H e l l o S e r v i c e . A s y n c I f a c e @ O v e r r i d e p u b l i c v o i d h e l l o ( S t r i n g n a m e , A s y n c M e t h o d C a l l b a c k c b ) { c b . o n C o m p l e t e ( " H e l l o , " + n a m e + ' ! ' ) ; } } c l i e n t . h e l l o ( " w o r l d " , n e w A s y n c M e t h o d C a l l b a c k < S t r i n g > ( ) { p u b l i c v o i d o n C o m p l e t e ( S t r i n g n a m e ) { . . . } . . . } ) ; 0 1 . 0 2 . 0 3 . 0 4 . 0 5 . 0 6 . 0 7 . 0 8 . 0 9 . 1 0 . 9
10. Wiring up S e r v e r B u i l d e r b u i l d e r = n e w S e r v e r B u i l d e r ( ) ; b u i l d e r . p o r t ( 8 0 8 0 , S e s s i o n P r o t o c o l . H T T P ) . p o r t ( 8 4 3 3 , S e s s i o n P r o t o c o l . H T T P S ) . s s l C o n t e x t ( . . . ) ; . s e r v i c e A t ( " / t h r i f t / h e l l o " , T h r i f t S e r v i c e . o f ( n e w M y H e l l o S e r v i c e ( ) ) ) ; S e r v e r s e r v e r = b u i l d e r . b u i l d ( ) ; s e r v e r . s t a r t ( ) ; 0 1 . 0 2 . 0 3 . 0 4 . 0 5 . 0 6 . 0 7 . 0 8 . 0 9 . 10
11. Wiring up ( cont ' d ) H e l l o S e r v i c e . I f a c e c l i e n t = C l i e n t s . n e w C l i e n t ( " t b i n a r y + h t t p : / / 1 2 7 . 0 . 0 . 1 : 8 0 8 0 / t h r i f t / h e l l o " , H e l l o S e r v i c e . I f a c e . c l a s s ) ; a s s e r t E q u a l s ( " H e l l o , A r m e r i a ! " , c l i e n t . h e l l o ( " A r m e r i a " ) ) ; 0 1 . 0 2 . 0 3 . 0 4 . 0 5 . 0 6 . 11
12. Wait , but why Thri ? • Suitable for complex & evolving distributed services : • Well - defined service definitions • Well known schema evolution strategy • Interoperability with many languages • Faster · more compact than text - based schemes • Many ways to encode : TBinary , TCompact , T JSON , TText , ... • We ' re using it already successfully at LINE . 12
13. Thri myths • Where ' s my Swagger that produces beautiful docs ? • Where ' s my cURL · telnet that sends a test request ? • I want to serve static files and webapps on the same port · JVM ! 13
14. Automated documentation generator 14
15. Testing from web browser • Share the URL to reproduce a Thri call . 15
16. Wiring up b u i l d e r . s e r v i c e U n d e r ( " / d o c s " , n e w D o c S e r v i c e ( ) ) ; • Add a single service and you ' re done ! • Automatically discovers all available Thri services 16
17. But , how do we migrate legacy apps ? • Servlet / JSP - based REST / RPC services • Serving static resources • Talking to other non - Thri services 17
18. Armeria Ὁ Servlets ! • Powered by a real Servlet engine ( Tomcat ) • Takes only the good part from Tomcat : • Tomcat opens ' zero ' sockets . • Tomcat gets HTTP /2 support for free ! • Let Armeria serve your RPC calls at its best performance while serving your rich · legacy webapps on the same port · JVM ! 18
19. Armeria is protocol - agnostic ! • H t t p F i l e S e r v i c e • Async HTTP /1 · 2 client • HTTP /2 even on a cleartext connection • No context switches when used in an Armeria service • more to come on your demand ! • More services • More protocols 19
20. Wiring up b u i l d e r . s e r v i c e U n d e r ( " / f i l e s " , H t t p F i l e S e r v i c e . f o r C l a s s P a t h ( " / s t a t i c " ) ) . s e r v i c e U n d e r ( " / w e b a p p " , T o m c a t S e r v i c e . f o r F i l e S y s t e m ( " m y . w a r " ) ) ; 0 1 . 0 2 . 0 3 . 0 4 . 20
21. Wiring up ( cont ' d ) S i m p l e H t t p C l i e n t c l i e n t = C l i e n t s . n e w C l i e n t ( " n o n e + h t t p : / / e x a m p l e . c o m / " , S i m p l e H t t p C l i e n t . c l a s s ) ; S i m p l e H t t p R e q u e s t r e q = S i m p l e H t t p R e q u e s t B u i l d e r . f o r G e t ( " / i n d e x . h t m l " ) . h e a d e r ( " A u t h o r i z a t i o n " , " . . . " ) . b u i l d ( ) ; F u t u r e < S i m p l e H t t p R e s p o n s e > f = c l i e n t . e x e c u t e ( r e q ) ; S i m p l e H t t p R e s p o n s e r e s = f . g e t ( ) ; 0 1 . 0 2 . 0 3 . 0 4 . 0 5 . 0 6 . 0 7 . 0 8 . 0 9 . 21
22. Things we did not achieve ( yet ) • Thri usability • Incorporating Thri compiler into build process easily • Code generated by Thri compiler isn ' t beautiful . • Integrated metrics • Incorporating metrics into documentation service • We have IMON support though . • Redis , HBase , Protobuf , RESTful API ... 22
23. 23 Case study : Shop ❤ Armeria
24. Overview 24
25. What we will visit • SearchFe ( Search front - end ) • ShopInternalService ( known as ' Shop server ') • Shop proxy • ... and asynchronification 25
26. Why a search API ? • Exposing Elasticsearch API directly gives too much control to clients • Can ’ t control ranking • Exposes potentially dangerous operations without complicated authentication schemes • JDK compatibility – Elasticsearch clients should use same JVM as cluster • Decouples index concepts ( localized string fields ) with query concepts 26
27. Why Armeria ? • Need an RPC layer , may as well use LINE standard • Thri is good for performance , compatibility , and clearness of API design • Can add features to Armeria and apply to all servers that use it : • Documentation service with request debugging • Graceful shutdown • Health check 27
28. s t r u c t S e a r c h R e q u e s t { / / T h e u s e r ' s i n p u t t e x t q u e r y . I f e m p t y , w i l l p e r f o r m n o t e x t m a t c h i n g - / / t h i s c a n b e u s e d f o r e x a m p l e t o r e t u r n s h o w c a s e s . 1 : o p t i o n a l s t r i n g q u e r y = " " , / / T h e u s e r ' s l o c a l e . 2 : S h o p . L o c a l e l o c a l e = { " l a n g u a g e " : " e n " , " c o u n t r y " : " U S " } , . . . } s t r u c t S e a r c h R e s p o n s e { / / T h e r e s u l t i n g p r o d u c t s f o r t h e q u e r y . 1 : S h o p . P r o d u c t L i s t r e s u l t s , . . . } s e r v i c e S e a r c h F e { S e a r c h R e s p o n s e s e a r c h ( 1 : S e a r c h R e q u e s t r e q u e s t ) } 28
29. @ C o m p o n e n t p u b l i c c l a s s S e a r c h F e H a n d l e r i m p l e m e n t s S e a r c h F e . A s y n c I f a c e { . . . p r i v a t e f i n a l C l i e n t s e a r c h C l i e n t ; . . . @ I n j e c t S e a r c h F e H a n d l e r ( C l i e n t s e a r c h C l i e n t , . . . ) { . . . } / / T h r i f t g e n e r a t e d c o d e d o e s n ' t a d d a t y p e p a r a m e t e r t o A s y n c M e t h o d C a l l b a c k . . . @ S u p p r e s s W a r n i n g s ( " r a w t y p e s " ) @ O v e r r i d e p u b l i c v o i d s e a r c h ( S e a r c h R e q u e s t r e q u e s t , A s y n c M e t h o d C a l l b a c k r e s u l t H a n d l e r ) t h r o w s T E x c e p t i o n { . . . } } 29
30. Shop server ( or ShopInternalService ) • Facebook Ni y → Armeria • Thri binary protocol → HTTP - need to use di fferent port • Added additional port with services configured , and switched clients to Armeria client with new port : • Shop proxy • Talk server 30
31. Just one connection 31
32. Reduced listen drops 32
33. Reduced ' Can ' t get a client ' errors 33
34. Shop proxy • Tomcat HTTP Thri → Armeria • Same protocol , no client changes ! • Started Armeria binary on same machine and redirected tra ffic gradually by reconfiguring nginx 34
35. Shop proxy + IMON • No code needed ( removed tons of monitoring code !) 35
36. Client - side load balancing • Armeria uses a single persistent HTTP /2 connection • Very e fficient • Incompatible with L 4 load balancing • .. which only balances connection open • Implemented simple round - robin client - side load balancer • Will be added to Armeria soon 36
37. Client - side load balancing in action 37
38. Shop proxy asynchronification • Blocking RPCs in a thread pool → fully asynchronous using futures • Can use all machine resources as needed , no more tuning 38
39. Look , less threads ! • Dramatic reduction considering this is O / S thread count . 39
40. Shop server asynchronification • In progress , many APIs ported • Dagger producers • Asynchronous dependency injection • Define methods that return L i s t e n a b l e F u t u r e < T >, define methods that take raw T, and everything is hooked up automatically by the compiler . • See the Devoxx 2014 slides . 40
41. Reduced latency • 3 consecutive blocking calls → 3 simultaneous async calls 41
42. Reduced latency ( cont ' d ) • Multiple nested synchronous MongoDB calls on cache miss → Now running asynchronously in parallel 42
43. Reduced CPU usage 43
44. 44 Learning more
45. Learning more • O fficial web site : line .github .io / armeria • Feel free to ask questions and to discuss ideas via issue tracker . • We need you to make it even better : • API usability issues • Documentation issues • Performance issues • Feature requests 45