Modularity in the Cloud: A PulseOn case study

Running in the cloud introduces many new challenges. How to deal with hot updates and auto-scaling for example? Modularity helps solving many of these challenges. In this presentation you will see how to create modular OSGi applications with RESTful web services, using MongoDB and cloud services such as BlobStores. You will also learn about deploying modular applications using Apache ACE and how to apply auto-scaling. Although the presentation gives real world examples, the session is highly technical and focussed on how developers could build and deploy applications with these kind of requirements.

Examples will be based on an educational system focussed on personalized learning that is currently being implemented at several Dutch secondary schools. It is highly modular and completely runs in the cloud.

Transcript

1. Modularity in the cloud A case study
2. Paul Bakker Architect at Luminis Technologies @pbakker Marcel Offermans Fellow at Luminis Technologies Member at Apache @m4rr5
3. The case Educational system focussed on personalized learning Used in high schools in the Netherlands Expand to other countries in the near future
5. Cross device HTML 5 + JS RESTful web services OSGi services Mongo Apache Felix A m d a t u S3
6. Amdatu: OSGi Cloud components JAX-RS MONGO DB BlobStores Multi-tenancy Search OpenSocial ... Apache Licensed open source project
7. Some numbers 190 bundles in a deployment 120 PulseOn bundles
8. School B Load Deployment Load Balancer PulseOn node PulseOn node PulseOn node Load Balancer PulseOn node PulseOn node PulseOn node School A Mongo Mongo Mongo Mongo Mongo Mongo
9. Availability zones Load Balancer PulseOn node PulseOn node PulseOn node PulseOn node PulseOn node PulseOn node Availability Zone 1 Mongo Mongo Mongo Mongo Mongo Mongo Availability Zone 2
10. Horizontal scalability Horizontal scaling requires stateless nodes HTML5 clients need less server side state Any state should go to some kind of store
11. Auto scaling Considerable higher loads during school hours Enough capacity Without paying for idle servers at night...
12. Load Balancer small node Cluster per school Always use a load balancer because we don’t want downtime during scaling
13. Load Balancer small node Early morning...
14. Load Balancer small node large node large node Early morning...
15. End of the day... Load Balancer small node large node large node
16. End of the day... Load Balancer small node
17. But how do we install our software on a node?
18. Provisioning servers
19. ace screenshot
20. Node AWS Auto Scaling 1. Start Load Balancer Apache ACE 2. register 3. register 4. provision deployment package Provisioning servers
21. ./as-­‐create-­‐launch-­‐config  demo            -­‐-­‐image-­‐id  ami-­‐0ee8e07a            -­‐-­‐instance-­‐type  m1.small            -­‐-­‐region  eu-­‐west-­‐1            -­‐-­‐group  sg-­‐ce1420ba            -­‐-­‐user-­‐data-­‐file  userdata.txt ./as-­‐create-­‐auto-­‐scaling-­‐group  demo          -­‐-­‐launch-­‐configuration  demo          -­‐-­‐min-­‐size  1          -­‐-­‐max-­‐size  1          -­‐-­‐availability-­‐zones  eu-­‐west-­‐1a          -­‐-­‐load-­‐balancers  demo          -­‐-­‐tag  "k=Name,v=demo,p=true"
22. Cluster OSGi Target Load Balancer MongoDB MongoDB MongoDB OSGi Target Infrastructure Management Component Provisioning Cluster Management System Monitoring Automatic Scaling Starts/stops nodes Deploys and updates software components Monitors Quality of Service aspects Adds/rem oves nodes from the cluster Instantiates one or more template based clusters Cluster Template Cluster Template Cluster Template Artifacts, Features & Deployments 5 Cluster Configuration 1 Requests nodes based on provided requirements 2 3 Assigns softw are distributions to targets 4 Gets configured with artifacts for deployments Provides usage data for scaling algorithms 6 7 8 9
23. Tooling and frameworks are now mature enough How does this fit other applications? The architecture fits most modern web applications Works in the cloud (but not a requirement)
24. amdatu.org
25. Cloud provisioning http://ace.apache.org/ Cloud OSGi services http://www.amdatu.org/ Eclipse OSGi plugin http://bndtools.org/ That’s us http://luminis.eu/ There is more...