Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Running your Dockerized application(s) on AWS ...

Running your Dockerized application(s) on AWS EC2 Container Service

Marco Pas

June 02, 2017
Tweet

More Decks by Marco Pas

Other Decks in Technology

Transcript

  1. Running your Dockerized application(s) on AWS EC2 Container Service Marco

    Pas Philips Lighting Software geek, hands on Developer/Architect/DevOps Engineer @marcopas
  2. Some stuff about me... • Mostly doing cloud related stuff

    ◦ Java, Groovy, Scala, Spring Boot, IOT, AWS, Terraform, Infrastructure • Enjoying the good things • Chef leuke dingen doen == “trying out cool and new stuff” • Currently involved in a big IOT project • Wannabe chef, movie & Netflix addict
  3. Something that runs into production • Docker • Security •

    Service Discovery • Logging & Monitoring • Rolling Deployments • Networking • Supervision • Container hosting • Docker Development Production Learning cliff
  4. Agenda • Containers • Creating a Container using Spring Boot

    • Container Services • Amazon EC2 Container Service (ECS) ◦ Pushing and Pulling containers ◦ Deploying containers ◦ Scaling your containers ◦ Service Discovery ◦ Logging ◦ Monitoring
  5. • OS Virtualization • Process Isolation • Automation • Images

    What are containers Portable Flexible Fast Efficient
  6. Creating a Docker Image # Dockerfile ~ example FROM alpine:latest

    ADD HelloWorld.class HelloWorld.class RUN apk --update add openjdk8-jre ENTRYPOINT ["java", "-Djava.security.egd=file:/dev/./urandom", "HelloWorld"] docker build -t <your imagename>:<tag> .
  7. // file: DemoApplication.java package springboot.docker.helloworld; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import

    org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RestController; @RestController @SpringBootApplication public class DemoApplication { public static void main(String[] args) { SpringApplication.run(DemoApplication.class, args); } @RequestMapping("/") String home() { return "Hello World!"; → say hello :) } }
  8. // file: build.gradle ~ some code intentionally removed dependencies {

    compile('org.springframework.boot:spring-boot-starter-web') compile('org.springframework.boot:spring-boot-starter-actuator') → add spring boot actuator testCompile('org.springframework.boot:spring-boot-starter-test') } String dockerImageName = "spring-boot-docker-helloworld" → set the image name task buildDockerImage(type:Exec) { → task to create an image group = 'docker' description = 'Build a docker image' commandLine 'docker', 'build', '-f', 'build/docker/Dockerfile', '-t', "${dockerImageName}", 'build/docker' doFirst { println ">> Creating image: ${dockerImageName}" // some code intentionally removed } }
  9. // file: build.gradle ~ some code intentionally removed doFirst {

    println ">> Creating image: ${dockerImageName}" copy { // copy files to build location, Dockerfile - Jar file } copy { // process Dockerfile to replace labels (Dockerfile label: @name@, @version@, @build-date@, … // copy files to build location, Dockerfile - Jar file from('src/main/docker/') { include 'Dockerfile' filter(ReplaceTokens, tokens: [ 'version': version, 'build-date': new Date().format("yyyy-MM-dd'T'HH:mm:ss'Z'", TimeZone.getTimeZone("UTC")), 'git-branch': gitBranch(), 'git-commit': gitCommitHash() ]) } file("build/docker/app/${jar.archiveName}").renameTo("build/docker/app/application.jar") }
  10. // file: Dockerfile example ~ some code intentionally removed FROM

    java:8u66-jdk LABEL com.acme.build-date="@build-date@" → provide data to the Dockerfile EXPOSE 8080 → expose port 8080, so the host can map this port # Create app that holds the application.jar file RUN mkdir -p /app → do some housekeeping, creating directories WORKDIR /app COPY /app/application.jar application.jar → copy the application.jar file into the container COPY /app/docker-entrypoint.sh docker-entrypoint.sh → copy startup script into the container # Set file permissions RUN chmod +x docker-entrypoint.sh → make the shel script executable # Set start script as default command CMD ["./docker-entrypoint.sh"] → execute the startup script when we start the container
  11. // file: build.gradle project.ext.dockerRegistry = System.env.DOCKER_REGISTRY → get the docker

    registry from environment String dockerImageName = "spring-boot-docker-helloworld" → set the image name task pushDockerImage(type: Exec) { group = 'docker' description = 'Push a docker image' commandLine 'docker', 'push', "${project.ext.dockerRegistry}/${dockerImageName}" doFirst { println ">> Checking dockerRepository" if (!project.ext.dockerRegistry) { throw new GradleException("Unable to push image, please provide correct 'dockerRegistry'") } println ">> Pushing image: ${dockerImageName}" } }
  12. Running the image using Docker Compose // file: docker-compose.yml version:

    '2' services: springboot-demo: → name if container image: spring-boot-docker-helloworld:latest → the image that is going to be used ports: - "8080:8080” → port mapping 8080 host -> 8080 container
  13. But wait… we have images now how do we run

    our containers? We need some help! Container Storage, Scheduling & Orchestration
  14. Container Services • Most used Container Services ◦ Amazon ECS

    ◦ Kubernetes by Google ◦ Docker Swarm ◦ Hashicorp Nomad ◦ Azure Container Service All have the some focus: Run your Services / Containers
  15. Container Services • Storage • Clustering support • Control &

    Monitoring • Scale up/down • Scheduling & Orchestration ◦ Flexible Container placement
  16. Placement Strategies • Strategy name ◦ node selected • Spread

    ◦ has the fewest containers, disregarding their states • Binpack ◦ most packed (i.e. has the minimum amount of free CPU/RAM) • Random ◦ chosen randomly
  17. • Amazon’s version of a Docker Registry • Registry contains

    Repositories ◦ unique namespace • Logins generated on demand with limited session length • Images: ◦ can be shared with AWS accounts ◦ at rest are encrypted and stored in S3 ◦ transmitted over HTTPS Container Registry
  18. Container Service Detail EC2 Container Instance EC2 instance with Docker

    & the ECS Agent installed ECS Agent Allows EC2 container instances to connect to a cluster
  19. Demo Description • Create the infrastructure • Deploy “HelloWorld” container

    to an ECS Container Instance • Make the endpoint publicly available via ALB • Scale the container instances
  20. Terraform • Provision resources ◦ Compute / Storage / Network

    • Manage resource lifecycles • Manage different resource providers (AWS, Google, Azure, …) • Automate deployments and configurations Infrastructure as Code
  21. // file: main.tf ~ some code intentionally removed module "vpc"

    { source = "github.com/terraform-community-modules/tf_aws_vpc" name = "my-vpc" cidr = "10.0.0.0/16" public_subnets = ["10.0.1.0/24", "10.0.2.0/24"] private_subnets = ["10.0.101.0/24", "10.0.102.0/24"] } module "bastion" { ... } module "ecs-cluster" { source = "./ecs-cluster" cluster_name = "demo" vpc_id = "${module.vpc.vpc_id}" subnet_ids = "${module.vpc.private_subnets}" }
  22. All fine, we have the infrastructure Now get some apps

    deployed :) Deployment of a Dockerized app on ECS
  23. Describing your Docker deployment Describes one or more Docker Containers

    that form your application (blueprint) Runs and maintains a desired number of tasks from a specified task definition Running container with the settings defined in the Task Definition
  24. Example: Task Definition 45 { "family": "webserver", → family of

    containers "containerDefinitions": [{ "name": "web", → name of the container "image": "nginx", → the image used for the container "cpu": 99, → cpu + memory credits "memory": 100, "portMappings": [{ → port mappings (expose port 80 in container to port 80 on host) "containerPort": 80, "hostPort": 80 }], "environment": [{ → environment variables, used in the container "name": "MYSQL_ROOT_PASSWORD", "value": "password" }] }] } Can you spot the problem?
  25. Run a task/service on ECS Container Service • AWS Console

    ◦ Use the AWS console and use the UI • Manual ◦ Using the AWS CLI / ECS CLI • Automated ◦ Using Cloudwatch or Terraform
  26. Demo Description • Create the infrastructure • Deploy “HelloWorld” container

    to an ECS Container Instance • Make the endpoint publicly available via ALB • Scale the container instances
  27. // file: main.tf ~ some code intentionally removed module "vpc"

    { ... } module "bastion" { ... } module "ecs-cluster" { ... } module "helloworld-service" { source = "./helloworld-service" environment = "test-env" vpc_id = "${module.vpc.vpc_id}" ecs_cluster_id = "${module.ecs-cluster.ecs_cluster_id}" docker_repository = "163079528612.dkr.ecr.us-east-1.amazonaws.com" public_subnet_ids = "${module.vpc.public_subnets}" iam_role = "${module.ecs-cluster.ecs_aws_iam_role_name}" desired_count = 1 }
  28. // file: task-definition.tpl [ { "name": "helloworld-service", "essential": true, "image":

    "${docker_repository}/springboot-docker-helloworld:${version}", "memoryReservation": 256, "portMappings": [ { "ContainerPort": 8080 } ] } ]
  29. Autoscaling your containers • Scaling is based upon metrics →

    Application Autoscaling ◦ Metrics on ECS/Service ▪ cpu load, memory usage, io, … • CloudWatch Alarm ◦ cpu > 80% for 1 minute ◦ cpu < 50% for 1 minute • Scaling Policy → “ChangeInCapacity” ◦ up +1 instance ◦ down -1 instance
  30. Demo Description • Create the infrastructure • Deploy “HelloWorld” container

    to an ECS Container Instance • Make the endpoint publicly available via ALB • Scale the container instances
  31. Configuring the Log Driver 62 { "family": "webserver", → family

    of containers "containerDefinitions": [{ "name": "web", → name of the container "image": "nginx", → the image used for the container // some intentionally omitted "logConfiguration": { → log configuration "logDriver": "awslogs", → to be used logdriver "options": { → logdriver options "awslogs-group": "awslogs-nginx", "awslogs-region": "ap-northeast-1", "awslogs-stream-prefix": "awslogs-example" } } }] }
  32. Recap • Running Docker containers on ECS is not hard

    ◦ Build your Dockerized Spring Boot applications and push them to ECS ◦ ECS Cluster with EC2 instances • Use a “Infrastructure as Code” approach to keep a grasp on what needs to be deployed • Do not forget about Logging and Monitoring these steps are important ◦ use CloudWatch or other monitoring tools to keep an eye on your infrastructure • Service Discovery using DNS or Consul