Share
Embed
Cloud Run for Kubernetes users
Ahmet Alp Balkan
March 12, 2021
Transcript
42 slides
-
about me • once loved/advocated for Kubernetes • now I
think it’s an overkill for many you may know me from: • kubectx / kubens • Krew (kubectl plugin manager) • Kubernetes NetworkPolicy recipes -
If you know Knative, Cloud Run is a hosted Knative
Serving API GKE Standard GKE Autopilot Google infra (Borg) Knative open source Kubernetes / GKE Knative API UI CLI YAML Cloud Run Anthos GKE-on-prem -
$ gcloud run deploy \ /-image=gcr.io/my-image:latest \ /-cpu=2 \ /-memory=512Mi
\ /-allow-unauthenticated \ /-set-env-vars FOO=BAR \ /-max-instances=100 \ /-concurrency=5 \ [//.] imperative deployments -
Very similar to Kubernetes Deployment+Service objects • Knative Service CRD
◦ K8s Service+Deployment merged. • same PodSpec as Kubernetes $ gcloud run services replace <MANIFEST> declarative deployments -
• designed for stateless applications that are serving HTTP-based protocols
• great for: ◦ microservices ◦ APIs ◦ frontends • not for: ◦ databases ◦ custom binary protocols ◦ game servers ◦ run-to-completion jobs what does it run? -
container contract • Linux executables (x86-64 ABI) • Deploy OCI
container images No Windows or ARM support. -
CPU allocated only during requests (no background threads) • very
low/no cpu otherwise • ok for garbage collection etc • not enough to push out metrics/traces CPU OFF CPU ON CPU OFF _ -
container lifecycle • new containers are started on ◦ first-time
deployment ◦ scale-up ◦ failed health checks • we wait until the container is listening on $PORT ◦ now ready to receive traffic • CPU is allocated only during requests • unused container gets terminated with SIGTERM -
networking: protocols works on Cloud Run: • HTTP/1 • HTTP/2
(h2c) • WebSockets • gRPC no arbitrary/binary protocols -
networking: TLS • Cloud Run provisions TLS certificates ◦ both
for *.run.app & custom domain • forced HTTP→HTTPS redirects • your app should prepare unencrypted responses ◦ encryption added later by the infra -
• built-in L7 load balancing for a service • routes
traffic between container instances • no unintentional sticky sessions (per-request) serving: load balancing -
• can split traffic between revisions (90% → v1, 10%
→ v2) • used for canary deployments, progressive rollouts • can have a domain for each revision: ◦ v1//-myservice.[//.].run.app ◦ commit-aeab13f//-myservice.[//.].run.app ◦ latest//-myservice.[//.].run.app serving: traffic splitting/canary -
break the glass & create Google Cloud L7 HTTPS LB.
◦ enable CDN ◦ BYO TLS certs ◦ multi-region load balancing/failover ◦ IAP (Identity-Aware Proxy a.k.a. BeyondCorp) serving: advanced load balancing -
serving: concurrency limits • typically functions/lambda: 1 req/container • Cloud
Run: supports concurrents requests (up to 250) why? • helps you limit your load • informs autoscaling decisions • overlapping requests are not double-charged -
• based on in-flight HTTP requests (no CPU/memory targets like
Kubernetes HPA) • concurrency=5 + inflight_requests=15 → 3 containers • you can limit max container instances (prevent unbounded spending) • no guarantee how long idle containers will be around autoscaling -
serving: request/response limits • max request timeout: 60 minutes •
no maximum request/response size (as much as you can recv or send) • requests or responses are not buffered (bi-directional streaming) -
pricing pay only "during" requests (for CPU/memory) • overlapping requests
are not double-charged networking egress costs + request count costs exist. generous free tier (cloud.google.com/free) FREE CHARGED FREE _ -
• cold starts exist! (inactive services scaling up 0/>1, new
container instance waking up to handle traffic) • minimize cold starts by using "minimum instances". ◦ these are kept "warm" ◦ costs ~10% of the active (serving) time cold start -
• gVisor sandbox ◦ Linux syscall emulation in userspace ◦
used internally at Google to run untrusted workloads on Borg ◦ low-level Linux APIs might not work: ▪ iptables, some ioctls, … • might change soon kernel -
• files written to local disk: ◦ are not persisted
◦ counts towards available memory of container instance • (currently) no support for mounting external volumes ◦ such as Cloud Filestore (NAS) or Cloud Storage filesystem/volumes -
• no current support • upcoming support for Google Cloud
Secret Manager ◦ it will look like Kubernetes secret volume/volumeMounts secrets/configmaps -
• individual container instances are not exposed to you (to
keep things simple) • auto-healing properties ◦ faulty containers (20 consecutive 5xx) are replaced ◦ crash-looping containers are replaced container instance fleet -
1. Know the full URL (xxx.run.app) 2. Get a JWT
token 3. Add as “Authorization” header to outgoing request Kubernetes or K8s+Istio experience shines here (for now). service-to-service calls -
• Sidecars / multiple containers per Pod ◦ Knative supports
this • Init containers • Health probes ◦ Cloud Run has primitive built-in startup health check ◦ What do periodic probes mean when you don't have CPU allocated? • mTLS/cert auth is still not possible Unsupported (for now) from Kubernetes -
• Cloud Run: ◦ solid highly-scalable serving layer ◦ scales
real-time, based on in-flight requests ◦ designed with spiky workloads in mind ◦ request buffering during autoscaling • Kubernetes: ◦ we all typically over-provision for unpredicted load ◦ CPU/memory metrics are a side-effect of load, and often collected w/ a delay ◦ without a buffering meat shield, excessive load will crash Pods. Kubernetes+Cloud Run=? -
Kubernetes /> Cloud Run: 1. use GCP workload identity 2.
get a JWT token from metadata service + add it to outbound request 3. set IAM permission on Cloud Run 4. Cloud Run serving layer automatically checks the JWT + IAM permissions of it. Cloud Run /> Kubernetes: 1. connect to the same VPC 2. provision an internal LB (or use VPC private IPs) 3. get a JWT token from metadata service + add it to outbound request 4. verify the JWT in your app coming from Cloud Run + DIY ACLs. Hybrid? -
apiVersion: apps/v1 kind: Deployment metadata: name: hello-web spec: replicas: 1
selector: matchLabels: app: hello tier: web template: metadata: labels: app: hello tier: web spec: containers: - name: main image: gcr.io/google-samples/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi Kubernetes Deployment Kubernetes Service apiVersion: v1 kind: Service metadata: name: hello-web spec: type: LoadBalancer selector: app: hello tier: web ports: - port: 80 targetPort: 8080 -
apiVersion: apps/v1 kind: Deployment metadata: name: hello-web spec: replicas: 1
selector: matchLabels: app: hello tier: web template: metadata: labels: app: hello tier: web spec: containers: - name: main image: gcr.io/google-samples/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi Kubernetes Deployment Kubernetes Service apiVersion: v1 kind: Service metadata: name: hello-web spec: type: LoadBalancer selector: app: hello tier: web ports: - port: 80 targetPort: 8080 no need, Knative will give us a $PORT -
apiVersion: apps/v1 kind: Deployment metadata: name: hello-web spec: replicas: 1
selector: matchLabels: app: hello tier: web template: metadata: labels: app: hello tier: web spec: containers: - name: main image: gcr.io/google-samples/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi Kubernetes Deployment Kubernetes Service apiVersion: v1 kind: Service metadata: name: hello-web spec: type: LoadBalancer selector: app: hello tier: web ports: - port: 80 targetPort: 8080 no need for all these labels and selectors -
apiVersion: apps/v1 kind: Deployment metadata: name: hello-web spec: replicas: 1
selector: matchLabels: app: hello tier: web template: metadata: labels: app: hello tier: web spec: containers: - name: main image: gcr.io/google-samples/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi Kubernetes Deployment Kubernetes Service apiVersion: v1 kind: Service metadata: name: hello-web spec: type: LoadBalancer selector: app: hello tier: web ports: - port: 80 targetPort: 8080 Knative autoscales -
apiVersion: apps/v1 kind: Deployment metadata: name: hello-web spec: replicas: 1
selector: matchLabels: app: hello tier: web template: metadata: labels: app: hello tier: web spec: containers: - name: main image: gcr.io/google-samples/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi Kubernetes Deployment Kubernetes Service apiVersion: v1 kind: Service metadata: name: hello-web spec: type: LoadBalancer selector: app: hello tier: web ports: - port: 80 targetPort: 8080 Knative creates both internal and external endpoints by default -
apiVersion: apps/v1 kind: Deployment metadata: name: hello-web spec: replicas: 1
selector: matchLabels: app: hello tier: web template: metadata: labels: app: hello tier: web spec: containers: - name: main image: gcr.io/google-samples/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi Kubernetes Deployment Kubernetes Service apiVersion: v1 kind: Service metadata: name: hello-web spec: type: LoadBalancer selector: app: hello tier: web ports: - port: 80 targetPort: 8080 No need for a container name if you have only one -
apiVersion: v1 kind: Service metadata: name: hello-web spec: type: LoadBalancer
selector: app: hello tier: web ports: - port: 80 targetPort: 8080 apiVersion: apps/v1 kind: Deployment metadata: name: hello-web spec: replicas: 1 selector: matchLabels: app: hello tier: web template: metadata: labels: app: hello tier: web spec: containers: - name: main image: gcr.io/google-samples/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi Kubernetes Deployment Kubernetes Service -
apiVersion: apps/v1 kind: Deployment metadata: name: hello-web spec: template: spec:
containers: - image: gcr.io/google-samples/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi -
apiVersion: apps/v1 serving.knative.dev/v1 kind: Deployment Service metadata: name: hello-web spec:
template: spec: containers: - image: gcr.io/google-samples/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi -
apiVersion: serving.knative.dev/v1 kind: Service metadata: name: hello-web spec: template: spec:
containers: - image: gcr.io/google-samples/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi