Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Applying Policy Throughout The Application Life...

Applying Policy Throughout The Application Lifecycle with Open Policy Agent

A talk from KubeCon San Diego all about policy as code. What is policy? Why should you describe it in code? How to use the Open Policy Agent ecosystem to test policy from local development, CI and in production.

Gareth Rushgrove

November 19, 2019
Tweet

More Decks by Gareth Rushgrove

Other Decks in Technology

Transcript

  1. Agenda What do we mean by policy 01 Introducing OPA

    and Conftest 02 Applying policy to a project 03 Policy in CI 04 Policy in production 05
  2. policy a set of ideas or a plan of what

    to do in particular situations that has been agreed to officially by a group of people, a business organization, a government, or a political party. Cambridge Dictionary noun [ C ] UK /ˈpɒl.ə.si/ US /ˈpɑː.lə.si/
  3. policy a set of ideas or a plan of what

    to do in particular situations that has been agreed to officially by a group of people, a business organization, a government, or a political party. Cambridge Dictionary noun [ C ] UK /ˈpɒl.ə.si/ US /ˈpɑː.lə.si/ All Go projects should have been updated to use Go 1.13
  4. policy a set of ideas or a plan of what

    to do in particular situations that has been agreed to officially by a group of people, a business organization, a government, or a political party. Cambridge Dictionary noun [ C ] UK /ˈpɒl.ə.si/ US /ˈpɑː.lə.si/ All Go projects should have been updated to use Go 1.13 Our open source projects should all use the Apache 2.0 license
  5. policy a set of ideas or a plan of what

    to do in particular situations that has been agreed to officially by a group of people, a business organization, a government, or a political party. Cambridge Dictionary noun [ C ] UK /ˈpɒl.ə.si/ US /ˈpɑː.lə.si/ All Go projects should have been updated to use Go 1.13 Our open source projects should all use the Apache 2.0 license Dockerfiles should all have a maintainers label and not use FROM with images tagged latest
  6. What is Open Policy Agent? github.com/open-policy-agent/opa - An open source

    policy engine - A CNCF project - Usable as a library and a service - A vibrant open source community - Provides a declarative DSL for writing policy called Rego Service OPA Query (any JSON value) Decision (any JSON value) Data (JSON) Policy (Rego) Request, Event, etc.
  7. A quick example Let’s suggest some places to eat this

    evening // Where should we eat while at KubeCon in San Diego? { "restaurants": [ "Campfire", "Galaxy Taco", "Olive Garden", "Dija Mara", "Mikkeller", "Wrench and Rodent" ] }
  8. A quick example Let’s describe a policy for our culinary

    preferences deny["We can't go somewhere with unlimited breadstick"] { input.restaurants[_] = "Olive Garden" } We should deny any input for which The “restaurants” list Contains a value of “Olive Garden”
  9. Open Policy Agent Usage today in the Kubernetes community Production

    Local development Continuous integration Open Policy Agent is normally used here
  10. Open Policy Agent Shifting policy left Cluster Local development Continuous

    integration What if we could use Open Policy Agent here as well?
  11. What is Conftest? github.com/instrumenta/conftest - Developer-focused UX for config policy

    - An open source project built on top of OPA - Easy to use with different inputs (JSON, YAML, INI, HCL, TOML, CUE, Dockerfile) - Build to be used as a testing tool (JSON, TAP and plain text output) - Built-in tools for sharing policy (via Git, OCI registries, S3 and more) $ conftest Test your configuration files using Open Policy Agent Usage: conftest [command] Available Commands: help Help about any command parse Print out structured data from your input pull Download individual policies push Upload OPA bundles to an OCI registry test Test your configuration files using Open P update Download policy from registry verify Verify Rego unit tests Decision (any JSON value)
  12. Conftest A simple CLI tool for asserting policy $ conftest

    test restaurants.json -p restaurants.rego FAIL - restaurants.json - We can't go somewhere with unlimited breadstick
  13. Python application example Check Python development environment settings package pipfile

    deny[msg] { version := to_number(input.requires.python_version) version < 3 msg := sprintf("Should be using Python 3, currently Using Python %v", [version]) } deny[msg] { not input.source[i].verify_ssl = true name := input.source[i].name msg := sprintf("You must verify SSL for %v", [name]) }
  14. Python application example Check Python development environment settings $ conftest

    test --input toml --namespace pipfile Pipfile FAIL - Pipfile - You must verify SSL for pypi FAIL - Pipfile - Should be using Python 3, currently Using Python 2.
  15. Python application example Check we are using specific testing tools

    $ conftest test --namespace pytest pytest.ini WARN - pytest.ini - Consider enforcing type checking when running tests WARN - pytest.ini - Consider enabling coverage reporting for test
  16. Python application example Run unit tests for our policies $

    conftest verify PASS - policy/policy/pytest_test.rego - data.pytest.test_require_black PASS - policy/policy/pytest_test.rego - data.pytest.test_require_isort PASS - policy/policy/pytest_test.rego - data.pytest.test_require_isort_and_black PASS - policy/policy/pytest_test.rego - data.pytest.test_recommend_coverage PASS - policy/policy/pytest_test.rego - data.pytest.test_recommend_type_checker PASS - policy/policy/pytest_test.rego - data.pytest.test_valid_with_required_options PASS - policy/policy/pytest_test.rego - data.pytest.test_no_warnings_with_recommended_option
  17. Python application example Check the Dockerfile for policy issues $

    conftest test --namespace docker Dockerfile FAIL - Dockerfile - Using latest tag on base image python
  18. Python application example Check policy in our Python unit tests

    def test_policy(conftest): run = conftest.verify() assert run.success def test_pytest_config(conftest): run = conftest.test("pytest.ini", namespace="pytest") assert run.success def test_kubernetes_manifest_for_warnings(conftest): run = conftest.test("snyky.yaml") result = run.results[0] assert not result.Warnings
  19. The current configuration explosion Kubernetes YAML files apiVersion: apps/v1 kind:

    Deployment metadata: name: hello-kubernetes spec: replicas: 3 selector: matchLabels: app: hello-kubernetes template: metadata: labels: app: hello-kubernetes spec: containers: - name: hello-kubernetes ~1.7 million Kubernetes configuration files public on GitHub
  20. Shared policies Porting KubeSec rules to Rego package main import

    data.lib.kubernetes # https://kubesec.io/basics/spec-hostnetwork/ deny[msg] { kubernetes.pods[pod] pod.spec.hostNetwork msg = kubernetes.format(sprintf("The %s %s is connected to the host network", [kubernetes.kind, kuber }
  21. Conftest Helm plugin $ helm conftest snyky FAIL - snyky

    in the Deployment garethr/snyky has an image, snyky, using the latest tag FAIL - snyky in the Deployment snyky does not have a memory limit set FAIL - snyky in the Deployment snyky does not have a CPU limit set FAIL - snyky in the Deployment snyky doesn't drop all capabilities FAIL - snyky in the Deployment snyky is not using a read only root filesystem FAIL - snyky in the Deployment snyky allows privilege escalation FAIL - snyky in the Deployment snyky is running as root Error: plugin "conftest" exited with error
  22. Tekton Pipeline Policy CI graph Run conftest verify Run Pipfile

    policies Run Dockerfile policies Run Helm chart policies Run pytest policies Run security policies
  23. Tekton Pipeline Describe a pipeline to run our policy apiVersion:

    tekton.dev/v1alpha1 kind: Pipeline metadata: name: snyky-pipeline spec: resources: - name: source-repo type: git tasks: - name: conftest-verify taskRef: name: conftest-verify resources: inputs: - name: source resource: source-repo - name: pipfile-conftest
  24. Tekton Pipeline Start a pipeline run $ tkn pipeline start

    snyky-pipeline ? Choose the git resource to use for source-repo: snyky-git (https://github.com/garethr/snyky.git) Pipelinerun started: snyky-pipeline-run-xrg96 In order to track the pipelinerun progress run: tkn pipelinerun logs snyky-pipeline-run-xrg96 -f -n default
  25. Tekton Pipeline View the pipeline logs $ tkn pipelinerun logs

    snyky-pipeline-run-xrg96 -f -n default ... [pytest-conftest : conftest] WARN - pytest.ini - Consider enforcing type checking when running tests [pytest-conftest : conftest] WARN - pytest.ini - Consider enabling coverage reporting for tests [conftest-verify : conftest-verify] PASS - policy/policy/pytest_test.rego - data.pytest.test_require_blac [conftest-verify : conftest-verify] PASS - policy/policy/pytest_test.rego - data.pytest.test_require_isor [conftest-verify : conftest-verify] PASS - policy/policy/pytest_test.rego - data.pytest.test_require_isor [conftest-verify : conftest-verify] PASS - policy/policy/pytest_test.rego - data.pytest.test_recommend_co [conftest-verify : conftest-verify] PASS - policy/policy/pytest_test.rego - data.pytest.test_recommend_ty [conftest-verify : conftest-verify] PASS - policy/policy/pytest_test.rego - data.pytest.test_valid_with_r [conftest-verify : conftest-verify] PASS - policy/policy/pytest_test.rego - data.pytest.test_no_warnings_ ...
  26. Gatekeeper Constraints and ConstraintTemplates apiVersion: templates.gatekeeper.sh/v1beta1 kind: ConstraintTemplate metadata: name:

    securitycontrols spec: crd: spec: names: kind: SecurityControls listKind: SecurityControlsList plural: securitycontrols singular: securitycontrol targets: - libs: - | package lib.kubernetes default is_gatekeeper = false
  27. Gatekeeper Generating ConstraintTemplates from Rego $ pk build SecurityControls.rego [SecurityControls]

    Generating a ConstraintTemplate from "SecurityControls.rego" [SecurityControls] Searching "lib" for additional rego files [SecurityControls] Adding library from "lib/kubernetes.rego" [SecurityControls] Saving to "SecurityControls.yaml"
  28. Gatekeeper Block deployments with policy violations $ kubectl apply -f

    deployment.yaml Error from server ([denied by enforce-deployment-and-pod-security-controls] nginx in the Deployment nginx-deployment does not have a memory limit set [denied by enforce-deployment-and-pod-security-controls] nginx in the Deployment nginx-deployment does not have a CPU limit set [denied by enforce-deployment-and-pod-security-controls] nginx in the Deployment nginx-deployment doesn't drop all capabilities [denied by enforce-deployment-and-pod-security-controls] nginx in the Deployment nginx-deployment is not using a read only root filesystem [denied by enforce-deployment-and-pod-security-controls] nginx in the Deployment nginx-deployment is running as root): error when creating "deployment.yaml": admission webhook "validation.gatekeeper.sh" denied the request: [denied by enforce-deployment-and-pod-security-controls] nginx in the Deployment nginx-deployment does not have a memory limit set [denied by enforce-deployment-and-pod-security-controls] nginx in the Deployment nginx-deployment does not have a CPU limit set
  29. Gatekeeper Audit running workloads against defined policy $ kubectl get

    SecurityControls audit-deployment-and-pod-security-controls -o yaml ... - enforcementAction: dryrun kind: Deployment message: nginx in the Deployment nginx-deployment doesn't drop all capabilities name: nginx-deployment namespace: audit - enforcementAction: dryrun kind: Deployment message: nginx in the Deployment nginx-deployment is not using a read only root filesystem name: nginx-deployment namespace: audit - enforcementAction: dryrun kind: Deployment message: nginx in the Deployment nginx-deployment allows privilege escalation
  30. Policy throughout the application lifecycle Production Local development Continuous integration

    Continuously enforce policy, and provide fast feedback to developers Gate your clusters against violations, and continuously audit workloads Make adopting good development practice easier
  31. 1. Open Source is pretty great OPA makes building on

    top easy. Conftest went from me hacking on something to 6 core maintainers in 6 months. Thanks tsandall, xchapter7x, brendanjryan, Proplex, jpreese, boranx and Blokje5
  32. 2. A Policy Toolkit OPA and Conftest are not tool

    or platform specific. That leaves lots of room for more domain specific tools built on-top.
  33. 3. Lets get sharing A lot of policy is at

    the organisation or community level, not per project. Lots of potential for reuse and sharing. This is the next frontier for policy as code.