Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ContainerDays 2019 - Write Maintainable Integra...

ContainerDays 2019 - Write Maintainable Integration Tests with Docker

Integration tests are extremely valuable but very hard to maintain and to write. You need to provision and manage the lifecycle for the dependencies that your application uses such as caches and databases. Testcontainer is an open source community all focused on solving this kind of problems across many languages. Gianluca is the maintainer of the Golang implementation that uses the Docker API to expose a test-friendly library that you can use in your test cases. During this talk, he will show you how to use testcontainer to write unit tests in Go with very low overhead.

Gianluca Arbezzano

June 26, 2019
Tweet

More Decks by Gianluca Arbezzano

Other Decks in Technology

Transcript

  1. © 2019 InfluxData. All rights reserved. 2 Who am I?

    Gianluca Arbezzano Site Reliability Engineer @InfluxData • http:/ /gianarb.it • @gianarb What I like: • I make dirty hacks that look awesome • I grow my vegetables • Travel for fun and work
  2. This doesn’t mean we need to stop doing unit tests!

    They still matter a lot! Disclaimer
  3. $ docker-compose up -d $ make test-integration 1. Orchestration Issue

    2. Validation issue 3. Flakiness 4. No Parallelization What about integration tests?
  4. The SDK ctx := context.Background() cli, err := client.NewClientWithOpts(client.FromEnv) if

    err != nil { panic(err) } cli.NegotiateAPIVersion(ctx) reader, err := cli.ImagePull(ctx, "docker.io/library/alpine", types.ImagePullOptions{}) if err != nil { panic(err) } io.Copy(os.Stdout, reader)
  5. Programmatically provision your integration tests config = MySqlContainer('mysql:5.7.17') with config

    as mysql: e = sqlalchemy.create_engine(mysql.get_connection_url()) result = e.execute("select version()")
  6. func TestNginxLatestReturn(t *testing.T) { ctx := context.Background() req := testcontainers.ContainerRequest{

    Image: "nginx", ExposedPorts: []string{"80/tcp"}, } nginxC, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{ ContainerRequest: req, Started: true, }) if err != nil { t.Error(err) } defer nginxC.Terminate(ctx) ip, err := nginxC.Host(ctx) if err != nil { t.Error(err) } port, err := nginxC.MappedPort(ctx, "80") if err != nil { t.Error(err) } resp, err := http.Get(fmt.Sprintf("http://%s:%s", ip, port.Port())) if resp.StatusCode != http.StatusOK { t.Errorf("Expected status code %d. Got %d.", http.StatusOK, resp.StatusCode) } }
  7. ctx := context.Background() req := ContainerRequest{ Image: "mysql:latest", ExposedPorts: []string{"3306/tcp",

    "33060/tcp"}, Env: map[string]string{ "MYSQL_ROOT_PASSWORD": "password", "MYSQL_DATABASE": "database", }, WaitingFor: wait.ForLog("port: 3306 MySQL Community Server - GPL"), } mysqlC, _ := GenericContainer(ctx, testcontainers.GenericContainerRequest{ ContainerRequest: req, Started: true, }) connectionString := fmt.Sprintf("%s:%s@tcp(%s:%d)/%s?tls=skip-verify", "root", "password", host, port, "database") db, err := sql.Open("mysql", connectionString)
  8. /** Integration test for Redis-backed cache implementation. */ public class

    RedisBackedCacheTest { @Rule public GenericContainer redis = new GenericContainer("redis:3.0.6").withExposedPorts(6379); private Cache cache; @Before public void setUp() throws Exception { Jedis jedis = new Jedis(redis.getContainerIpAddress(), redis.getMappedPort(6379)); cache = new RedisBackedCache(jedis, "test"); } @Test public void testFindingAnInsertedValue() { cache.put("foo", "FOO"); Optional<String> foundObject = cache.get("foo", String.class); assertTrue("When an object in the cache is retrieved, it can be found", foundObject.isPresent()); assertEquals("When we put a String in to the cache and retrieve it, the value is the same", "FOO", foundObject.get()); } https://github.com/testcontainers/testcontainers-java/tree/master/examples
  9. @Rule public GenericContainer dslContainer = new GenericContainer( new ImageFromDockerfile() .withFileFromString("folder/someFile.txt",

    "hello") .withFileFromClasspath("test.txt", "mappable-resource/test-resource.txt") .withFileFromClasspath("Dockerfile", "mappable-dockerfile/Dockerfile"))
  10. public class ElasticsearchStorageRule extends ExternalResource { static final Logger LOGGER

    = LoggerFactory.getLogger(ElasticsearchStorageRule.class); static final int ELASTICSEARCH_PORT = 9200; final String image; final String index; GenericContainer container; Closer closer = Closer.create(); public ElasticsearchStorageRule(String image, String index) { this.image = image; this.index = index; } @Override protected void before() { try { LOGGER.info("Starting docker image " + image); container = new GenericContainer(image) .withExposedPorts(ELASTICSEARCH_PORT) .waitingFor(new HttpWaitStrategy().forPath("/")); container.start(); if (Boolean.valueOf(System.getenv("ES_DEBUG"))) { container.followOutput(new Slf4jLogConsumer(LoggerFactory.getLogger(image))); } System.out.println("Starting docker image " + image); } catch (RuntimeException e) { LOGGER.warn("Couldn't start docker image " + image + ": " + e.getMessage(), e); } https://github.com/apache/incubator-zipkin/blob/b8646142fa15c8c5f47ff2a2a48dc663c7bb65b3/zipkin-storage/elasticsearch/src/test/java/zipkin2/elasticsearch/inte gration/ElasticsearchStorageRule.java#L30
  11. Write your own test util functions You can write your

    packages that include functions to spin up and configure your environments.
  12. ctx := context.Background() k := &KubeKindContainer{} err := k.Start(ctx) if

    err != nil { t.Fatal(err.Error()) } defer k.Terminate(ctx) clientset, err := k.GetClientset() if err != nil { t.Fatal(err.Error()) } ns, err := clientset.CoreV1().Namespaces().Get("default", metav1.GetOptions{}) if err != nil { t.Fatal(err.Error()) } https://github.com/testcontainers/testcontainers-go/pull/67
  13. Links and Credits • https://medium.zenika.com/dockerize-your-integration-tests-8d26a7425baa • https://www.testcontainers.org/ • https://github.com/bmuschko/testcontainers-demo •

    https://gianarb.it/blog/testcontainers-go • https://rnorth.org/better-junit-selenium-testing-with-docker-and-testcontainers • https://github.com/testcontainers