Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Creating Products through DevOps: The Story of VSHN

Creating Products through DevOps: The Story of VSHN

Since 2014, our slogan at VSHN has been The DevOps Company. Following that ethos, VSHN applies all of its DevOps, Agile, and Sociocracy practice to creating our services and products. How to take into account diverging opinions during product development? How to match ethical business practices with market needs? How can a company be respectful of its employees and profitable simultaneously? Is there a working alternative to hierarchies? In this session, you will discover how DevOps shaped all our decisions, from idea to market. In particular, we will describe the challenges, processes, and decisions we took while creating our latest product, “APPUiO Cloud.”

Presentation shown at the Conf42: DevOps 2023 conference in January 2023.

Adrian Kosmaczewski

January 26, 2023
Tweet

More Decks by Adrian Kosmaczewski

Other Decks in Technology

Transcript

  1. VSHN – The DevOps Company
    Adrian Kosmaczewski, Developer Relations, VSHN
    Creating Products
    through DevOps
    The Story of VSHN
    Thank you so much for this opportunity to talk about
    VSHN and APPUiO Cloud in the Conf42 DevOps 2023
    event. Through this talk we would like to give you an
    idea of how we built this new product using DevOps as
    a guiding philosophy.
    Speaker notes
    1

    View full-size slide

  2. VSHN – The DevOps Company
    1. About VSHN’s Culture
    2. What is APPUiO Cloud?
    3. A DevOps Way of Working
    4. Some crunchy details
    Agenda
    Today I’m going to talk about how we built APPUiO
    Cloud at VSHN. For that I will first start by explaining a
    bit our culture, what is APPUiO Cloud, and then how we
    used DevOps to create it. I’ll give you some practical
    details about how we made it, too.
    Speaker notes
    2

    View full-size slide

  3. VSHN – The DevOps Company
    Pronounced ˈvɪʒn – like "vision"
    The DevOps Company
    Founded 2014, 50 VSHNeers located in Zürich, Switzerland
    Switzerland’s leading DevOps, Docker & Kubernetes partner
    ISO 27001 certified & ISAE 3402 Report Type 1 verified
    First Swiss Kubernetes Certified Service Provider
    Before we start, I’d like to introduce VSHN to those who
    have never heard of it before.
    That’s how you pronounce the name, and we’re a
    company of 50 people based in Zürich, founded in
    2014 with the objective of providing companies with
    DevOps services. The slogan of VSHN is, actually, "The
    DevOps Company." We embrace the DevOps philosophy
    completely, and as you’ll see today, we use its
    principles and ideas in everything we do.
    What does VSHN do? We provide various services and
    products:
    We offer "DevOps as a Service" (or DaaS), with a full
    team of Kubernetes and OpenShift experts ready to
    monitor your applications 24/7 on any cloud.
    We help companies become self-sufficient and cloud-
    enabled; we help software engineers to "build
    bridges" between Dev and Ops, building CI/CD
    pipelines in various platforms such as GitLab,
    OpenShift, or Argo CD.
    We are Kubernetes & OpenShift specialists, to the
    point that our strategy is 100% oriented towards
    Kubernetes; everything we do runs on Kubernetes.
    Speaker notes
    3

    View full-size slide

  4. VSHN – The DevOps Company
    The DevOps Company
    Self-funded, organic growth
    Sociocracy
    Handbook:
    Our Culture
    handbook.vshn.ch
    Aside from our technological choices, the important
    thing to know about VSHN is that we have chosen to
    drive the growth of our company in various ways that
    are completely nonstandard:
    We are "The DevOps Company", and as such, we
    embraced the DevOps mantra completely. Everything
    we do is as automated as possible, freeing our
    brains to think.
    We have decided as a company not to grow through
    venture capital, instead relying in the good old
    method of organic growth. We have been self-funded
    since day one (2014), and we have been consistently
    profitable and growing since 2017.
    We use Sociocracy as a management and growth
    framework. This means that all decisions, and I mean
    all of them, happen through consensus among all
    VSHNeers (that’s how we call ourselves, by the way.)
    We have created a Handbook, freely available online,
    which in printed form takes 573 pages, explaining
    everything we are and do at VSHN with quite an
    incredible level of detail. I invite you to check it out at
    handbook.vshn.ch and you will learn everything
    there’s to learn about us.
    Speaker notes
    4

    View full-size slide

  5. VSHN – The DevOps Company
    vshn.ch/en/blog/how-vshns-organization-evolves-using-sociocracy-3-0
    Sociocracy is our evolution framework, and we have a
    small team of people at VSHN whose only job is to help
    us evolve into this framework continuously. In
    particular, any VSHNeer is able to raise issues,
    problems, and to ask for help to change procedures or
    situations that are hurting their happiness at VSHN.
    VSHNeers are able to create "VSHN Improvement
    Proposals" or VIPs as we call them; they are simply
    tickets in our Jira that explain a current situation or
    decision, the drawbacks and negative impact, and
    propose a solution to be discussed by everyone
    involved and/or interested in the issue.
    This simple mechanism has completely transformed the
    way we work in the past 3 years, and as a result we all
    feel part of the structure we built, and we all feel
    responsible for it at all times.
    Speaker notes
    5

    View full-size slide

  6. VSHN – The DevOps Company
    Hiring
    All of these choices have shaped our culture in ways
    that are really not common at all, and have interesting
    consequences in our day-to-day operations. For
    example, our hiring policies are different to those of
    most IT companies; not only we do pay attention to the
    IT skills of those who want to join our team, but place a
    very high degree of attention to the human factor. We
    want people to feel great at VSHN, and one of the
    primary factors we evaluate during our hiring process is
    the "likeness" of the person, that is, how much would
    we like to work with them every day?
    As Steve Jobs once said, "we don’t hire smart people to
    tell them what to do. We hire smart people so that they
    tell us what to do."
    Speaker notes
    6

    View full-size slide

  7. VSHN – The DevOps Company
    Remote & Async
    Another interesting consequence of how VSHN works is
    that we had embraced remote and asynchronous
    working well before the pandemic; when the Bundesrat
    mandated everyone to work from home in March 2020,
    we simply stayed home and continued working as if
    nothing had happened. The important bit of information
    here is the asynchronous word; not so much that we
    work remote, but that we work in a non synchronous
    way. This particular mindset has shaped our company
    greatly.
    Speaker notes
    7

    View full-size slide

  8. VSHN – The DevOps Company
    But let’s get back in time a little bit, and see how
    APPUiO Cloud came to be.
    In 2016 VSHN and Puzzle ITC (a well known Swiss IT
    and software consulting) launched a joint venture called
    APPUiO. This is a word in Esperanto, meaning
    "Support".
    APPUiO consists of a series of products built around
    Red Hat OpenShift.
    Speaker notes
    8

    View full-size slide

  9. VSHN – The DevOps Company
    For those who haven’t heard of it yet, OpenShift is the
    most widely used Kubernetes-based platform in the
    enterprise world. It is quite popular with big companies,
    and it incorporates a hardened and highly available
    Kubernetes cluster surrounded by lots of relevant
    software: a container repository, a management
    console, CI/CD pipelines, with a very nice and
    professional GUI on top.
    We decided we wanted to be a part of the OpenShift
    market, but we also realized that installing and
    operating OpenShift is a huge endeavour, and many
    companies could not use OpenShift because of the lack
    of staff or budget. So we decided to join forces with
    Puzzle ITC.
    Speaker notes
    9

    View full-size slide

  10. VSHN – The DevOps Company
     
    APPUiO is our response to the complexity of Red Hat
    OpenShift. With APPUiO, customers can get a ready-to-
    use cluster, together with the know-how of VSHN and
    Puzzle ITC. We at VSHN we specialize in the setup and
    maintenance of OpenShift clusters; we have been
    operating OpenShift clusters since version 3. Puzzle ITC
    are specialists in the creation of software solutions for
    OpenShift, which is something we don’t do. Together,
    the APPUiO team can help companies make the most
    out of their OpenShift investment.
    Speaker notes
    10

    View full-size slide

  11. VSHN – The DevOps Company
    APPUiO Public  APPUiO Cloud
    APPUiO Managed
    APPUiO Self-Managed
    Flavors
    APPUiO has been historically available in various forms:
    APPUiO Public, based on OpenShift 3, was the first
    Swiss-based shared OpenShift cluster available to
    customers. It was a shared platform, where
    customers can run their projects without having to
    care about management or anything else. There were
    APPUiO Public clusters running in various cloud
    providers, such as Cloudscale in Switzerland and
    AWS in Germany.
    APPUiO Managed is the next step. With APPUiO
    Managed, organizations get their own OpenShift
    cluster, for their exclusive use, and Puzzle ITC and
    VSHN take care of the operations of the cluster
    transparently for their users.
    APPUiO Self-Managed is the final step in the
    evolution of organizations: with it, organizations not
    only get an OpenShift cluster "keys in hand" and
    ready to run, but we teach their IT teams how to
    manage and maintain the cluster by themselves. We
    gradually "fade in the background" and provide help,
    until at some point they become completely
    independent.
    APPUiO Cloud is the latest offering in the APPUiO
    family, officially launched in September 2021.
    Speaker notes
    11

    View full-size slide

  12. VSHN – The DevOps Company
    What is APPUiO Cloud? Simply put, APPUiO Cloud is to
    OpenShift 4 what APPUiO Public was to OpenShift 3.
    But given the major architectural changes between
    OpenShift 3 and 4, instead of migrating our APPUiO
    Public infrastructure to OpenShift 4 we decided to
    create a new project from scratch, and we gave it a
    different name and even a different visual identity.
    We notified our APPUiO Public customers of the
    upcoming phasing out of the service, with an offer to
    help them migrate their payloads to APPUiO Cloud.
    APPUiO Public was fully decommissioned in September
    2022, merely one year after APPUiO Cloud started
    operations.
    Speaker notes
    12

    View full-size slide

  13. VSHN – The DevOps Company
    cloudscale-lpg-2  Lupfig (AG)
    exoscale-ch-gva-2-0  Genève (GE)
    Regions
    As said previously, APPUiO Cloud is based exclusively
    on OpenShift 4. At the moment we have two APPUiO
    Cloud zones available to our customers:
    cloudscale.ch in Kanton Aargau
    Exoscale au Canton de Genève
    We plan to open more regions in the future, as required
    and following the demand from our customers.
    Speaker notes
    13

    View full-size slide

  14. VSHN – The DevOps Company
    We started working on APPUiO Cloud in Spring 2021,
    and we released to the public in Autumn that year. We
    reused lots of code and infrastructure we had created
    for our work previously:
    K8up, a Kubernetes backup operator that has been
    picked up by the Cloud Native Computing Foundation
    as a sandbox project:
    Project Syn, a suite of tools that allow for the remote
    management of Kubernetes clusters of any kind, from
    a central location using a GitOps philosophy and
    workflow:
    Speaker notes
    k8up.io
    syn.tools
    14

    View full-size slide

  15. VSHN – The DevOps Company
    Startups
    DevOps & CI/CD Pipelines
    Mobile app backends
    Education
    Technology trial
    Resellers
    Target Audience
    APPUiO Cloud, just like its predecessor APPUiO Public,
    is meant to be an "entry level" product, catering at the
    "long tail" of OpenShift customers, who might be
    interested in getting access to a working OpenShift
    cluster without the hassle of installing and operating it.
    As such, we identified a few target groups:
    Startups: Spin a new OpenShift namespace on
    APPUiO Cloud, deploy your MVP, and go back to
    raising more venture capital.
    DevOps & CI/CD Pipelines: Run your CI/CD pipelines,
    deploy, and preview your application on a running
    OpenShift cluster right now before going live.
    Mobile App Backends: For iOS or Android developers
    needing to deploy back-ends on a scalable, trusted
    environment.
    Education: Let students experience the full power of
    a real OpenShift cluster with their own individual
    namespace.
    Technology Trial: For users interested in APPUiO
    Managed cluster, but needing to hedge the risks of
    trying out new technology.
    Resellers: Resell APPUiO Cloud and let us manage
    the cluster for you.
    Speaker notes
    15

    View full-size slide

  16. VSHN – The DevOps Company
    Instant On
    Pay-per-use
    User management
    Integrated backups
    Pre-installed operators
    Community support
    Features
    What is included in APPUiO Cloud?
    Instant On: Get your own OpenShift namespace in
    minutes, ready to use.
    Pay-per-use: Only pay for the resources you actually
    use.
    User Management: Organize your namespaces in
    teams and organizations, and assign users to those
    teams; control who can access which namespaces at
    a glance.
    Backup: Backup all your work with the pre-installed
    K8up operator.
    Pre-Installed and Configured Operators: APPUiO Cloud
    provides the following OpenShift operators pre-
    installed and pre-configured, ready to be used:
    K8up: Kubernetes Backup Operator.
    Cert Manager: X.509 certificate management for
    Kubernetes.
    Community Support: Need help? Check out our
    APPUiO Cloud forums and community chat. For those
    needing more help, there are support packages
    available at extra cost.
    Speaker notes
    16

    View full-size slide

  17. VSHN – The DevOps Company
    Strict maintenance policies
    Status information:
    Resource availability: no guarantees
    SLA: Best-effort
    Fair-use policy
    No privileged containers
    Log retention: 72 hours
    No other operators (yet)
    Restrictions
    status.appuio.cloud
    Because it’s a public platform, APPUiO Cloud comes
    with some gotchas:
    Maintenance Policies: There are two types of
    maintenance: APPUiO Cloud Zones receive automatic
    revision updates (for example, from 4.7.1 to 4.7.2).
    This includes OpenShift and worker nodes updates.
    These updates can happen at any time without prior
    announcement. Upgrades of minor (for example from
    4.7 to 4.8) and major (4 to 5) OpenShift versions are
    announced in advance, with a description of all
    possible breaking changes. On request, we can
    provide you with access to an already upgraded
    APPUiO Cloud Zone, for you to test your deployment if
    needed.
    Status Information: We communicate the status of
    the platform on status.appuio.cloud.
    Resource Availability: APPUiO Cloud is provided
    without any guarantees of resource availability.
    SLA: Best-effort.
    Fair-Use Policy: APPUiO Cloud is a shared platform.
    Unless otherwise stated, this fair-use principle
    applies to the use of our services. APPUiO Cloud
    users must use their resources moderately, so as not
    to degrade the service level available to other users.
    Privileged Containers: Privileged containers can’t run
    on APPUiO Cloud.
    Log retention: The OpenShift integrated logging
    (Elasticsearch / Kibana) retains collected logs for
    72h (3 days), after that time-period logs are
    permanently deleted.
    Other Operators: It is not possible (at the moment) to
    run other OpenShift operators than the ones we are
    offering. We evaluate them on a case-by-case basis,
    following the requests from our customers.
    Speaker notes
    17

    View full-size slide

  18. VSHN – The DevOps Company
    A DevOps Way of Working
    What do we mean by "a DevOps way of working"? Let’s
    see first what we mean by DevOps, one of those words
    that can mean anything and everything dependending
    on who you ask.
    Speaker notes
    18

    View full-size slide

  19. VSHN – The DevOps Company
    twitter.com/acloudguru/status/1318624283200020487
    This is clearly not what we mean by DevOps. Although
    to be honest, there’s a lot of YAML involved in what we
    do.
    Speaker notes
    19

    View full-size slide

  20. VSHN – The DevOps Company
    Usually when people talk about DevOps they think
    about this physical division between developers and
    operation teams, and how they don’t communicate
    anymore, and how much better it would be if they did.
    Speaker notes
    20

    View full-size slide

  21. VSHN – The DevOps Company
    Then DevOps comes along, with its long list of
    technologies and buzzwords…
    Speaker notes
    21

    View full-size slide

  22. VSHN – The DevOps Company
    … and somehow all barriers are destroyed, and we can
    once again collaborate and work better together.
    For us at VSHN, this is a limited view of what DevOps is
    and can bring; it is an important part, but not all.
    Instead, we prefer to think about DevOps as a set of
    three principles, following what some authors have
    written about it.
    Speaker notes
    22

    View full-size slide

  23. VSHN – The DevOps Company
    In particular, we think the best people to talk about
    DevOps is the author of "The DevOps Handbook" and
    "The Phoenix Project": Gene Kim. The latter book is
    actually a modern rewriting and reinterpretation of a
    classic management book from the 1980s called "The
    Goal" by Eliyahu Goldratt, but quite faithful in spirit.
    Speaker notes
    23

    View full-size slide

  24. VSHN – The DevOps Company
    1. The Principles of Flow
    2. The Principles of Feedback
    3. The Principles of Continual Learning and Experimentation
    In those books, DevOps is usually defined by the "three
    ways":
    1. The Principles of Flow
    2. The Principles of Feedback
    3. The Principles of Continual Learning and
    Experimentation
    Let’s see how these three principles helped us build a
    new product.
    Speaker notes
    24

    View full-size slide

  25. VSHN – The DevOps Company
    1. The Principles of Flow
    Let’s start with the principles of flow and see what it
    means for product development.
    Speaker notes
    25

    View full-size slide

  26. VSHN – The DevOps Company
    Value Stream
    The first thing was to decide where to start, that is,
    what was the value stream we wanted to provide first.
    We wanted to have actual results as early as possible,
    because seeing things happen and appearing is one of
    the best ways to keep a team in activity, motivated, and
    delivering.
    Speaker notes
    26

    View full-size slide

  27. VSHN – The DevOps Company
    First: Product Documentation
    That work brought together the Product Documentation.
    That’s right, the first thing we created through
    discussion was a written documentation of what we
    wanted to offer.
    Why written? Because we work asynchronously. That
    means that some of us work better at night, while some
    work better in the morning; having everything written
    down helped everyone, commenting down drafts of the
    documents until there was agreement. Agreement from
    whom? From the Product Owners to the DevOps
    engineers who would have to maintain the solution at
    the end.
    This way, the operations team knows exactly what is it
    that’s going to happen. There are no surprises down the
    hall, and they feel empowered and listened to. All the
    features of APPUiO Cloud are, simply put, possible to
    release; either now or later, but they are possible.
    Speaker notes
    27

    View full-size slide

  28. VSHN – The DevOps Company
    vshn.ch/en/blog/reverse-engineering-conways-law
    The important thing here is that we started by applying
    Conway’s Law. That is, we first structured the team that
    would work on APPUiO Cloud, and then we got to create
    the system. The end result of this process is that the
    architecture of APPUiO Cloud, following Conway’s Law,
    strictly mirrors the structure of our team. We do not
    fight against Conway’s Law; we embrace it.
    Speaker notes
    28

    View full-size slide

  29. VSHN – The DevOps Company
    For Product Owners
    For System Engineers
    For Users
    Documentation
    products.docs.vshn.ch/products/appuio/cloud/
    kb.vshn.ch/appuio-cloud/
    docs.appuio.cloud/
    The result of this work of architecture can be
    summarized in three different documentation websites
    for APPUiO Cloud; you’ve heard right, we have created
    three different sets of documentation, and we keep
    them updated every day:
    Product owner documentation at
    System engineer documentation at
    End user documentation at
    We have made all of the documentation publicly
    available and viewable, even editable, because
    transparency is one of our values at VSHN. We want all
    of our customers to know exactly we’re doing things the
    way we do; this, in turn, generates trust in our existing
    customers, and shows our know-how to prospective
    customers. These three documentation sites are,
    simply put, great marketing tools!
    Speaker notes
    products.docs.vshn.ch/products/appuio/cloud/
    kb.vshn.ch/appuio-cloud/
    docs.appuio.cloud/user/
    29

    View full-size slide

  30. VSHN – The DevOps Company
    1. Reduce batch sizes
    2. Reduce work intervals
    3. Build quality in
    Flow
    The principle of Flow requires teams to make work
    visible, recuding batch sizes and intervals of work, and
    to build quality in. We limited work in progress to the
    strict minimum, and we automated as much as
    possible of the process.
    Speaker notes
    30

    View full-size slide

  31. VSHN – The DevOps Company
    Automation
    This automation involves removing the human factor
    from the maintenance of those clusters as much as
    possible. One of the key factors for doing this was
    Project Syn, a suite of tools we started building in 2019
    that allows our small team to manage hundreds of
    clusters from a central location. We created Project Syn
    as a way to be able to operate our customers' assets
    with a reduced human footprint, but it turned out to be
    a great way to handle our own work on APPUiO Cloud,
    too.
    Speaker notes
    31

    View full-size slide

  32. VSHN – The DevOps Company
    Thanks to Project Syn, DevOps engineers can specify
    and deploy changes to lots of Kubernetes clusters from
    a central location, using a GitOps strategy; just commit
    your changes as "infrastructure as code" to a Git
    repository, and wait a few seconds until all clusters
    apply those changes.
    We use Project Syn to deploy Kyverno security policies
    to our APPUiO Cloud clusters, so that all regions
    conform to the same rulebook.
    We also configured each of the APPUiO Cloud zones
    with the mandatory differences between the cloud
    providers we use; Exoscale and cloudscale.ch do not
    offer exactly the same features, and being able to see
    those differences in written form allows us to manage
    those systems, to take decisions for the future, and to
    inform our customers of any possible tradeoff.
    Speaker notes
    32

    View full-size slide

  33. VSHN – The DevOps Company
    2. The Principles of Feedback
    We know that APPUiO Cloud is a complex system, built
    out of complex systems, that are prone to failure at any
    given time. It is not a matter of "if", but rather a matter
    of "when."
    Speaker notes
    33

    View full-size slide

  34. VSHN – The DevOps Company
    Observability
    We have built observability and management tools
    immediately from the start in our work of APPUiO Cloud.
    We have reused the management infrastructure
    provided by OpenShift, the same one we were using for
    our private customers, and we have built APPUiO Cloud
    to be observable at all times.
    Speaker notes
    34

    View full-size slide

  35. VSHN – The DevOps Company
    Policies
    Builds
    Configuration
    Infrastructure
    Documentation
    "Everything as Code"
    Using "everything as code" as a basis for our work
    means that every time we fix an issue on the platform,
    we have to change a configuration file somewhere. This
    information is later stored in the Git repo, as part of the
    project history; not only that, but we also update the
    required documentation files, both internal and
    external, so that everyone knows (asynchronously and
    at their own rhythm) what happened, when, where, and
    most importantly, why.
    And when we say "everything as code", we mean it:
    Policies
    Builds
    Configuration
    Infrastructure
    Documentation
    All of this is described in their corresponding files, and
    versioned in Git repos. We use GitLab, and its
    integrated CI/CD pipelines are configured to
    automatically build, test, and eventually deploy changes
    as required. Thanks to Project Syn, all of the feedback
    the bring back to the system is automatically deployed
    whenever possible, reducing the amount of human
    brain work required to keep things running.
    Speaker notes
    35

    View full-size slide

  36. VSHN – The DevOps Company
    Documentation as Code
    Even our documentation is automated: we use the
    Antora documentation generator tool, which can
    automatically extract and integrate documentation from
    various sources into a single website, and we use
    GitLab pipelines for that as well. With this process,
    engineers only have to update the documentation
    sources (using the Asciidoc format, very similar to
    Markdown) and git push their changes. They are
    immediately picked up, built, verified (we have
    automatic styling and syntax checks built-in in our
    pipelines) and deployed.
    Speaker notes
    36

    View full-size slide

  37. VSHN – The DevOps Company
    3. The Principles of Continual
    Learning and Experimentation
    APPUiO Cloud is not, and will never be, finished. It is a
    product that changes continuously, sometimes in small
    ways, and sometimes in bigger ones.
    Speaker notes
    37

    View full-size slide

  38. VSHN – The DevOps Company
    console.cloudscale-lpg-2.appuio.cloud
    This is a screenshot of the APPUiO Cloud console in the
    cloudscale.ch region around May last year.
    Can you see the red banner on top? This is the result of
    us learning something interesting.
    Speaker notes
    38

    View full-size slide

  39. VSHN – The DevOps Company
    CPU Requests & Quotas
    Issue with CPU requests resolved. The
    resolution includes a slight change to the
    pricing model.
    Here is the text of the red banner in the previous
    screenshot.
    This, as you can imagine, is the result of a learning
    process. We realized that, in our preparation, we had
    not designed our CPU request pricing properly. As a
    result, as soon as the first users started using the
    platform this year, we realized that some of them were
    consuming disproportionate amounts of CPU; this was
    a huge problem, since they were not aware of that, and
    we would have to cover for those extra costs at first.
    Speaker notes
    39

    View full-size slide

  40. VSHN – The DevOps Company
    Solution
    As of 2022-05-01: The underlying infrastructure
    has a fixed ratio between memory and CPU
    resources. CPU requests exceeding that ratio
    will be counted as well. The requested CPU
    cores will be multiplied with the platforms'
    ratio. This yields an equivalent in MiB. The
    memory to CPU ratio can be different per zone.
    See zone listing for the exact values.
    products.docs.vshn.ch/products/appuio/cloud/pricing.html#_compute
    We modified the policies in the clusters, made a
    communication to all of our customers, and updated
    our documentation as shown on the slide.
    This was an unexpected and unplanned learning; a local
    discovery that brought a global improvement in APPUiO
    Cloud for all users. We did cover some of the costs, but
    we rectified our policies openly, and communicated
    clearly with our customers. The result? Not only all of
    them acknowledged and understood the changes, but
    we didn’t lose a single customer because of this
    change. This level of cooperation with our customers is
    one of the things we’re most proud of.
    Speaker notes
    40

    View full-size slide

  41. VSHN – The DevOps Company
    Crunchy Details
    Let me give you now some details about the work we
    did, including team sizes, tech stack used, and many
    other details. Transparency is one of our values, so
    we’re very happy to tell you everything about it.
    Speaker notes
    41

    View full-size slide

  42. VSHN – The DevOps Company
    1 Project manager / Product manager
    1 Product owner
    3 DevOps engineers core team (6 nowadays)
    1 System architect
    + 3~5 engineers from other teams
    + Marketing & sales team members
    1.
    The Team: Aldebaran [1]
    handbook.vshn.ch/vshn_teams.html
    The team called "Aldebaran" in VSHN was mostly in
    charge of the design, deployment and operation of
    APPUiO Cloud. They have also received help from other
    teams, in particular those with experience in the
    deployment and operation of OpenShift clusters, and of
    course from Marketing and Sales, to coordinate
    communication and marketing campaigns to get new
    users onto the platform.
    The Project Manager and main Product Manager of
    APPUiO Cloud is Tobias Brunner, one of the founders of
    VSHN and its current CTO, who provided a very strong
    vision (no pun intended) about how APPUiO Cloud
    should behave and look like.
    Speaker notes
    42

    View full-size slide

  43. VSHN – The DevOps Company
    Red Hat OpenShift 4.11
    Policies: Kyverno
    IdP: Keycloak
    Secrets: Vault
    Storage: Rook
    CNI: Isovalent Cilium Enterprise
    Backup: K8up
    GitOps: Project Syn
    Documentation: Antora
    Tech Stack
    kyverno.io
    keycloak.org
    vaultproject.io
    rook.io
    isovalent.com/product
    k8up.io
    syn.tools
    antora.org
    Let’s go into some details about how APPUiO Cloud is
    built. This slide contains the list of major components
    we’ve chosen for it.
    Speaker notes
    43

    View full-size slide

  44. VSHN – The DevOps Company
    Zoom
    Discourse
    RocketChat
    Jira & Confluence
    Visual Studio Code Live Share
    Collaboration Tools
    discourse.org
    rocket.chat
    During our day to day asynchronous communication and
    collaboration we used various tools. Here’s a small
    sample of them.
    Speaker notes
    44

    View full-size slide

  45. VSHN – The DevOps Company
    2021-02-19: First discussions about "APPUiO Public 2.0"
    Spring/Summer 2021: Design & development
    2021-06: Benchmarking storage options
    Rook, Ceph, or Longhorn? [1]
    2021-07-29: "APPUiO Cloud" name chosen
    "appuio.cloud" domain registered
    1. Spoiler: Rook won!
    Timeline (1/5)
    vshn.ch/en/blog/benchmarking-kubernetes-storage-solutions
    Here’s a short timeline of events leading to the release
    of APPUiO Cloud.
    We started talking about "APPUiO Public 2.0" around 2
    years ago. In July we had chosen the product name and
    we had registered the domain name.
    Speaker notes
    45

    View full-size slide

  46. VSHN – The DevOps Company
    2021-08: Project Syn components for APPUiO Cloud
    2021-08: cloudscale.ch cluster ready for testing
    2021-09-17: published
    2021-09-20: Public announcement [1]
    1.
    Timeline (2/5)
    docs.appuio.cloud
    vshn.ch/en/blog/announcing-appuio-cloud
    Things accelerated during the Summer of 2021. We
    made the public announcement in September…
    Speaker notes
    46

    View full-size slide

  47. VSHN – The DevOps Company
    2021-10-28: Users start migrating apps out of APPUiO Public
    2021-12-01: Testing phase ended
    2021-12-07: Pricing calculator released
    2021-12-16: New logo
    2021-12-21: Partnership with Isovalent [1]
    1.
    Timeline (3/5)
    vshn.ch/en/blog/partnership-vshn-isovalent
    … and our users started migrating their apps in October
    already. By December we announced a partnership with
    Isovalent, to use their CNI plugin on APPUiO Cloud.
    Speaker notes
    47

    View full-size slide

  48. VSHN – The DevOps Company
    2022-02-02: Exoscale Geneva region available [1]
    2022-02-07: released
    2022-02-09: Getting started guide published
    2022-09-01: APPUiO Public fully decommissioned
    1.
    Timeline (4/5)
    portal.appuio.cloud
    vshn.ch/en/blog/bonjour-geneve-je-mappelle-appuio-cloud
    Last year we opened a new region in Geneva, and
    released the APPUiO Cloud Portal so that our users can
    manage their projects, users, and groups
    autonomously.
    Speaker notes
    48

    View full-size slide

  49. VSHN – The DevOps Company
    2022-09-14: AppCat announced, supporting S3 buckets [1]
    2022-10-27: Vertical Pod Autoscaler available [2]
    2022-12-20: Workload monitoring available for users [3]
    2023-01-17: New AppCat feature: DBaaS by Exoscale [4]
    1.
    2.
    3.
    4.
    Timeline (5/5)
    vshn.ch/en/blog/announcing-appcat-on-appuio-cloud
    vshn.ch/en/blog/vertical-pod-autoscaler-on-appuio-cloud
    vshn.ch/en/blog/openshift-4-11-and-user-workload-monitoring
    vshn.ch/en/blog/announcing-dbaas-by-exoscale-on-appcat
    Finally, we released our new product AppCat, which
    allows APPUiO Cloud users to specify dependencies
    such as S3 buckets, databases, message queues, and
    other systems directly in YAML from their OpenShift
    projects. We also enabled vertical pod autoscaling and
    workload monitoring for our users.
    Speaker notes
    49

    View full-size slide

  50. VSHN – The DevOps Company
    Conclusion
    Can other organizations use a similar process to create
    a product? We believe that yes, it is possible. However,
    there are a few caveats, that we know some companies
    should have to work on those items first, in order to
    have a successful DevOps journey.
    Speaker notes
    50

    View full-size slide

  51. VSHN – The DevOps Company
    1. Writing skills
    2. Cloud-Native technology
    3. Trust
    First of all, writing skills are fundamental. We need
    DevOps engineers to be writers, and to put everything
    down. Not only as "everything as code" (security,
    infrastructure, business rules, etc) but also as
    documentation writers, making sure that both engineers
    and users are able to refer to a written document that
    explains the reasons why things happen. Yes, keeping
    that written documentation is part of the work; it is not
    a chore, it is not a bonus; it is part of the deliverables,
    and it must be updated, reviewed, and proofread.
    Second of all, Cloud Native technologies have been
    designed to work faster than ever. Containers,
    Kubernetes, CI/CD pipelines, Open Source, and all of
    the ecosystem of Cloud Native technology is the
    greatest enabler of our world. The technological context
    constitutes a fantastic "giant’s shoulder" where we can
    stand on, and go faster and better. We definitely could
    have never done this work without the ecosystem of
    Open Source Cloud Native technologies available today.
    But third of all, trust is paramount. You have to have
    trust in your teams. We actually think that trust is more
    important than flat hierarchies; even though these have
    helped us, without trust there’s no way we could have
    created APPUiO Cloud in such a short amount of time.
    Trust allows teams to work independently, moving fast,
    and without the inherent fear typical of a "blame
    culture". And trust is the key ingredient for an
    asynchronous work culture. You cannot really go full
    async if you do not trust your teams. We stress this
    point, because this factor is the deal breaker for many
    teams in this country.
    These are, we think, the three most important pillars of
    our DevOps culture: writing, technology, and trust.
    Those who have helped us shape APPUiO Cloud into a
    Speaker notes
    51

    View full-size slide

  52. VSHN – The DevOps Company
    No.
    Easy?
    Is it easy to work like this in DevOps mode? Of course
    not, there are lots of things that can go wrong.
    Speaker notes
    52

    View full-size slide

  53. VSHN – The DevOps Company
    Yes!
    Worth it?
    Is it worth it? Let’s put it this way: after all this time, we
    have internalized this way of working so much, that we
    couldn’t do things any other way. We think it is totally
    worth it, and as a result, we just do things like this.
    With APPUiO Cloud, VSHN has demonstrated that we
    can deliver world-class products in a short amount of
    time, with a small team of experts, and with fast cycles
    of feedback and experimentation baked in the process.
    Speaker notes
    53

    View full-size slide

  54. VSHN – The DevOps Company
    Behind the scenes
    About the APPUiO Cloud API
    How billing works
    All APPUiO Cloud posts
    vshn.ch/en/blog/behind-the-scenes-at-appuio-cloud
    vshn.ch/en/blog/about-the-appuio-cloud-api
    vshn.ch/en/blog/appuio-cloud-billing
    vshn.ch/en/blog/appuio-cloud
    We regularly publish blog posts telling the story of our
    product, and sharing news about future features or
    developments. Check it out at
    Speaker notes
    vshn.ch/en/blog/appuio-
    cloud
    54

    View full-size slide

  55. VSHN – The DevOps Company
    Go to and use the voucher code CONF42
    Try APPUiO Cloud for 30 days!
    appuio.cloud/register
    Try APPUiO Cloud by yourself!
    Speaker notes
    55

    View full-size slide

  56. VSHN – The DevOps Company
    Adrian Kosmaczewski, Developer Relations, VSHN –
    VSHN AG – Neugasse 10 – CH-8005 Zürich – +41 44 545 53 00 – –
    Thanks! Questions?
    [email protected]
    vshn.ch [email protected]
    Thank you so much for your attention!
    Speaker notes
    56

    View full-size slide