At Oath, we deploy a heavily customized version of OpenStack across many different regions and clusters. We need a way to build each OpenStack component with all of our customizations into a self-contained package that is deployable in our data centers. We also need fine grained control over our deployments such that we can swiftly deploy fixes/enhancements to single components in a single cluster with minimal downtime.
We designed a build platform on our open source tool, Screwdriver, which is capable of building and publishing our customized, self contained packages as both RPMs and PyPi packages. We then built CI/CD pipelines using Screwdriver and Chef which pull down the custom packages that are published by our build system and deploy them individually in minutes. The pipelines are capable of deploying individual components without redeploying the entire stack. Each cluster has its own pipeline which gives us very fine grained control over our deployments
Attendees can expect to learn how we do continuous deployment at Oath while also maintaining stability. This involves deploying changes/enhancements on top of stable releases rather than the master branch which is important for us (and perhaps other large deployers) because we don’t want to run an actively developed branch in production. We will also be outlining best practices and design principles to follow when attempting to build your own CI/CD pipelines for OpenStack. Attendees will also learn about the challenges we faced with building our deployment infrastructure and how those challenges influenced our decisions and the design of our new system. Making customizations to OpenStack can be challenging to do in a clean and maintainable way. We will be discussing the types of changes we make to OpenStack and how others can go about making similar changes in a way that will incur minimal overhead when upgrading to a new release.
