At CERN we need to handle large amounts of data coming out of our complex set of particle detectors and accelerators. Every year we store over 50 Petabytes of physics data, turning storage into one of the hard infrastructure challenges we need to tackle.
With several of our workloads moving into containers, we have spent significant time making sure the integration with our storage systems allows us to scale easily while keeping high performance. Our container offer relies on OpenStack Magnum and includes Docker Swarm, Mesos/DCOS and Kubernetes, with the latter being the most popular solution.
This talk will describe the Container Storage Interface (CSI) driver for CephFS, as well as the OpenStack Manila external provisioner for Kubernetes which allows us to offer our users auto provisioning of volumes on Kubernetes. We will also present the results of a benchmarking effort with a large amounts of clients concurrently accessing a set of data shares.
In this talk we will describe the Container Storage Interface (CSI) and how having a standard way to integrate storage with containers has made our lives easier. We will also include some of the improvements introduced in the path to version 1.0 which also helped with this goal.
We will cover the work we’ve done on the CSI driver for CephFS and the added features it brings in comparison to the in-tree CephFS support in Kubernetes. We will introduce how we offer dynamic volume provisioning on Kubernetes using the OpenStack Manila external provisioner, with shares being hosted on our CephFS clusters.
Finally we will present the results of a benchmarking effort where we tested system behavior and performance with 1000s of clients provisioning and accessing storage on CephFS from Kubernetes.
