Building the Community Around Containerised OpenStack Deployment

Kolla provides production-ready containers and deployment tools for operating OpenStack clouds that are scalable, fast, reliable and upgradable — using community best practices.

I still remember the kind of applause I got from my colleagues when I got my first OpenStack deployment working. Most of the things then worked around DevStack, the development environment of OpenStack, and vendors were still trying to find the right mix of DevOps tools to deploy OpenStack. The evolution process involved working with tools like Chef, Puppet and Ansible, with vendors creating a deployment model around them. Then came Docker, creating a buzz with a lost Linux feature called containers. A group of community members came together to see if they could containerise the OpenStack services and deploy them. The project started with the sole mission of containerising OpenStack into microservices and providing additional tooling to simplify management.

The entry barrier for Kolla

The arrival of Kolla was not particularly welcomed by the community, which was busy with already proven technologies to automate the OpenStack deployment process. At the same time, the Kolla team had its own set of challenges to cope with. Docker was itself very new at this stage, with very limited automation frameworks to use. The project went through a set of repeated events, while rediscovering the architectural changes and tools to be used. The initial implementation focused on creating the individual Docker images for core services for Centos and Ubuntu operating system flavours and deploying them with Kubernetes, which was just picking up. The configuration management part was still a half-solved puzzle, and networking was far below the requirements of both OpenStack deployment as well as its users. The deployment part was then rewritten in docker-compose, which at that time was the most popular deployment framework for multiple containers to add one more version of project rewrites.

Deploying containerised OpenStack with Kolla and Ansible

With the time spent fiddling around with the OpenStack Docker images, the Kolla team decided to reinvent the architecture again, and look for alternatives that could help deploy the containerised services with better control. This time, the team turned towards Ansible — at that time one of the latest configuration management tools on the horizon; it was still awaiting wider community adoption, so there were few preconceived notions about containers. The Kolla team used the same container images and a few known bits of the Ansible framework to get the first successful deployment. The configuration management part was well managed with Ansible. During the process, the team made a lot of changes in the Ansible Docker module, which was then contributed back to the Ansible project.

Making baby steps towards wider community adoption

The initial deployment of Kolla with Ansible was an achievement that attracted a lot of attention towards Kolla as a deployment framework for OpenStack, resulting in a number of contributors and innovative ideas from users who did different PoCs (proofs-of-concept). The team’s diversity increased and it was accepted as a part of the Big Tent governance model of the OpenStack Foundation. The Docker images were central to the adoption of Kolla and the image creation got a major revamp. Jinja based image templates took the place of individual Docker files and a configuration based build framework to build specific images. The same framework was again extended to include changes so that vendors could create custom images with little changes. As the deployment matured, along with demand from the community, the team took steps to create reusable independent modules. The project was split into two major reusable components, the image build part and the deployment part. This gave vendors better control over the consumption of Kolla based container images in their existing frameworks — as an alternative to bare-metal/VM based deployment.

Deployment with Kubernetes

As Kubernetes matured, and in response to requirements from major vendors who wanted to use Kubernetes as their orchestration platform, Kubernetes based deployment again came into the picture for Kolla images. This development is still in its nascent stage, with the 1.0 milestone in sight with core services deployment. It is still getting rewritten with different existing Ansible automation features as well as a native Kubernetes tools like Helm.

Contributing to Kolla

The Kolla project is now one of the popular repositories in the OpenStack community with containerised images of almost all Big Tent projects. The deployment automation for most of the projects is also complete with the remaining undergoing reviews. There are three major deliverables:

  • kolla, the image build repository
  • kolla-ansible, the Ansible based deployment automation
  • kolla-kubernetes, the Kubernetes based deployment automation

The Kolla repositories are hosted under OpenStack GitHub, under the Apache 2 licence. Have a look at the project documentation, connect with the community on Freenode at #openstack-kolla channel or subscribe to the openstack-dev mailing list. The Docker images are also available on Docker Hub. You can have a look at project milestones, features and bug reports on Launchpad.

Originally published at on May 8, 2018.


TC Candidacy – Swapnil Kulkarni (coolsvap)


I am Swapnil Kulkarni(coolsvap), I have been a ATC since Icehouse and I wish
take this opportunity to throw my hat for election to the OpenStack Technical
Committee this election cycle. I started contributing to OpenStack with
introduction at a community event and since then I have always utilized every
opportunity I had to contribute to OpenStack. I am a core reviewer at kolla
and requirements groups. I have also been active in activities to improve the
overall participation in OpenStack, through meetups, mentorship, outreach to
educational institions to name a few.

My focus of work during TC would be to make it easier for people to get
involved in, participate, and contribute to OpenStack, to build the community.
I have had a few hickups in the recent past for community engagement and
contribution activities but my current employment gives me the flexibilty
every ATC needs and I would like to take full advantage of it and increse
the level of contribution.

Please consider this my application and thank you for your consideration.


OpenStack PTG Denver – Day 5

Day 5 of PTG started as day for hackathons, general project/cross-project discussion for most project teams with many people left from PTG and few preparing for their travel plans or site-seeing in Colarado. The kolla team started the day with alternate Dockerfile build tool review. Later in the day was something everything in OpenStack and containers community was looking forward to the OpenStack – Kubernets SIG with Chris Hodge leading the effort to get everyone interested in same room. Some key targets for the release were identified including contributors interested. We then had most pressing issue for all deployment projects based on containers, the build and publishing pipeline for kolla images with openstack-infra team. Most of the current requirements, needs and blocking points were identified for rolling this feature. The kolla team and openstack infra team will work together to get this rolling in the starting phase of this cycle once zuul v3 rollout stabelizes. The kolla team ended day early for some much needed buzz for the whole week’s work at Station 26.


This is all from this edition of PTG see you next at Dublin.

OpenStack Queens PTG – Day 4

Day 4 of PTG started with next Kolla discussions related to kolla-ansible. Discussion started with kolla dev-mode effort started by pbourke. discussion was about currently missing pieces in dev_mode like installing clients, libs and virtualenv bindmount. The goal in the cycle is to fill the missing pieces, verify options for multinode dev_mode, investigate on options for remote debugging and also consider using PyCharm.

One of the important topics in kolla is the gating. Currently kolla has around 14 different gates for deployment testing and it has to be improved with testing the deployment for sanity with Tempest. This will help the validate the entire deployment in the gates. Upgrades testing is also one key requirement, kolla team will model something like grenade testing for it. The key is to maximize the testing of scenarios that kolla supports in gate, but since we are restricted with openstack infra resources as well as the time each test takes to validate. It is agreed that team members will create a list of scenarios and assign to everyone to verify and record the results in a central location like a google sheet. This will also help evaluate stability of kolla deployment in each release.

Skip level upgrades is one of the major talking point in the current PTG. Kolla team will evaluate fast forward upgrades for each service deployed with kolla to decide on skip level upgrade support in kolla. This would be a PoC in current cycle.

Second half of the discussion was around the kolla-kubernetes, where the team discussed the roadmap for current cycle. That will include upgrade prototyping for z stream & x stream services, validate the logging solution with fluent-bit, automated deployment, remove deprecated components and improve documentation.

Most of the teams have wrapped up their design discussions on Thursday and will be having hackathons on the last day.

OpenStack Queens PTG – Day 3

Day 3 of Queens PTG started with project specific design discussions, I joined the Kolla team where we started with the topic first for all the design summits we have had and very much important for the community, the “Documentation“. We broke down the discussion in documentation for quick-start with kolla, contributor, operators and reference documentation. The documentation available currently is scattered across projects after project split and its essential that it has a common landing page on OpenStack Deployment guides where everyone can refer to. We had representatives from the Documentation team Alex, Doug and Petr who are working on improving the doc experience by migrating the docs to a common format across the community. They understood the problem kolla is facing and we had a working discussion where we created the table of contents for all available and required documentation required for kolla.

Kolla team then joined the TripleO team which is consuming the kolla images for OpenStack deployment for discussion about collaboration of effort. The teams will work together to improve the build and publish pipeline for kolla images, improving & adding more CI jobs for the kolla/kolla-ansible/kolla-kubernetes, Configuration management post deployment of containers. The tripleo team has come up with basic healthchecks for containerized deployment, the kolla team will help get those checks in current kolla images and improve on those to better monitor the contaierized OpenStack deployment. The teams will also collaborate on improving the orchestration steps, container testing, upgrades and creating metrics for OpenStack deployment.

During lunch we had extended the discussion with Lars and kfox for discussion around Monitoring for OpenStack, Prometheus and other monitoring tools.

Post lunch, kolla team started with key discussion to the heart of operators, the OpenStack plugins deployment with kolla. There are multiple issues currently related to plugin as when would be ideal time to make them available, during build/deployment? Plugins might have non-matching depedencies to OpenStack components and so further. The team came up with multiple permutation of options available which would need to be PoCed during the release.

Since the inception of project loci there has been discussion around kolla-images size and the team had an interesting discussion on how to reduce that. The important part is to remove the things like apt/yum cache, removing the fat base image and so further. The team also discussed about utilizing althernate container build tooling to writing own image build tool. The team will hack on Friday removing the fat base images and see if that improves the image size.

External tools like Ceph are common pain points when we are doing OpenStack deployment. When kolla community evaluated the options for Ceph as storage backed for containerized openstack deployment there was no thing like containerized ceph. The team build it from scratch and got it working. The ceph team has currently come up with ceph-docker and ceph-ansible. It would be useful for operators that kolla uses the tools directly available from vendors for. We had a discussion with representative from ceph to initiate the collaboration to deprecate current ceph deployment in kolla and use the combination of ceph-docker & ceph-ansible. It will help both the communities will benefit exchange things done better at each end.

I got a surprise gift of vintage OpenStack swag from the PTG team

and I had another photo with the marketing team for with the TSP members.

The day ended with hanging out with kolla team mates at Famous Dave‘s

OpenStack Queens PTG, Denver – Day 1

It is my first time joining the Project Teams Gathering since it started in Atlanta. The location of the event is pretty unique its own way. I had never been to this part of USA and you can feel the difference. The event is held in the Renaissance Denver for 5 days between Sep 11-Sep 15.

I arrived here on Sep 10 and I could already see the active contributors in the hotel lobby discussing something or the other. I met some of the friends and took rest on the day further.

Sep, 11, the day started with registration for the event. I joined the #openstack-ptg channel to get the updates about the day and there I got introduced to ptgbot. Initially many people including me were a bit confused with how it works, but as we got familiar with it, we got more used to it for tracking the events.

As per schedule, the first two days of the event are dedicated to inter-project discussions.

I headed directly to the infra/stable/release/requirements room for discussions of requirements team. We had a discussion around topics to be worked on in Queens which include the per-project/independent/divergent requirements, OpenStack client testing, Python 3. The discussion was pretty good with insights provided by tonyb, promentheanfire, dirk, mordred, notmyname

Post lunch I joined Kolla team with discussions around collaboration across different deployment tooling in OpenStack. We had discussions around architecture, health monitoring, the role of containers, kubernetes and security.

I also attended the TC meeting for Rebooting of the Stewardship WG and Onboarding new community members.

The day ended with unofficial PTG happy hour at the elevated lounge in Renaissance Denver.