The OpenStack Blog

Author Archive

OpenStack Technical Committee Update (June 25)

Two weeks ago, Russell Bryant inaugurated a series of blogposts about what the Technical Committee (“TC”) is working on. We will regularly post about the outcomes of the TC meetings, and rotate writers to give all TC members a chance to participate and describe what happens in their own words. This post will focus on what happened during the last two meetings.

The TC has several missions, and the topics we cover in TC meetings generally fall into one of them.

Mission #1: Integrated release contents

One of the missions of the TC is to determine what is part of the OpenStack “integrated release” that we collectively produce every 6 months. We manage the “incubation” process, through which selected projects can become part of the integrated release. We also keep an eye on already-integrated projects in case their scope evolves.

That’s the case currently for Glance, whose mission is evolving from cataloging and serving Nova disk images to cataloging and serving other artifacts consumed by other OpenStack services, like for example Heat templates. This scope evolution has been under discussion at the last two meetings. The principle of expanding Glance’s scope is pretty much accepted at this point, but the precise words to describe the new scope are still under discussion.

In order to set clear base expectations for projects in the incubation/integration process track, last cycle the TC came up with a reference document listing all the requirements for incubation, integration and the first integrated cycle that are consensual across TC members. This document is constantly revisited as we continue to raise the quality and convergence bar between integrated projects.

Last two meetings we have been discussing adding a translation support requirement for projects wishing to graduate from incubation. The main objection to it at this point is the lack of automated testing of the translated strings (which resulted in undetected I18N problems in the past). It looks like when this is addressed, this requirement will probably make it to the reference document.

Raising requirements for new entrants is one thing, but sometimes existing integrated projects do not fill those new requirements. This creates a gap that we need to address. During this cycle we reviewed most existing integrated projects (Heat and Swift are still pending). When a gap was raised, PTLs responded with a proposed “gap coverage plan” to address it.

The last project to go through that exercise was Glance, where a single gap around testing coverage was raised. Mark Washenberger (Glance PTL) created a gap coverage plan to address it and that plan was blessed by the TC at the last meeting.

Having plans is a good thing, but we also need to check that projects meet the deadlines that they set in such plans. Last week, after the juno-1 development milestone, we reviewed the progress on gap coverage plans. Ceilometer plan is on track, still targeting the juno-2 milestone for fully covering the gap. Horizon plan is mostly on track. Neutron has an ambitious gap coverage plan, and work on all gaps has started; gap 4 is a bit late (was planned to be completed by juno-1), but it’s also been determined to just be one API call. Trove plan is on track, and work started on all items.

Finally, having responsibility over the integrated release also means making sure we use terminology across integrated projects consistently. An issue was raised about the use of “certified” terminology to qualify CI testing on some projects. After open discussion on the -dev mailing-list and at the TC meeting, there was agreement that “certified” terminology was too loaded and should be phased out in favor of some variation around “tested”.

Mission #2: Representation of technical contributors

The TC is a directly-elected body representing all the technical contributors to the project. It is the final decision-making entity over technical matters in OpenStack as a whole. Some issues that can’t be solved at a lower level are sometimes escalated to the TC for final resolution, and the TC is also a convenient conduit for other OpenStack governance bodies to ask for general technical input.

Two issues were brought to the TC recently. The first one is about expected election behavior for our technical elections (PTLs and TC members). Our current election procedure doesn’t clearly describe expected behavior, and a proposal was raised to cover that. While everyone agrees on what is acceptable behavior and what is not, there are two ways of addressing issues if they arise. One is to piggy-back on the Community Code of Conduct, which clearly states that you should respect the election process. The other is to call out out-of-line behavior and trust the voters to make their own judgment about it. Both options are and will stay available, but we are still discussing which of those options (if any) we should encourage by making it part of the TC resolution. This discussion is on-going and will continue in a future meeting.

The other issue being brought to the TC were requests from the Foundation Board of Directors “Defcore” subcommittee for technical input to use as part of their work on trademark rules. There are two types of requested inputs. One is to provide for each project “designated sections” of code that you need to run in order to use the “OpenStack” trademark. The other is to give precise scoring for “core capabilities”: for each capability, indicate whether it’s part of the TC future direction or if it’s on its way to be deprecated.

While those inputs are technical (and we even voted on guidelines to help coming up with answers), some TC members expressed clearly their discomfort. Asking the representation of OpenStack contributors to designate parts of OpenStack that may just be replaced by proprietary alternatives (while still being called “OpenStack”) just crosses the line as to what they consider acceptable. Our “technical” answer might be read as an endorsement, or collaboration toward a behavior we don’t really want to encourage.

This tension has been surfacing every time Defcore was discussed at the TC meeting, and it’s difficult to address it between two topics in a one-hour online meeting. We have therefore scheduled a Defcore-specific TC meeting for July 1st (20:00 UTC in #openstack-meeting on Freenode IRC), and will try to clearly list concerns beforehand to ensure meeting clarity.

Technical Committee & Grizzly Update

The OpenStack development teams continue to make progress in many areas. We recently published the grizzly-2 development milestone. It marks the middle of the “Grizzly” development cycle, which will end on April 4. So far 99 feature blueprints have been completed, and 113 more are still likely to be included before our feature freeze date on February 19th. This will mean nearly 200 improvements by the time the final release arrives.

Some changes are very visible, like the introduction of new versions of APIs and support for new networking, storage and authentication backends. Also, this release includes the much-anticipated work on Compute Cells that makes it easier to deploy and manage OpenStack clouds at very large scale. Some other changes are less visible, but as important, like the creation of common libraries to reduce technical debt and minimize duplication of code between projects.

Our development infrastructure, handling about 2000 commits per month, continues to improve. It now runs even more unit and integration tests before accepting new code, while reducing the overall time it takes to run them!

The Technical Committee has begun meeting regularly and working on a number of items. It decided to accept into incubation the Ceilometer and Heat projects, adopted new policies on 3rd-party API and Python support, and is participating in the joint committee working on defining the future of Incubation and Core with the Board of Directors.

Waiting list for the Design Summit

The first 200 open seats for the Essex Design Summit were registered in less than 9 days. This event, geared towards existing and prospective OpenStack developers, is separate from the OpenStack Conference and has a more limited capacity. For the last 50 seats, the Design Summit organization committee needs to give priority to OpenStack core developers and well-known contributors that missed the boat, so we set up a waiting list.

This list will be reviewed regularly, and people that are confirmed will receive an email. The last 50 seats will be gradually assigned over the next weeks. If you want to make sure to have a seat at the OpenStack Conference, you should probably register for that event separately, since there can be no guarantee you will get one of the last Essex Design Summit seats. Sign up early on the Summit waiting list to increase your chances !

What to expect from the Conference and Design Summit

In less than two weeks now, the community around OpenStack will gather in Santa Clara for two co-hosted events: the OpenStack conference and the Diablo Design Summit. In this post I’ll explain what attendees should expect from both, and give some precisions on how the Design Summit will be run.

OpenStack Conference

The OpenStack conference will run on two days, from Tuesday April 26 to Wednesday April 27. This is a classic conference, with speakers making presentations on various OpenStack-related topics. We’ll have keynote sessions on Tuesday morning, then parallel technical and community tracks will run during Tuesday afternoon. The day will end with a reception sponsored by Cisco. On the Wednesday, the “User track” will run during the morning and the “Service provider” track will run on the afternoon.

Diablo Design Summit

The Diablo Design Summit will run on three days, starting Wednesday morning to end on Friday evening. It’s a gathering of OpenStack developers to discuss how the next releases of OpenStack should be made and brainstorm the design of the key features we’ll add. Developers submit blueprints, those are turned into session topics and scheduled in the available session slots.

Since developers are currently busy getting the “Cactus” release out, we expect the schedule to start to take shape during next week. You should expect sessions on release management to occur on the Wednesday morning, and once the expectations are set, various sessions on Nova, Swift, Glance and other OpenStack projects spread throughout the 3 days of the event. Sessions will be categorized so that it’s easy to tell what the topic is by looking at the schedule.

The sessions are held in fishbowl-style room layout, with a session lead (usually the submitter of the blueprint) moderating the discussion. You can look up the whole process in more details at http://wiki.openstack.org/Summit

During the three days, after lunch, we’ll also have Lightning talks to let attendees have a forum to quickly talk about their crazy idea, present their product or do a quick demo. Every subject is OK as long as it’s OpenStack-related ! Those will be openly-scheduled on a whiteboard, everyone is free to book available 5min slots.

On the social side, Wednesday evening there will be an OpenStack Developer Gathering in the developer lounge, sponsored by Niciria. On the Thursday evening we’ll have our traditional Developer party, sponsored by Cloudscaling.

I hope this clarifies what the events are all about, and I’m excited to see you all in less than two weeks now !

Coming up in OpenStack Bexar release

(This article is an updated version of a post originally posted here).

OpenStack is busy with so much development activity it’s hard to keep up.  42 (!) specs were targeted for the 3-month long Bexar development cycle… and there are more than 150 active branches. Over December alone, we saw more than 900 commits by 60 different people ! Taking a step back, what new features should you expect to land on February 3rd, in the Bexar release ?

Swift (OpenStack object storage)

The big news in Swift is support for unlimited object size, through the implementation of client-side chunking. The only size limit for your objects is now the available size in your Swift cluster ! You can read more about that exciting feature in John Dickinson’s blog post. We also hope to ship Swauth, DevAuth highly scalable replacement, directly into Swift codebase. Exposure of most of the S3 API in Swift may or may not make it.

Glance (OpenStack image registry and delivery service)

The Glance image service will expose a unified REST API (no more distinction between the image registry and the image delivery services). We will also have the possibility to upload image data and metadata over one single call. Unified client classes will be shipped directly in Glance. We also hope to have a S3 backend

Nova (OpenStack compute)

There is so much coming up in Nova it’s hard to summarize. Nova will make use of those new Glance client classes, obviously. We will support booting VMs from raw disk images (rather than a kernel/ramdisk/image combination) and have a rescue mode to mount your faulty disks under a sane environment. We plan to have instance snapshots ready. API servers can now expose optional admin features (through the –allow_admin_api flag), like a specific XenServer instance pause or suspend feature.

Lots of improvements might go unnoticed, like the internationalization of messages, the standardization on services using eventlet, more robust logging, or the move of the IP allocation down the stack. We’ll also finalize some incomplete features, like access to your project VLAN through a VPNsecurity groups that work in all network modes, and we hope to finally ship Hyper-V support.

We hope to have much more: a web-based serial console to access your VMs, ipv6 support, the possibility to deploy hardware in a staging area of your cloud, support for highly available block volumes through Sheepdog, instance diagnostics allowing to retrieve a history of actions on instances, the possibility to do live migration in nova-manage, iSCSI support for XenAPI… But let’s be realistic, not everything will land in time. What doesn’t make it will certainly be in the next release, Cactus, which will be released in April !

Congrats to our awesome development team for making all this possible. Those last two months have been a very fun ride for me :)

Back to top