OpenStack Developer Activity Review (February 28 – March 4)

Many people have asked for more insight into the developer activities for OpenStack as the large number of code changes and proposals make it difficult to monitor everything happening. In hopes of exposing more of the developer activities, I plan to post a weekly or biweekly blog post on the latest development activities. If you have any ideas for this blog post, please email me at [email protected]. I am always ready to listen to the community for new ideas.

Activities

Developer Mailing List (archive: https://lists.launchpad.net/openstack/)

This is select list of topics discussed this week in the developer mailing list and is not a complete list.  Please visit the archive to see all the topics.

  • Authn Authz Proposal – Vish Ishaya posted information on a prototype solution for Authentication (authn) and Authorization (authz) at http://plansthis.com/auth. Instructions for the prototype are included.
  • Gzip Compression – Brian Lamer has implemented a simple WSGI implementation of gzip compression however it does not support streaming and he is opening a discussion on how best to move forward. Several developers responded but the overall consensus is to allow a tool such as Nginx/Apache/etc to handle this outside of the system as Glance does. Jorge Williams suggested that Brian submit the code anyway as someone may find another use for it.
  • Multiple Versions in OpenStack API – Brian Waldon asked about the plan to support both OpenStack API 1.0 and 1.1 as the Versions WSGI application seems to be able to deploy multiple versions of the OS API within the same codebase.  Several developers including Eric Day, Ewan Mellor, and Sandy Walsh responded to this question; more information at https://lists.launchpad.net/openstack/msg01187.html.
  • OS API server password generation – Dan Price announced a blueprint on adding support for password generation when creating services and asked for feedback. Blueprint – https://blueprints.launchpad.net/nova/+spec/openstack-api-server-passwords; Details – http://etherpad.openstack.org/openstack-api-server-passwords.  Further discussion from Ed Leafe, Justin Santa Barbara, Mark Washenberger, Scott Moser, George Reese, and Rick Clark can be followed at https://lists.launchpad.net/openstack/msg01175.html.
  • Management API – Glen Campbell has created a preliminary proposal for admin/management API for Nova at http://wiki.openstack.org/NovaAdminAPI. Glen’s focus is to determine the features required for Rackspace, which may or may not be needed by the community.

Statistics

  • Number of OpenStack Developers on Contributors List – 153 (+2 for week)
  • Cactus Release Status – Blueprints (http://wiki.openstack.org/releasestatus/)
    • Essential – 5 Design Approved; 1 Implemented – 1 Needs Code Review – 1 Beta Available
    • High – 12 Blueprints; 3 Implemented – 1 Needs Code Review – 1 Blocked
    • Medium – 21 Blueprints; 2 Needs Code Review; 3 Implemented
    • Low – 16 Blueprints; 1 Needs Code Review; 6 Implemented

For the latest on development activities on OpenStack please check these sites for more details:

Tags:

Community Weekly Newsletter (February 28 – March 4)

OpenStack Community Newsletter – March 4, 2011

This weekly newsletter is a way for the community to learn about all the various activities occurring on a weekly basis. If you would like to add content to a weekly update or have an idea about this newsletter, please email [email protected].

Midokura booth at Cloud Days in Tokyo

HIGHLIGHTS

EVENTS

DEVELOPER COMMUNITY

GENERAL COMMUNITY

COMMUNITY STATISTICS (2/25– 3/3)

  • Data Tracking Graphs – http://wiki.openstack.org/WeeklyNewsletter
  • OpenStack Compute (NOVA) Data
    • 27 Active Reviews
    • 166 Active Branches – owned by 49 people & 10 teams
    • 1,849 commits by 62 people in last month
  • OpenStack Object Storage (SWIFT) Data
    • 3 Active Reviews
    • 48 Active Branches – owned by 19 people & 4 teams
    • 215 commits by 11 people in last month
  • Twitter Stats for Week:  #openstack  tweets;  re-tweets; all OpenStack total tweets  ** Looking for new tool to gather this data **
  • Bugs Stats for Week:  277  Tracked Bugs; 44 New Bugs; 33 In-process Bugs; 6  Critical Bugs; 30 High Importance Bugs; 106 Bugs (Fix Committed)
  • Blueprints Stats for Week:  165 Blueprints; 8 Essential, 15 High, 26 Medium, 22 Low, 94 Undefined
  • OpenStack Website Stats for Week: 8,791 Visits, 19,315  Pageviews, 53.16% New Visits
    • Top 5 Pages: Home 43.24%; /projects 11.34%; /projects/compute 16.86%; /projects/storage 11.38%; /Community 7.21%

OPENSTACK IN THE NEWS

Tags:

OpenStack Governance Update

We’ve built quite an open source community together since we launched less than eight months ago!  What started as a small group of people committed to building an open cloud standard, has grown to hundreds of developers and more than 50 participating organizations virtually overnight.  From the beginning, this community was founded with the goal of diversity of participation and a firm commitment to what we call “the 4 opens”:  Open Source, Open Design, Open Development and Open Community.

As we take stock of the amazing interest and growth, keeping in mind the initiative’s goals and commitment to openness, the time has come to evolve the governance process to match the new reality of a larger, more diverse community.  To that end, the governance process has been updated, with full details published here.

As you read through the highlights below, we encourage you to get personally involved to steer this community to an even bigger, brighter future.  Whether it’s participating in a spirited debate on the mailing list, attending the bi-annual design summits, or even running for one of the elected positions, there are a lot of ways to get involved and there’s no time like the present to dive in.  Nominations and elections will be held later this month for many elected positions.

Highlights:

  • Each Project — OpenStack Compute (Nova), OpenStack Object Storage (Swift), and the OpenStack image service (Glance) will elect their own Project Technical Leads (starting later this month, March 2011) to run the projects and make day-to-day technical decisions.  Elections will be held every six months, just prior to each design summit, and these elected leaders will be instrumental in guiding those public design summits and setting the future direction of their project.
  • The Project Oversight Committee – which has been charged with setting policies that span projects as well as determining when new projects should be added – will be renamed the Project Policy Board effective immediately, to better reflect their mission.
  • This Project Policy Board will be revamped to become more nimble and ensure broad representation.  Specifically, 2/3 of the seats on the board will now be elected rather than appointed by Rackspace:
    • 5 General Board Seats elected to one-year terms, with elections occurring prior to each design summit (2 each spring*, 3 each Fall)
    • 3 Board Seats reserved for the winners of the Project Technical Lead elections* (more as we add projects)
    • 4 seats appointed by Rackspace
  • We are establishing an OpenStack Advisory Board of senior advisors comprised of major commercial sponsors (those who are building businesses on OpenStack), enterprises and service providers who are deploying it, and category experts.  The primary function of this body is to provide guidance on OpenStack’s mission, and to evangelize on its behalf.  Prior to the Spring 2011 Design Summit, Rackspace will appoint the initial members from a variety of organizations – but the board will then determine its own plans and requirements for expansion.

*Upcoming Elections:  As noted above, a total of 5 seats are up for election later this month, March 2011, prior to the Spring 2011 Design Summit.  3 of these will be Project Technical Leads for the respective projects, and will also sit on the Project Policy Board representing those respective communities, and 2 will be General Board Members.  More details soon regarding the nomination and election process.

Again, we invite everyone to get involved and have your voice heard.  If you’re interested in running for the Project Policy Board, or becoming a Project Technical Leader, now’s the time to throw your hat in the ring.  Registration for the second public Design Summit will open in the next few days, in which members of the community set the roadmap and make technical decisions to drive the projects forward.  You can get plugged in with our new community page at openstack.org/community.

Building the Complete Open Source Cloud with Xen.org and OpenStack

The Xen.org community, home of the open source Xen hypervisor, announced today the availability of Xen Cloud Platform (XCP) 1.0.  XCP 1.0 provides a full-featured virtualization platform solution from bare metal including the Xen hypervisor, network and storage support, and a management stack. This platform is a perfect complement to the OpenStack cloud infrastructure solution by providing the virtualization layer for a complete open source cloud stack from bare metal to cloud orchestration software.

The release of XCP 1.0 enables the Xen community to leverage the industry leading Xen hypervisor in creating a virtualization platform for the OpenStack community. Citrix Systems, a leader in the OpenStack and Xen communities, enabled not just the Xen hypervisor but also tightly integrated XCP 1.0 with OpenStack Compute and OpenStack Object Storage. This newly available joint solution provides solution providers building public and private clouds an opportunity to fully deploy a complete open source solution for their customers and employees.

Not only are the two communities actively working together to ensure a quality customer solution, but Xen.org is also taking a leading role in the governance of OpenStack with Ewan Mellor participating as a member of the OpenStack Project Oversight Committee. Ewan’s leadership role in OpenStack provides the community with an experienced open source and enterprise developer helping to guide the technology and development process.

Finally, the two projects have also submitted a joint paper proposal for OSCON 2011; “Achieving Hybrid Cloud Mobility with OpenStack and XCP” from Paul Voccio at Rackspace and Todd Deshane at Xen.org. The connection between these two open source communities is off to a great start and we look forward to more alignment in the future as open source cloud computing becomes a significant player in the industry.

You can obtain XCP 1.0 here.

Tags:

OpenStack Party at SXSW in Austin, TX

Join Rackspace Hosting, the world’s leading specialist in the hosting and cloud computing industry, and OpenStack, the fastest growing open source cloud project creating the open standard cloud operating system, for some fun and arcade games! We hope to see you there, free food and drinks for all!

Kung Fu Saloon
510 Rio Grande
March 16, 2011
6pm

More information at http://schedule.sxsw.com/events/event_OE00338.

Tags:

10 Steps to Initiating an OpenStack Cloud Service

This blog post originates from Eric Day at http://oddments.org/?p=591. I have copied it here for your reference:

OpenStack currently consists of three main components: Nova (Compute), Swift (Object Storage), and Glance (Image Service). There are some other projects such as a dashboard and mobile apps as well. You can see the full list here. This is great start, but in order for OpenStack to compete long term other infrastructure and platform services will need to be brought in. I’d like to talk about the process I’m taking with a new message queue service.

Step 1 – Idea

The first step is to figure out what is missing. What new service would compliment the software already available? What hasn’t been solved yet? What are users asking for? A message queue seemed like an appropriate next step as most applications that need to scale out and be highly available will make use of a queue at some point (sometimes not in the most obvious form). It will also allow other cloud services to be built on top of it. In fact, the current OpenStack projects could even leverage a queue service for new features.

Step 2 – Initial requirements

Before you write up a proposal and send it out, it might be a good idea to gather some initial requirements and figure out what it may look like. Don’t worry about details as the community will help flush this out later. Some of the major requirements when thinking about OpenStack projects are horizontal scalability, multi-tenancy, modular API, REST API, zones and locality awareness, and no single points of failure (high availability). This is a pretty heavy set of requirements before even getting into service specifics, but this will help you think about how to approach a service. You may have to diverge away from traditional thinking for a particular service. For example, what worked in a rack or a data center may not work in the cloud. You need to account for this up front and state behavioral differences from what folks may expect. For the queue service, this meant not taking a traditional approach you see in some queue protocols and services, and instead integrating ideas from distributed services.

A multi-tenant cloud is a very different environment from what many people are used to and usually requires a different approach to solve problems. If folks tell you you’re re-inventing the wheel, take their concerns into consideration, but also realize you may not be. You may be writing a jet engine.

Step 3 – Wiki and Mailing List Proposal

Once you have a good idea and a rough outline, you’ll probably want to run it by a couple people for feedback before sending it to everyone. You’ll then want to create a new wiki page on the OpenStack wiki and send a note to the public mailing list that mentions the wiki page and asks for community feedback. For example, the queue service proposal I wrote can be found here. There is an enormous amount of collective experience and brain power on the mailing list which will help point out any issues with the proposal. The service you initially propose may look nothing like the service you actually build. It’s also quite possible the service you propose is not a good fit for the cloud or OpenStack. The community will help iron all these details out.

Step 4 – Wait

It can take folks a while to catch up on public mailing lists, so be patient. Let people know about the proposal by other means (blog, tweet, irc, …) and help facilitate the conversation as people respond.

Step 5 – Prototype

Once you feel the community is content with the proposal and it’s a viable idea (don’t expect consensus), prototype it! This shows the community you are serious and this exercise will help work out more issues in the proposal. Let the community know about it and again wait for any feedback. This doesn’t need to be anything fancy, for the queue service I put this together over a weekend.

Step 6 – Name and Language

Now comes the difficult part, choosing a project name. I’d suggest not using the mailing list for this as it will be a lot of noise for a matter that isn’t too important. Ask a couple folks who may also be interested for ideas and make sure it’s not already taken (search on github, Launchpad, Google, etc). For the queue service we decided on “Burrow”.

You’ll also need to figure out the most appropriate language. For middleware and services, Python is a good default. If efficiency is a concern, look at Erlang or C/C++. Be sure to send another mail to the list and ask for feedback. With the queue service I initially proposed C++ with Erlang as an alternative since efficiency is a major concern (especially around utilizing multiple cores), and the community came back mixed but with more enthusiasm for Erlang.

Step 7 – Bootstrap the Project on Launchpad

We’re using Launchpad for OpenStack project management. You’ll need to create a project and a number of groups to manage it. For example, the queue project can be found here. The groups have the following roles (replace burrow with your project name):

  • burrow – Public group that anyone can join. This currently includes members on the main OpenStack mailing list, but we’re setup this way in case we need to break projects out into their own list.
  • burrow-drivers – The group responsible for maintaining the project, managing blueprints, and making releases.
  • burrow-core – The group responsible for performing code reviews.
  • burrow-bugs – The group responsible for managing bugs.

Step 8 – Lock onto Releases and Milestone Schedule

While not important right away, it might be a good idea to start working with the OpenStack release cycle. Releases are currently every three months with milestones setup in each release for feature freeze, bug freeze, and releases. See the release page for more details. Launchpad makes it fairly simple to manage this, you’ll just want to create a new series (for example, “cactus” right now), and a couple milestones within that series for the freezing and release. Ask on the mailing list or on IRC if you need any help, but a good rule of thumb is to follow what other established projects do (like Nova).

Step 9 – Code!

Get to work and try to recruit other developers to help you. Keep the community updated with progress by using IRC, mailing list, planet.openstack.org, and tweets.

Step 10 – Submit to the Project Oversight Committee

Up until this point your project has not been an official OpenStack project. It is a well thought-out idea driven by the community that probably has a good chance though. You’ll need to make a proposal to the POC using this page once the project can stand on it’s own. You probably don’t need a final version, but you need something that is functional and more robust than a prototype. The POC meets weekly, although it may take more time (and some conversations) to decide if your project is ready. The queue service I’ve been driving has not been proposed since it’s not ready, so you may want to take all this with a grain of salt. It is my hope to have the first version ready to propose in April as part of the Cactus release.

Final Thoughts

This process will vary and can certainly be refined. I’m stating what I’ve done with a new project, but existing projects will obviously need to take a different route. The main idea to keep in mind though is that any OpenStack project should be seen as community driven, not just by an individual or company. One or more individuals may carry out a large part of the work of the community initially, but community concerns and feedback should always be taken with the utmost importance.

Tags:

OpenStack Conference / Design Summit Update

Here is the latest information on the upcoming OpenStack Conference / Design Summit:

Dates: April 26-29, 2011
Location: Hyatt Regency in Santa Clara, CA (http://santaclara.hyatt.com/hyatt/hotels/index.jsp?null)
Cost: $100 for all attendees; late registrations will pay $150
Event Size: 350 attendees is the maximum

The OpenStack Conference will be April 26-27, 2011 for business development, technologists and others interested in the OpenStack open source project. The OpenStack Design Summit for community developers will be April 27-29, 2011 and will consist of three days of blueprint reviews and discussions for future OpenStack releases. The agenda for both events is still in process with a list of currently submitted topics at http://etherpad.openstack.org/SubmittedTopics and working agenda at http://etherpad.openstack.org/DesignSummitApril2011. The Program Committee is currently looking at the submitted topics and fitting them into the agenda as well as recruiting more speakers/topics.

The event registration website is currently being completed and is expected to be published by the middle of the week. The registration site will also contain a link to the Hyatt Regency so attendees can take advantage of the discounted room block. If you are staying at the Hyatt, please leverage the room block as the community is required to cover the costs of all rooms not booked which could be a significant budget item.

Finally, several community participating companies have stepped up to sponsor the event and that list will be released soon. If you are interested in sponsoring, please review the current prospectus at http://www.box.net/shared/ng0r7u6n44.

If you have additional questions, want to speak at the event, or are interested in being a sponsor please contact me at [email protected].

Tags: