The OpenStack Blog

Category: Development

OpenStack Jenkins dashboard available for testing Ubuntu snapshots

The keener eyed of you may have noticed:

https://jenkins.qa.ubuntu.com/view/Precise%20OpenStack%20Testing/

James Page has setup the jobs in the Ubuntu OpenStack QA Lab to start publishing to the public Jenkins QA instance this morning. We now have automated build testing of all core OpenStack components triggered from upstream trunk commits. This is followed by automated deployment (-deploy) of OpenStack in the lab with a serving of testing (-test) once its all up and running.

Credit to Adam Gandelman for the Juju charm work, deployment framework and test execution and to Chuck Short for the hugely misnamed tarball.sh script which completes the git/bzr/packaging fu to build and deploy OpenStack packages!

The plan is to get the upstream Tempest test suite running in the lab; at the moment we are running a more limited test script just to ensure that you can spin up and instance and see it on the network.

(Crossposted from cloud.ubuntu.com)

The First OpenStack Bug Squashing Day Is Coming On Feb 2nd

Shine your keyboards, hackers of OpenStack: on February 2nd 2012 you will be called to fight the ever growing number of bugs that keep creeping in our beloved code.  On Bug Squashing Day all the OpenStack developer community will focus mainly on Nova to:

  • Close old fixed bugs. Old bugs are nasty. Even when they are long dead, they clog bug views and render the lists unusable. Just look at old bugs and check if they still apply ! If they don’t, close them as FixReleased (if you can pinpoint when they were fixed) or Invalid (if you can’t).
  • Fix bugs. The best thing you can do on a bug squashing day is to kill a live one. Just look at the list of Confirmed or Triaged and pick your target. Submit a change that fixes it. Ask for review help on the channel.
  • Triage incoming bugs. It’s sometimes hard to distinguish fresh bugs from false alarms. You can help by using your expertise or reproduction skills on New bugs. If you can confirm the issue, set the bug to Confirmed. If you can fix it, read the previous chapter. If you need more info from the reporter, set it to Incomplete. And if it happens to not really be valid, set it to Invalid.

You don’t have to be an experienced Nova developer to participate, and we believe that February 2nd will be a great way to get started with the OpenStack community. You can get started by looking at Devstack to build your complete OpenStack development environment. The other projects are welcome to focus on quality that day but Nova is the one that will get more attention.

The event will happen mostly online, in a dedicated #openstack-bugsquash IRC channel on Freenode (that all participants are encouraged to join for the duration of the event). There will also be live meetings in Austin and San Francisco, hosted by Rackspace with food, drinks and games.

Do you want to host a Bug Squash Day on Feb 2nd? Let us know and we’ll add it to the list.

 

Happy Ada Lovelace Day

Ada Lovelace day, October 7th, is a day for bloggers to write a story about an inspirational influence in their life in technology.

For me, there were two influential woman in my life as an undergraduate chemistry student in the early 90s at Butler University in Indianapolis, Indiana. One was my first college chemistry professor, Anne McCowan, and the other was Butler’s scientific librarian, Mrs. Howes. Both influenced me through words, and bringing the importance of words to my attention. Professor McCowan stated on the first day of class:

“Chemistry is a study of nomenclature. Once you understand the naming and vocabulary, the world of chemistry is opened to you.”

It was such a simplification of an intimidating subject that it crystallized the learning process for me. If I studied the vocabulary, the rest would follow. And here I am, combining the wonder of worlds and technology every day.

So on today, Ada Lovelace day, I want to ask, how can OpenStack be a welcoming community for women in technology? I have ideas and want to share them with the community. These are both small ideas and large ideas.

  • Inspire girls when they’re young. I have volunteered with an organization called GirlStart here in Austin, Texas, and I think they’ve got the right idea, influence girls to enter technology in middle school and elementary and encourage them to go to college. A few years ago I went to lunch once a month with middle school girls where we talked about simple ideas such as “what does it mean to be smart?” That group of girls will be in high school now, and I hope they find technology a good path for them.
  • Invite women specifically. I spoke with Noirin Plunkett at OSCon this summer, and she said that women don’t necessarily have the confidence (or is it ego) to understand they are being specifically invited to participate in a tech initiative or open source project. You can specifically say to a group of female collage students for example, by saying “our project needs you specifically, not just your male colleagues.”
  • Start in your neighborhood, at your company. Since Rackspace is a huge supporter and founder of OpenStack, we want to ensure that we bring our women to the project and make them feel like Stackers are their kind of people. Stackers are professional, mature, and respectful of each other. We certainly have heated discussions but all input is valuable. I want to start locally by inviting women to Austin Cloud User Group meetings, by recruiting women for Rackspace jobs, and putting myself out there constantly, which is not always comfortable but it is rewarding.

How about your perspective here? Where will you start and when? Let’s take these first steps towards inviting more women to join our open source cloud computing efforts.

OpenStack Announces Diablo Release

We are pleased to announce Diablo, the fourth release of OpenStack.  In the 6 months since the Cactus release, we have seen the OpenStack community grow to over 1500 people and 110 member companies, and a great increase in the number of production deployments across the globe.  OpenStack continues to mature and build upon the momentum of the previous Cactus release.

This release marks the first 6 month release cycle of OpenStack.  The next release, Essex, will also be a 6 month release cycle and development is now officially underway. While Diablo includes over 70 new features, the theme is scalability, availability, and stability.

Feature Highlights

We’ll discuss briefly some of the new features among the core OpenStack projects.  For a complete list, please see the official release notes.

OpenStack Compute (Nova)

  • Distributed scheduling across zones, allowing larger compute clusters
  • A multi-host networking mode providing higher availability in DHCP and VLAN networks
  • Ability to snapshot, clone and boot from volumes
  • Ability to pause and suspend KVM instances
  • Automated instance migration during host maintenance
  • Support for Virtual Storage Arrays

OpenStack Storage (Swift)

  • Multi cluster synchronization, allowing replication to multiple geographical locations on a container by container basis.
  • Initial release of Swift Recon, middleware that allows monitoring of storage nodes and processes
  • Ability to report statistics at the container level

OpenStack Image Service (Glance)

  • New filtering and searching capabilities, simplifying management of large image stores
  • Ability to share images between tenants
  • Delayed deletion of images for increased performance

New Projects Overview

During the course of the Diablo release, there was quite about of activity around new and existing projects in the OpenStack eco-system that are poised to take on more prominent roles as of the Diablo release.  Two such projects, Identity and Dashboard, were promoted to core status during the Diablo cycle, meaning they will be officially supported projects as of Essex.  Quantum, OpenStack’s network as a service project, is in incubation status for Essex.  To learn more about the project incubation and core status promotion polices, please see The New Project Process.

OpenStack Identity (Core)

OpenStack Identity (code-named Keystone) provides identity management services across all OpenStack projects via a simple token based authentication system.  This will enable those running OpenStack in their environments to authenticate against their existing identity management systems.  Keystone will provide an abstract interface to a number of identity systems and support is planned for LDAP, ActiveDirectory, SAML, and OAuth.

OpenStack Dashboard (Core)

The Dashboard project provides a modular web based user interface to OpenStack.  This project highlights the functionality within OpenStack while providing a pluggable architecture that makes it easy for companies building products and services that extend OpenStack’s core functionality to integrate with the platform.  Today it provides both end users and administrators way to visualize and manage virtual infrastructure, quotas, object storage, network and security resources, and more.

OpenStack Quantum (Incubation)

Quantum is OpenStack’s network as a service solution, enabling advanced network topologies beyond what is possible today in Nova’s existing networking models.  Quantum will provide support for layer 2 over layer 3 tunneling to avoid the limitations of VLANs, as well as rate limiting and quality of service guarantees.  It will also provide support for monitoring protocols like NetFlow.

Upcoming for the Essex Release

Congratulations to everyone that helped Diablo come together!  Up next is the OpenStack Design Summit in Boston.  We will be planning the features and progress made during the last 6 months, and looking forward to the next 6 months of the Essex release.  Join us October 3rd through 7th at the Boston Intercontinental Hotel!

Waiting list for the Design Summit

The first 200 open seats for the Essex Design Summit were registered in less than 9 days. This event, geared towards existing and prospective OpenStack developers, is separate from the OpenStack Conference and has a more limited capacity. For the last 50 seats, the Design Summit organization committee needs to give priority to OpenStack core developers and well-known contributors that missed the boat, so we set up a waiting list.

This list will be reviewed regularly, and people that are confirmed will receive an email. The last 50 seats will be gradually assigned over the next weeks. If you want to make sure to have a seat at the OpenStack Conference, you should probably register for that event separately, since there can be no guarantee you will get one of the last Essex Design Summit seats. Sign up early on the Summit waiting list to increase your chances !

Developer Weekly (August 12)

Many people have asked for more insight into the developer activities for OpenStack as the large number of code changes and proposals make it difficult to monitor everything happening. In hopes of exposing more of the developer activities, I plan to post a weekly or biweekly blog post on the latest development activities. If you have any ideas for this blog post, please email me at stephen.spector@openstack.org. I am always ready to listen to the community for new ideas.

Activities

Developer Mailing List (archive: https://lists.launchpad.net/openstack/)

This is select list of topics discussed this week in the developer mailing list and is not a complete list.  Please visit the archive to see all the topics.

  • Tenants and Service Relationship… - Liem Manh Ngueyn asks “can I have a tenant associated with the “swift” service in Region X and another “swift” service in Region Y?” Yogeshwar Srikrishnan replies that Keystone would have different endpoint_template for each of those regions and provides and example.
  • Monitoring RabbitMQ Messages – Joshua Harlow asks if there is a tool to see all the messages passing thru rabbitmq. Craig Vyvial suggested changing the config options for rabbitmq (http://www.rabbitmq.com/management.html#configuration). Narayan Desai suggested using rabbitmqctl list_queues to see what the queue depth for each NOVA service was.
  • Problems connecting Dashboard and Nova – Mauricio Arango submitted the error information when the Dashboard fails to connect to Nova. Several developers offered various ideas to solve the problem – Mark Gius, Rafael Duran Castaneda, Joseph Heck, Arvind Somya, Vand ish Ishaya . The complete flow of ideas and responses is at https://lists.launchpad.net/openstack/msg03456.html.

Statistics

For the latest on development activities on OpenStack please check these sites for more details:

Developer Weekly Activity Review – August 5, 2011

Many people have asked for more insight into the developer activities for OpenStack as the large number of code changes and proposals make it difficult to monitor everything happening. In hopes of exposing more of the developer activities, I plan to post a weekly or biweekly blog post on the latest development activities. If you have any ideas for this blog post, please email me at stephen.spector@openstack.org. I am always ready to listen to the community for new ideas.

Activities

Developer Mailing List (archive: https://lists.launchpad.net/openstack/)

This is select list of topics discussed this week in the developer mailing list and is not a complete list.  Please visit the archive to see all the topics.

  • Dashboard newbie question – Joshua Harlow asks about getting the Dashboard to work with the latest version of Nova; he is getting a 400 https response when deploying from the dashboard. Devin Carlen recommended using the Diablo-3 milestone release. Thierry Carrez asked that any issues be added to the bug report, which Carlo Impagliazzo listed for, but 820972. This issue is still being discussed at https://lists.launchpad.net/openstack/msg03430.html.
  • [Gerrit] Getting “invalid author” - Yuriy Taraday indicated that he is trying to use Gerrit to propose changes to Keystone. He is following the instructions from the wiki and is getting “invalid author” errors. James Blair indicated that the issue was due to how Launchpad was importing account information and made a change, which solved the initial problem seen by Yuriy. Yuriy then indicated that he is seeing a “missing Change-ID in commit message” error. Several suggestions were presented as solutions. More details on those options at https://lists.launchpad.net/openstack/msg03421.html.
  • Questions About test_localization.py – Zed Shaw indicated that he is going to clean up tests and source files and has questions about the tests/test_localization.py file and what its purpose is. The discussion on  the purpose of the test and some ways to beef up what is tested were discussed.

Statistics

For the latest on development activities on OpenStack please check these sites for more details:

Building and Maintaining Image Templates for the Cloud

As IaaS platforms like OpenStack gain traction in delivering compute and storage resources on demand, we’re seeing telco and enterprise IT customers increasingly focus on “software on demand”. Typically existing software delivery processes are too lengthy to take full advantage of the “instant-on” nature of the cloud. End users need to be able choose and instantly provision software and applications, then decommission them when no longer required. Consequently this raises many questions: how do I get apps onto an IaaS platform? how do I ensure software governance if somebody else is providing the apps as part of an app store? how do I build and maintain software images through time, making cloud deployments predictable and consistent?

Today, software images are often built manually, making them difficult to update and maintain over time. Cloud users are now realizing the need to work with transparent image templates which enable them to trace individual software components, versions and licenses. They are combining these templates with automated software delivery processes, using APIs to industrialize image creation and maintenance. This enables them to easily track components, add or update software automatically, and generate to one or many clouds. The image remains consistent and predictable, whether it’s used only within a private cloud or as part of a hybrid model for bursting into Amazon, for example.

Customers can take different approaches to building and deploying these images. Firstly, the devops model uses a base image that contains only basic packages to boot the OS. Once it’s installed, a phone home feature enables the devops platform to install all the packages that a particular service needs. This is a great model for many customers. However, others are realizing that it takes a long time to stand up the service, if you’re installing say 400 VMs from an outside repository. In a cloud model, you’re also paying for bandwidth and other resources over that time.

Secondly, customers can use more complete images that include not only the OS packages but also middleware and applications. They can then combine these with a devops platform for configuration, which is a very flexible way to push configuration information without some of the disadvantages we discussed earlier.

Finally, there are “fully baked”, self-contained images that include all the software components, as well as configuration logic. These images can be used to turn a specific solution on within a private or public cloud without a great deal of expertise and are often used by ISVs for quickly ramping up POCs, for example, at a customer site.


Whichever approach you take, it’s essential to remember that all three approaches require to control and maintain the base image over time. You must be able to track the packages and components you’re using, even in a devops base image, otherwise you’ll quickly end up with scores of unmanageable images.

Easy traceability and maintenance will also help you transition to the next phase of cloud software deployments: moving from a monolithic image for small deployments or test purposes to multi-tier images for pre-production and production deployments on a larger scale. You’ll be able to more easily piece together multiple VMs, provision, maintain and decommission complete solutions over multiple cloud nodes.

James Weir

CTO, UShareSoft

High Level Nova Architecture Review – Flags and Services

Joseph Heck has written an excellent blog post on the Nova high level architecture for flags and services. Here is the intro:

I’ve been doing a lot of spelunking into the nova codebase, digging around and trying to learn some of the under pinnings. Some of these pieces were a bit confusing to me, so I’m stashing them up here for Google to find and share with others in the future.

Before I dive into the gritty details, it’s worth getting a high level overview so that some of this (hopefully) makes sense. OpenStack’s service architecture is made up of services that all talk with each other to get things done. nova-network, nova-scheduler, etc. There’s a lot of underpinning in the nova codebase to make those services relatively easy to write and work together – I was mostly curious about how they passed messages back and forth. As I dove in, the two pieces that stood out as needing to be understood first were the unified service framework in nova and configuration using flags (which it heavily depends upon).

Rest of Post

Developer Activity Review (June 17 – 24)

Many people have asked for more insight into the developer activities for OpenStack as the large number of code changes and proposals make it difficult to monitor everything happening. In hopes of exposing more of the developer activities, I plan to post a weekly or biweekly blog post on the latest development activities. If you have any ideas for this blog post, please email me at stephen.spector@openstack.org. I am always ready to listen to the community for new ideas.

Activities

Developer Mailing List (archive: https://lists.launchpad.net/openstack/)

This is select list of topics discussed this week in the developer mailing list and is not a complete list.  Please visit the archive to see all the topics.

  • Feedback on Portable Configuration Drive Blueprint – Thorsten von Eicken provided feedback to the Configuration Drive blueprint at https://blueprints.launchpad.net/nova/+spec/configuration-drive.  Scott Moser, Paul Voccio (blueprint author), and Vish Ishaya provided feedback and discussion
  • Swift+Keystone error – Tres Henry reported issues in trying to get a Swift+Keystone development environment setup. Jesse Andrews responded with an updated Keystone GitHub link. Todd Wiley responded with a proxy-server.conf setting to try; account_autocreate = true.  Tres responded with the error showing up even after trying this configuration file setting.

Statistics

For the latest on development activities on OpenStack please check these sites for more details:

Back to top