Tag: Diablo


OpenStack Deployments Abound at Austin Meetup (12/9)

December 21st, 2011 — 11:58am
I (Rob Hirschfeld) was very impressed by the quality of discussion at the Deployment topic meeting for Austin OpenStack Meetup (#OSATX). Of the 45ish people attending, we had representations for at least 6 different OpenStack deployments (my employeer Dell, HP, ATT, Rackspace Internal, Rackspace Cloud Builders, Opscode Chef)!  Considering the scope of those deployments (several are aiming at 1000+ nodes), that’s a truly impressive accomplishment for such a young project.

Figure 1 Diablo Software Architecture. Source Dell/OpenStack (cc w/ attribution)

 

Even with the depth of the discussion (notes below), we did not go into details on how individual OpenStack components are connected together.  The image my team at Dell uses is included below.  I also recommend reviewing Rackspace’s published reference architecture.

Notes

Our deployment discussion was a round table so it is difficult to link statements back to individuals, but I was able to track companies (mostly).

  • HP
    • picked Ubuntu & KVM because they were the most vetted. They are also using Chef for deployment.
    • running Diablo 2, moving to Diablo Final & a flat network model. The network controller is a bottleneck. Their biggest scale issue is RabbitMQ.
    • is creating their own Nova Volume plugin for their block storage.
    • At this point, scale limits are due to simultaneous loading rather than total number of nodes.
    • The Nova node image cache can get corrupted without any notification or way to force a refresh – this defect is being addressed in Essex.
    • has setup availability zones are completely independent (500 node) systems. Expecting to converge them in the future.
  • Rackspace
    • is using the latest Ubuntu. Always stays current.
    • using Puppet to setup their cloud.
    • They are expecting to go live on Essex and are keeping their deployment on the Essex trunk. This is causing some extra work but they expect it to pay back by allowing them to get to production on Essex faster.
    • Deploying on XenServer
    • “Devs move fast, Ops not so much.”  Trying to not get behind.
  • Rackspace Cloud Builders (RCB) is running major releases being run through an automated test suite. The verified releases are being published to https://github.com/cloudbuilders (note: Crowbar is pulling our OpenStack bits from this repo).
  • Dell commented that our customers are using Crowbar primarily pilots – they are learning how to use OpenStack
    • Said they have >10 customer deployments pending
    • ATT is using OpenSource version of Crowbar
    • Need for Keystone and Dashboard were considered essential additions to Diablo
  • Hypervisors
    • KVM is considered the top one for now
    • Libvirt (which uses KVM) also supports LXE which people found to be interesting
    • XenServer via XAPI are also popular
    • No so much activity on ESX & HyperV
    • We talked about why some hypervisors are more popular – it’s about the node agent architecture of OpenStack.
  • Storage
    • NetApp via Nova Volume appears to be a popular block storage
  • Keystone / Dashboard
    • Customers want both together
    • Including keystone/dashboard was considered essential in Diablo. It was part of the reason why Diablo Final was delayed.
    • HP is not using dashboard
  • OpenStack API
    • Members of the Audience made comments that we need to deprecate the EC2 APIs (because it does not help OpenStack long term to maintain EC2 APIs over its own).  [1/5 Note: THIS IS NOT OFFICIAL POLICY, it is a reflection of what was discussed]
    • HP started on EC2 API but is moving to the OpenStack API

Austin Meetup Housekeeping

  • Next meeting is Tuesday 1/10 and sponsored by SUSE (note: Tuesday is just for this January).  Topic TBD.
  • We’ve got sponsors for the next SIX meetups! Thanks for Dell (my employeer), Rackspace, HP, SUSE, Canonical and PuppetLabs for sponsoring.
  • We discussed topics for the next meetings.  We’re going to throw it to a vote for guidance.

 

Comment » | Meetup

Developer Weekly (August 12)

August 12th, 2011 — 4:57pm

Many people have asked for more insight into the developer activities for OpenStack as the large number of code changes and proposals make it difficult to monitor everything happening. In hopes of exposing more of the developer activities, I plan to post a weekly or biweekly blog post on the latest development activities. If you have any ideas for this blog post, please email me at [email protected]. I am always ready to listen to the community for new ideas.

Activities

Developer Mailing List (archive: https://lists.launchpad.net/openstack/)

This is select list of topics discussed this week in the developer mailing list and is not a complete list.  Please visit the archive to see all the topics.

  • Tenants and Service Relationship… - Liem Manh Ngueyn asks “can I have a tenant associated with the “swift” service in Region X and another “swift” service in Region Y?” Yogeshwar Srikrishnan replies that Keystone would have different endpoint_template for each of those regions and provides and example.
  • Monitoring RabbitMQ Messages – Joshua Harlow asks if there is a tool to see all the messages passing thru rabbitmq. Craig Vyvial suggested changing the config options for rabbitmq (http://www.rabbitmq.com/management.html#configuration). Narayan Desai suggested using rabbitmqctl list_queues to see what the queue depth for each NOVA service was.
  • Problems connecting Dashboard and Nova – Mauricio Arango submitted the error information when the Dashboard fails to connect to Nova. Several developers offered various ideas to solve the problem – Mark Gius, Rafael Duran Castaneda, Joseph Heck, Arvind Somya, Vand ish Ishaya . The complete flow of ideas and responses is at https://lists.launchpad.net/openstack/msg03456.html.

Statistics

For the latest on development activities on OpenStack please check these sites for more details:

Comment » | Communication, Development, Governance, Newsletter

Developer Weekly Activity Review – August 5, 2011

August 5th, 2011 — 7:22pm

Many people have asked for more insight into the developer activities for OpenStack as the large number of code changes and proposals make it difficult to monitor everything happening. In hopes of exposing more of the developer activities, I plan to post a weekly or biweekly blog post on the latest development activities. If you have any ideas for this blog post, please email me at [email protected]. I am always ready to listen to the community for new ideas.

Activities

Developer Mailing List (archive: https://lists.launchpad.net/openstack/)

This is select list of topics discussed this week in the developer mailing list and is not a complete list.  Please visit the archive to see all the topics.

  • Dashboard newbie question – Joshua Harlow asks about getting the Dashboard to work with the latest version of Nova; he is getting a 400 https response when deploying from the dashboard. Devin Carlen recommended using the Diablo-3 milestone release. Thierry Carrez asked that any issues be added to the bug report, which Carlo Impagliazzo listed for, but 820972. This issue is still being discussed at https://lists.launchpad.net/openstack/msg03430.html.
  • [Gerrit] Getting “invalid author” - Yuriy Taraday indicated that he is trying to use Gerrit to propose changes to Keystone. He is following the instructions from the wiki and is getting “invalid author” errors. James Blair indicated that the issue was due to how Launchpad was importing account information and made a change, which solved the initial problem seen by Yuriy. Yuriy then indicated that he is seeing a “missing Change-ID in commit message” error. Several suggestions were presented as solutions. More details on those options at https://lists.launchpad.net/openstack/msg03421.html.
  • Questions About test_localization.py – Zed Shaw indicated that he is going to clean up tests and source files and has questions about the tests/test_localization.py file and what its purpose is. The discussion on  the purpose of the test and some ways to beef up what is tested were discussed.

Statistics

For the latest on development activities on OpenStack please check these sites for more details:

1 comment » | Communication, Development, Newsletter

Community Weekly Newsletter (May 27 – June 3)

June 3rd, 2011 — 10:03am

OpenStack Community Newsletter – June 3, 2011

This weekly newsletter is a way for the community to learn about all the various activities occurring on a weekly basis. If you would like to add content to a weekly update or have an idea about this newsletter, please email [email protected].

Paul Pettigrew, Mach Technology from OpenStack on Vimeo.

HIGHLIGHTS

EVENTS

DEVELOPER COMMUNITY

GENERAL COMMUNITY

COMMUNITY STATISTICS (5/27– 6/2)

  • Data Tracking Graphs – http://wiki.openstack.org/WeeklyNewsletter
  • OpenStack Compute (NOVA) Data
    • 12 Active Reviews
    • 256 Active Branches – owned by 72 people & 15 teams
    • 1344 commits by 65 people in last month
  • OpenStack Object Storage (SWIFT) Data
    • 1 Active Reviews
    • 68 Active Branches – owned by 22 people & 6 teams
    • 137 commits by 15 people in last month
  • OpenStack Image Registry (GLANCE) Data
    • 2 Active Reviews
    • 21 Active Branches – owned by 6 people & 5 teams
    • 84 commits by 8 people in last month
  • Twitter Stats for Week:  #openstack 214 total tweets; OpenStack 1064 total tweets  (does not include RT)
  • Bugs Stats for Week: 385 Tracked Bugs; 71 New Bugs; 38 In-process Bugs; 0 Critical Bugs; 21 High Importance Bugs; 255 Bugs (Fix Committed)
  • Blueprints Stats for Week:  205 Blueprints; 13 Essential, 16 High, 20 Medium, 26 Low, 130 Undefined
  • OpenStack Website Stats for Week:  12,896 Visits, 35,054 Pageviews, 50.28% New Visits
    • Top 5 Pages: Home 39.26%; /projects 11.55%; /projects/compute 17.08%; /projects/storage 11.06%; /community 7.18%

OPENSTACK IN THE NEWS

1 comment » | Communication, community, Newsletter

Developer Activity Review – May 20

May 20th, 2011 — 3:53pm

Many people have asked for more insight into the developer activities for OpenStack as the large number of code changes and proposals make it difficult to monitor everything happening. In hopes of exposing more of the developer activities, I plan to post a weekly or biweekly blog post on the latest development activities. If you have any ideas for this blog post, please email me at [email protected]. I am always ready to listen to the community for new ideas.

Activities

Developer Mailing List (archive: https://lists.launchpad.net/openstack/)

This is select list of topics discussed this week in the developer mailing list and is not a complete list.  Please visit the archive to see all the topics.

  • Graphical Console for VMs w/o Network Stacks – Donal Lafferty is looking for a blueprint for a graphical console instead of text console. Mike Scherbakov replied with instructions for using the current Dashboard.
  • Unassigned Essential Diablo Specs – Thierry Carrez listed a set of blueprints that are planned for Diablo but currently have no owner. The list is at https://lists.launchpad.net/openstack/msg02432.html.
  • Python-novaclient vs. python-openstack.compute - Soren Hansen brings up the issue that we have 2 distinct but similar libraries doing the same thing; Sandy Walsh, Dan Prince, Gabe Westmaas, Ed Leafe, and Josh Kearney all joined in the discussion of how to handle this issue.
  • Global deployment of Glance – Glen Campbell raises the issue of having replicas of Glance in multiple installations and how to maintain synch. Jay Pipes, Chris Behrens, and Eric Day discussed the idea of using SWIFT to manage the multiple zones issue and offload the problem from Glance.

Statistics

For the latest on development activities on OpenStack please check these sites for more details:

Comment » | Communication, Development, Newsletter

OpenStack Announces Cactus Release

April 15th, 2011 — 10:31am

With the availability of the Cactus release of OpenStack today the momentum and progress of the project continues to grow. A tremendous amount of effort and contribution from the large, and growing, community has added significant features, fixed a lot of bugs, and debated and discussed many technical issues. I am impressed with the progress that has been made since the Bexar release just 10 weeks ago and believe the projects and code are tracking to fill the promise of being the ubiquitous, open source cloud solution.

New features in Nova (OpenStack Compute) include:

  • Two additional virtualization technologies: LXC containers and VMWare/vSphere ESX / ESXi 4.1, Update 1. Driven by a common compute control infrastructure (Nova) this brings the options for OpenStack host virtualization to 8 (adding to Microsoft Hyper-V, KVM, QEMU, UML, Xen, and Citrix XenServer).
  • Live Migration support for KVM-based systems landed in the Cactus release; it is now possible to move running VMs from one physical host to another without a shut down.
  • Lots of new features were added to XenServer support: network and file injection, IPv6 support, instance resize and rescue, network QoS, and VM instance parameters.
  • The OpenStack Compute API version 1.0 is available, with the OpenStack Compute API version 1.1 marked as “experimental” for Cactus. The intent is to finalize the 1.1 API at the Diablo design summit and have it complete and stable in the Diablo release. Multi-tenant accounting support was added to OpenStack API, allowing multiple accounts (projects) and admin API access to create accounts & users.
  • The OpenStack Compute API version 1.1 supports a standardized extension mechanism, this allow developers to innovate more quickly by adding extensions to their local OpenStack installations ahead of the code being accepted by the OpenStack community as a whole;
  • Nova can now start instances from VHD images that include customer data and kernel in one unified image.
  • Volume backend support has been enhanced; Nova now supports volumes residing on HP SANs and Solaris iSCSI devices.
  • Continued work on feature uniformity and parity across network types and hypervisors; IPv6 is now supported in all network modes, including FlatManager and VlanNetworkManager. Basic network injection is now supported under XenAPI.
  • Multi-cluster region support, which allows administrators to manage servers in clusters, and create fault zones and availability zones.

New features in Glance (OpenStack Image Registry and Delivery) include:

  • New command line interface tool (aptly-named “glance”) that allows direct access to Glance services through the API.
  • Support for multiple image formats through a new disk_format and container_format metadata definition.
  • Uploaded images can now be verified against a client-provided checksum, to ensure the integrity of the transfer.

New features in Swift (OpenStack Object Storage) include:

  • The option to serve static website content directly from a Swift installation using container listings in index.html displays. Swift will automatically translate requests to possible /index.html resolutions, where the index.html display is configurable per container.
  • To more quickly detect errors for often-served files, Swift now performs content checksum validation during object GET actions.
  • Performance of many request types has been improved through a refactoring of the Swift Proxy Server.
  • To avoid slowdowns for common operations when deleted items build up over time, Swift now has improved indexing of the SQLite databases for account and container listing and tracking.
  • An enhanced authentication system (SWauth) is available.
  • The ability to collect and serve data that enables integration of service provider billing solutions or internal chargebacks.

In addition to the work done on the project code, there have been several other things happening to improve the state of OpenStack. Primary amongst these was the election of Project Team Leaders for the three current OpenStack projects… Congratulations to Vish Ishaya (vishy) [Nova], John Dickinson (notmyname) [Swift], and Jay Pipes (jaypipes) [Glance] as new PTL’s, they also join the OpenStack Project Policy Board.

The OpenStack Project Policy Board also had elections, with 5 board members holding elected seats. These are Thierry Carrez (ttx), Rick Clark (dendrobates), Eric Day (eday), Soren Hansen (soren), and Ewan Mellor (ewanmellor). Congratulations folks!

OpenStack has defined a process for bringing in new projects, both as core projects and those that are being incubated.  (See http://wiki.openstack.org/Governance/Approved/NewProjectProcess). The initial incubation project is “Burrow”, a simple queuing service for OpenStack being led by Eric Day (eday). At the upcoming Diablo Design Summit I expect several more projects to be proposed for incubation; including Load Balancing and Database Services.

The Diablo Design Summit is setting up to be the most dynamic and content-filled summit to date! The entire week is completely filled with attendees and items for discussion. While ttx and the PTL’s are busy scheduling all the sessions here are a few of the highlights:

  • Network as a Service. In order to fulfill the vision of OpenStack as a secure cloud infrastructure with the ability to federate across clouds it is imperative that the underlying network support isolation, federation, and the ability to manage these topologies. The NaaS discussion has many important participants working hard to collaborate on this very technical set of issues.
  • Volume services. Extending the initial Nova volume management for richer block storage solutions.
  • Additional machine types (GPU accelerators, larger multi-core processor systems).
  • Consistent authentication and authorization across OpenStack projects.
  • Multi-zone support, intra-data center and federation across data centers.
  • Project management discussions.
  • Stability and QA automation. A key theme of the Diablo release will be to automate the build and test infrastructure for OpenStack to ensure that trunk is always runnable. With the proliferation of virtualization architectures, machine architectures, and service options this will be a key element to success of the project.
  • Complete and stable OpenStack version 1.1 API.
  • Target large scale service provider deployments, with proof of concepts happening in large OpenStack contributor sites.

A job well done to all of the folks that contributed and made the Cactus release come together and get released. I will see all of you at the Design Summit in Santa Clara and look forward to the discussions around the Diablo release and the future of OpenStack!

Lastly, the OpenStack Project Team Leaders are hosting a Webinar on Tuesday April 19th at 3:00 pm CST. More information at http://www.openstack.org/blog/2011/04/openstack-cactus-webinar/.

John
Director, OpenStack@Rackspace

32 comments » | Communication, Development

OpenStack Developer Activity Review (January 21 – 28)

January 28th, 2011 — 2:21pm

Many people have asked for more insight into the developer activities for OpenStack as the large number of code changes and proposals make it difficult to monitor everything happening. In hopes of exposing more of the developer activities, I plan to post a weekly or biweekly blog post on the latest development activities. If you have any ideas for this blog post, please email me at [email protected]. I am always ready to listen to the community for new ideas.

Activities

Developer Mailing List (archive: https://lists.launchpad.net/openstack/)

  • State of Glance? -  Jay Pipes gives an overview of all features developed for Bexar in Project Glance as well as the current thinking for the Cactus Project Glance release. More details at https://lists.launchpad.net/openstack/msg00358.html
  • Network Service for L2/L3 Network Infrastructure Blueprint – Ewan Mellor responded to interest from several developers on the status of the bexar-network-service blueprint and its status. The blueprint will be postponed to Cactus and a discussion about this blueprint was started.
  • Cactus Release Preparation – Rick Clark gives an overview of the strategy behind the Cactus release and requests that all new features for Cactus be proposed to the community by February 3, 2011. The Cactus release schedule is http://wiki.openstack.org/CactusReleaseSchedule

Statistics

For the latest on development activities on OpenStack please check these sites for more details:

1 comment » | Communication, Development, Newsletter

Back to top