The OpenStack Blog

Author Archive

An introductory tour of OpenStack Cloud Messaging as a Service

Post by Mark Atwood, Director of OpenSource Engineering for HP Cloud

5.23.13-Cloud-Messaging

The need for well understood intra-application messaging was one of the signs that new application design patterns beyond just the LAMP stack were needed.

The need for an OpenStack Messaging Service was recognized by the San Diego Grizzly Summit, and in an unconference track design meeting on the last day, a crowd of interested people met, talked out some requirements, and from that the OpenStack Message Bus project was born. It was codenamed “Marconi”, in honor of Guglielmo Marconi, the inventor of wireless messaging. The Marconi team collaborates on Launchpad.

At the same time, HP was looking for a way to make it’s already under development application messaging service available to users of the HP Cloud. When the Marconi project appeared, HP decided that instead of splitting the developer community in a pointless API standards war, it made more sense to clone and track the public Marconi API, and then contribute to open source Marconi project itself.

As part of the development process, HP is running the messaging service in “developer preview”, meaning that you have to ask to be enrolled in the preview program to access it, and neither the service nor the API are stable. Oh, and right now it is free of charge.

Log into your HP Cloud account, and then go to the URL https://account.hpcloud.com/services and look for “Beta Services” and then “Request Access” to “Messaging”. Your request will be reviewed, and then activated. This can potentially take a couple of days, because there are human beings in this decision loop.

All parts of OpenStack and HP Cloud are controlled via RESTful APIs, and Messaging is no exception. We will access the raw API with the cURL command line tool. Using a specialized client tool can be simpler to use, and such tools are under development. However, walking this process step by step the first time helps build an understand of how all the parts work.

First thing we need is the API credentials to your HP Cloud account.

After logging into your HP Cloud account, go to the URL https://account.hpcloud.com/account/api_keys and look for following information:

  • “Project ID”, something like “58345815996918″
  • “Access Key #1″, something like “2DJ3Z58RZB5JJ7V2JKS3″
  • “Show Secret Key”. When you click on it, it will reveal something like “bPH13haenb/DlvR1th+u4Uj5ehvYKsF7ApYact6i”
  • The URL for “Identity” “region-a.geo-1″. It will be something like https://region-a.geo-1.identity.hpcloudsvc.com:35357/v2.0/
  • The URL for “Messaging” “region-a.geo-1″. It will be something like https://region-a.geo-1.messaging.hpcloudsvc.com/v1.1/58345815996918

Now open a text editor, and construct a file named keystone-req.json with JSON contents like the following, only with your own access key, secret key, and tenant id (aka “project id”).

{
"auth": {
"apiAccessKeyCredentials": {
"accessKey": "2DJ3Z58RZB5JJ7V2JKS3",
"secretKey": "bPH13haenb/DlvR1th+u4Uj5ehvYKsF7ApYact6i"
},
"tenantId": "58345815996918"
}
}

Now, at the command prompt, run the following cURL command. Notice that the URL is the one for “Identity” “region-a.geo-1″ we looked up earlier, with the path part “/tokens” appended to it.

curl -X POST -H "Content-Type: application/json" \
https://region-a.geo-1.identity.hpcloudsvc.com:35357/v2.0/tokens \
-d @keystone-req.json \
| python -mjson.tool > keystone-rsp.json
Notice that we pipe the output through "python -mjson.tool". This is a useful trick for prettyprinting JSON to make it more readable.

Open up the resulting file keystone-rsp.json in a text editor, and take a look.

Search down for the string “token”, and you should see something lilke

"token": {
"expires": "2013-05-21T12:38:16.962Z",
"id": "HPAuth10_3e1c44e527f0e64387ff0705e1b09d0ca3ef47c47a6c6afe1738d77f896b4f20",
"tenant": {
"id": "58345815996918",
"name": "[email protected]"
}
},

The important part is the “access.token.id” value. We are going to paste that string into the HTTP X-Auth header for the next few cURL commands.

Now search the file for the string “hpext:messaging”, and you will find a block of JSON that looks like this:

{
"endpoints": [
{
"publicURL": "https://region-a.geo-1.messaging.hpcloudsvc.com/v1.1/58345815996918",
"publicURL2": "",
"region": "region-a.geo-1",
"tenantId": "58345815996918",
"versionId": "1.1",
"versionInfo": "https://region-a.geo-1.messaging.hpcloudsvc.com/v1.1",
"versionList": "https://region-a.geo-1.messaging.hpcloudsvc.com"
}
],
"name": "Messaging",
"type": "hpext:messaging"
},

The “publicURL” is the URL we can use to issue REST commands to the Messaging service. Here is how we list all the queues we can use. Notice that we pass the keystone authentication token in on the X-Auth-Token header.

curl -X GET -H "Content-Type: application/json" \
-H "X-Auth-Token: HPAuth10_3e1c44e527f0e64387ff0705e1b09d0ca3ef47c47a6c6afe1738d77f896b4f20" \
https://region-a.geo-1.messaging.hpcloudsvc.com/v1.1/58345815996918/queues \
| python -mjson.tool

We will probably see

{
"queues": []
}

which means, logically enough, no queues have been created.

Let’s create one. We change the method to PUT and append queues/foo to create a queue named “foo”. There is no need to use the JSON prettyprinter, because no request body will be returned.

curl -X PUT -H "Content-Type: application/json" \
-H "X-Auth-Token: HPAuth10_3e1c44e527f0e64387ff0705e1b09d0ca3ef47c47a6c6afe1738d77f896b4f20" \

https://region-a.geo-1.messaging.hpcloudsvc.com/v1.1/58345815996918/queues/foo

And then let’s again list all the queues.

curl -X GET -H "Content-Type: application/json" \
-H "X-Auth-Token: HPAuth10_3e1c44e527f0e64387ff0705e1b09d0ca3ef47c47a6c6afe1738d77f896b4f20" \
https://region-a.geo-1.messaging.hpcloudsvc.com/v1.1/58345815996918/queues \
| python -mjson.tool

which returns

{
"queues": [
{
"name": "foo"
}
]
}

Look at that that! A queue named foo.

Now let’s put something on that foo queue. We do that by POSTing some JSON to the queue URL with the path part “/messages” appended, like so:

curl -X POST -H "Content-Type: application/json" \
-H "X-Auth-Token: HPAuth10_3e1c44e527f0e64387ff0705e1b09d0ca3ef47c47a6c6afe1738d77f896b4f20" \
https://region-a.geo-1.messaging.hpcloudsvc.com/v1.1/58345815996918/queues/foo/messages \
-d '{ "body": "Hello World!" }'

And then read it back.

curl -X GET -H "Content-Type: application/json" \
-H "X-Auth-Token: HPAuth10_3e1c44e527f0e64387ff0705e1b09d0ca3ef47c47a6c6afe1738d77f896b4f20" \

https://region-a.geo-1.messaging.hpcloudsvc.com/v1.1/58345815996918/queues/foo/messages

which will remove the item and display

{"id":"b41ea44e-3962-4010-80ad-4f87dcd6ba8d","body":"Hello World!"}
And then let’s delete the queue

curl -X DELETE \
-H "X-Auth-Token: HPAuth10_3e1c44e527f0e64387ff0705e1b09d0ca3ef47c47a6c6afe1738d77f896b4f20" \

https://region-a.geo-1.messaging.hpcloudsvc.com/v1.1/58345815996918/queues/foo

As always, if you have any questions, send us an email. You can contact me directly at [email protected]

..m

OpenStack Conference Registration – Are you Coming?

The OpenStack community is once again meeting in less than a month in Boston, MA to educate, discuss, and plan for the future of cloud computing. The community is asking for cloud computing enthusiasts, open source gurus, IT leaders, and technologists to take part in our open discussion. The event is held at the Boston Intercontinental Hotel from October 5 – 7, 2011 with industry leaders from Dell, HP, Nebula, NetApp, Canonical, NTT, Rackspace, DreamHost, CloudScaling, RightScale, MPStor, SolidFire, and Piston to name just a few. Registration is now open with discounted hotel rooms available until September 16.

The OpenStack Design Summit, held three days prior to the OpenStack Conference is the community gathering of developers putting together the features and architecture for the next community release, Essex. This event is now sold out and the waiting list on Launchpad is being closed on Tuesday, September 13 at Noon GMT.

Rackspace Running OpenStack Compute Today for Select Customers, Discusses Broader Rollout Plans

Lew Moorman, President of the Rackspace Cloud, recently discussed the state of Rackspace’s OpenStack plans in a blog post.

In addition to running OpenStack Object Storage (code named Swift) for over a year to power their cloud, Rackspace is now running OpenStack Compute (code named Nova) for select customers. Head over to the Rackspace Cloud blog to learn more.

They also recently announced dates for training classes on OpenStack, and will be in full attendance at the OpenStack Conference in Boston next month.

OpenStack Conference – Agenda Highlights

The Program Committee is in its final stages of crafting the OpenStack Conference agenda and I wanted to share some highlights from the agenda to get you ready for next week’s formal agenda announcement…

Highlights:

  • Keynote Speakers from HP, Dell, Nebula, Canonical, NetApp, and NTT
  • User Case Studies from CERN and Acens Technologies (more being planned)
  • Panel Discussions on OpenStack Governance, OpenStack “Distributions”, and Service Providers
  • Business and Technical Sessions on all OpenStack Projects including the two new projects, Dashboard & Keystone

Be sure to register now for this gathering of OpenStack enthusiasts in Boston, MA from October 5-7, 2011

 

OpenStack Events in September

The OpenStack community has two confirmed events next month to book on your calendars (and a third in final confirmation)…

Paris, France

 

 

 

 

 

OpenStack in Action!
September 21, 2011 from 8:30 am – 5:30 pm
Current Sponsors: Enovance, UShareSoft, Canonical, Rackspace, and OW2
Event Registration: https://openstackinaction.eventbrite.com/

Boston, MA

 

 

 

 

 

Boston OpenStack User Group
September 21 from 6:00 pm – 8:00 pm
Sponsor: Fidelity
Event Registration: http://bostonopenstack.eventbrite.com

OpenStack Governance Elections – Voting Process

This Wednesday at Noon CST, the OpenStack Governance Elections Nominations process will close and all approved candidates will be entered into the Election Tool.  Elections will run from August 25 – September 4 at Noon CST.

Elections

The following four separate votes will be conducted during this election cycle:

  • NOVA Project Team Lead (1 Position)
  • SWIFT Project Team Lead (1 Position)
  • GLANCE Project Team Lead (1 Position)
  • Project Policy Board (3 Open Seats)

In addition, Eric Day has resigned his seat on the PPB with a term ending Spring 2012. To fill his seat for the next 6 months, the 4th place vote getter in the Project Policy Board election will replace Eric and complete his term. This process change simplifies having to run a special election for his seat. Please contact me if you have any questions on this process for replacing Eric on the board.

Voter Eligibility

Each election has a separate policy for who can vote in the election. A complete review of this process can be found at http://www.openstack.org/blog/2011/08/openstack-governance-elections-coming-soon/. To be eligible to vote, please ensure that you meet these requirements by Noon CST on Wednesday August 25th.

Candidates

The current list of nominees is available at http://etherpad.openstack.org/Fall2011-Nominees.  If you are interested in being a nominee, please add yourself by Wednesday August 25th by Noon CST at which time nominations will close.

Community Weekly Review (August 12 – 19)

OpenStack Community Newsletter – August 19, 2011

This weekly newsletter is a way for the community to learn about all the various activities occurring on a weekly basis. If you would like to add content to a weekly update or have an idea about this newsletter, please email [email protected].

HIGHLIGHTS

EVENTS

  • China OpenStack Launch – September 6 in Shanghai, China – In Active Development
  • Ohio LinuxFest 2011 – September 9-11, 2011 in Columbus, OH – http://www.ohiolinux.org/
  • PyTexas 2011 – September 10-11, 2011 in College Station, Texas http://pytexas.org/PyTexas2011
  • OpenStack Conference – Boston, MA – October 5-7, 2011

DEVELOPER COMMUNITY

GENERAL COMMUNITY

COMMUNITY STATISTICS (8/12– 8/18)

  • Data Tracking Graphs – http://wiki.openstack.org/WeeklyNewsletter
  • OpenStack Compute (NOVA) Data
    • 22 Active Reviews
    • 335 Active Branches – owned by 88 people & 16 teams
    • 4,004 commits by 82 people in last month
  • OpenStack Object Storage (SWIFT) Data
    • 3 Active Reviews
    • 76 Active Branches – owned by 24 people & 6 teams
    • 82 commits by 15 people in last month
  • OpenStack Image Registry (GLANCE) Data
  • Twitter Stats for Week:  #openstack 306 total tweets; OpenStack 1012 total tweets  (does not include RT)
  • Bugs Stats for Week: 745 Tracked Bugs; 97 New Bugs; 33 In-process Bugs; 12 Critical Bugs; 74 High Importance Bugs; 529 Bugs (Fix Committed)
  • Blueprints Stats for Week:  225 Blueprints; 6 Essential, 14 High, 23 Medium, 25 Low, 157 Undefined
  • OpenStack Website Stats for Week:  14,728 Visits, 35,737 Pageviews, 51.09 % New Visits
    • Top 5 Pages: Home 41.13%; /projects 13.06%; /projects/compute 16.21%; /projects/storage 10.00%; /imageserver 6.15%

Evolving Roles, Welcome Stefano Maffulli to the Community

The explosion of OpenStack over the past year has once again highlighted the significant impact that an open source community can have on an industry. Within a single year, over 100 participating companies and 1,300 community members have joined together to create the de-facto open source cloud computing standard. With this enormous growth comes a broad array of possible work opportunities that are unique in challenge and scope, and I have decided to move on from being your Community Manager to take up new projects within OpenStack. It has been a pleasure for me professionally and personally to have participated in this community as your Community Manager, and I thank you for your amazing support and efforts in driving OpenStack forward. I look forward to its continued success and welcome Stefano Maffulli as your Community Manager.

Developer Weekly (August 12)

Many people have asked for more insight into the developer activities for OpenStack as the large number of code changes and proposals make it difficult to monitor everything happening. In hopes of exposing more of the developer activities, I plan to post a weekly or biweekly blog post on the latest development activities. If you have any ideas for this blog post, please email me at [email protected]. I am always ready to listen to the community for new ideas.

Activities

Developer Mailing List (archive: https://lists.launchpad.net/openstack/)

This is select list of topics discussed this week in the developer mailing list and is not a complete list.  Please visit the archive to see all the topics.

  • Tenants and Service Relationship… - Liem Manh Ngueyn asks “can I have a tenant associated with the “swift” service in Region X and another “swift” service in Region Y?” Yogeshwar Srikrishnan replies that Keystone would have different endpoint_template for each of those regions and provides and example.
  • Monitoring RabbitMQ Messages – Joshua Harlow asks if there is a tool to see all the messages passing thru rabbitmq. Craig Vyvial suggested changing the config options for rabbitmq (http://www.rabbitmq.com/management.html#configuration). Narayan Desai suggested using rabbitmqctl list_queues to see what the queue depth for each NOVA service was.
  • Problems connecting Dashboard and Nova – Mauricio Arango submitted the error information when the Dashboard fails to connect to Nova. Several developers offered various ideas to solve the problem – Mark Gius, Rafael Duran Castaneda, Joseph Heck, Arvind Somya, Vand ish Ishaya . The complete flow of ideas and responses is at https://lists.launchpad.net/openstack/msg03456.html.

Statistics

For the latest on development activities on OpenStack please check these sites for more details:

Community Weekly Newsletter (August 5-12)

OpenStack Community Newsletter – August 12, 2011

This weekly newsletter is a way for the community to learn about all the various activities occurring on a weekly basis. If you would like to add content to a weekly update or have an idea about this newsletter, please email [email protected].

HIGHLIGHTS

EVENTS

DEVELOPER COMMUNITY

GENERAL COMMUNITY

COMMUNITY STATISTICS (8/5– 8/11)

  • Data Tracking Graphs – http://wiki.openstack.org/WeeklyNewsletter
  • OpenStack Compute (NOVA) Data
    • 22 Active Reviews
    • 335 Active Branches – owned by 88 people & 16 teams
    • 4,004 commits by  82 people in last month
  • OpenStack Object Storage (SWIFT) Data
    • 3 Active Reviews
    • 76 Active Branches – owned by 24 people & 6 teams
    • 82 commits by 15 people in last month
  • OpenStack Image Registry (GLANCE) Data
  • Twitter Stats for Week:  #openstack 298 total tweets; OpenStack  1139 total tweets  (does not include RT)
  • Bugs Stats for Week: 690 Tracked Bugs; 100 New Bugs; 41 In-process Bugs; 12 Critical Bugs; 59 High Importance Bugs; 478 Bugs (Fix Committed)
  • Blueprints Stats for Week:  222 Blueprints; 6 Essential, 17 High, 26 Medium, 25 Low, 148 Undefined
  • OpenStack Website Stats for Week:  15,446 Visits, 38,868 Pageviews, 54.88 % New Visits
    • Top 5 Pages: Home 40.52%; /projects 13.17%; /projects/compute 16.66%; /projects/storage 10.37%; /imageserver 5.95%

OPENSTACK IN THE NEWS

Back to top