Introducing the OpenStack Operations Guide

Planning to run, or design an OpenStack Cloud?  There’s a new book you should take a look at – the OpenStack Operations Guide.  Get your free download now at http://docs.openstack.org/ops/!
OpenStack Operations Guide
You may have already seen the blog post, We Did It: Zero to Book in Five Days, from Anne Gentle, the OpenStack documentation coordinator who came up with the idea to write the book sprint-style and gathered the team – including myself.
Screen Shot 2013-03-13 at 10.29.48 AM
Our overarching guidance (and why you should read this book) when writing was to share our experiences. The authors have all lost sleep and worked through holidays to nurse OpenStack clouds into a production-ready state. To that end, the book covers two main sections: Architecture and Operations.The aim of the Architecture section is to, once you’ve understood enough about OpenStack to realise that you are actually going to build a cloud using it as a basis, step you through many of the common and non-so-common concerns to deliberate while in design. Some of it might be principles you consider standard already, but others look to specifically address the hard questions specific to this platform. Questions like “How do you scale your OpenStack cloud?”, “How many servers should you buy for control infrastructure?”, “Should your storage be external or in the compute node?” are covered in an opinionated manner, with realistic perspective on what works.This book does not include installation instructions. This is already covered in the installations guides at http://docs.openstack.org/install/, so the book segues into Operations assuming that you now have a working cloud – though a pre-install skim is a good idea too.In the Operations section, we really tried to consider running OpenStack from the perspective of a time-poor systems administrator. It starts with a to-the-point tour of your cloud via the commandline as a way of familiarising yourself with what will be your working environment. Following this is a compendium detailing ways in which you’ll be working with the cloud. “User Facing Operations” attempts to give you enough of a perspective on what users are trying to do at the IaaS level that you can can help them, while “Maintenance, Failures and Debugging” works on the dual principle of fixing broken services, or preventing them from breaking in the first place.Two important chapters end the section – “Customise” and “Upstream OpenStack”. Openstack is as much of a community as it is a software project, and the latter gives some insights on how to interact with the community – including how to get help when you’ve tried everything else. The former, probably the most advanced chapter in the book, assists in taking advantage of the framework to extend it without touching the core code. Hopefully you’ll use the two in conjunction and share your work with the community!For excellent bed time reading – perhaps to help understand that others are in the same boat, and reduce those stress levels – there’s an appendix called “Tales from the Cryp^H^H^H^H Cloud”. Here, a few of us share our OpenStack operations stories to help you understand the processes we go through when the stakes are high.

It’s been a privilege working on this book for you – learning as much as contributing – and I hope you find it useful.

History behind the book:

The creation of this book happened in just five days, but the story that goes with it is a little longer. More or less since people started using OpenStack for their production clouds (let’s call that Cactus timeframe – those before that were fairly ‘brave’) there’s been a request for information on the best practices for designing and running our favourite cloud software. As it were, the small number of people working on documentation were struggling just to keep track of the hundreds of configuration options that make this framework so flexible. Then, in May last year, Anne Gentle created a seemingly innocent blueprint “Create and OpenStack Operators Manual”, and started gathering a team – people who’d spent months sacrificing their sleep to tune their OpenStack deployments, and able to share their knowledge.

Originally, the proposal for a book sprint was submitted to Google Summer of Code’s documentation track – it didn’t make it in. The OpenStack foundation came to the rescue, providing the funding needed to get the writers to the one place and keep them fed. Thanks to the Book Sprint methodology, from February 25th to March 1st, 2013, the authors worked out of Austin, Texas producing more than 10,000 words  a day, allowing a launch of the book the following week.

Tags:

A Pretty Good Place to Be

One of the best things about my work with OpenStack is the excitement I get when I envision the impact the work we do here today will have on the IT departments of the (very near) future.

Imagine a stay in a hospital, where instead of of nurses and med techs coming in every four hours around the clock to monitor your vital signs, a small monitoring device with specific sensors sends a steady stream of medical information to a central monitoring station. Software at the station not only looks for large anomalies, but also innocuous patterns that could indicate something serious.

Or imagine that kind of medical data being delivered to a central monitor when the patient is in their home. Or at work.

Fill a city with climate and environmental sensors, like noise and light levels, and use those sensors to identify emergencies before someone can dial a phone or long-term problems about the health of a particular neighborhood.

Put some sensors in a farm field in a developing country, and let farmers use the gathered data to know exactly how much water and nutrients to use for different parts of the field. For pennies a sensor, the yield of the field is maximized to help support a family for another year and maybe bring a little extra to the market.

A train car that knows exactly where it is at any given moment, cutting down on waste and unexpected routings.

A smart home that can control temperature, ambient light, and even alert a parent when a toddler is opening a cabinet they are not supposed to be in.

These are all just a very few examples of what the media calls the Internet of Things, either very real or in development as you read this. If you look an an object and imagine what that object could do if it were connected to the Internet, you can think of cool new applications on your own.

“Internet of Things” is a romantic notion. It’s not like web servers and file servers were animate objects all this time. But  whatever you call it, this new interconnected network of everyday objects will greatly impact people all of the world.

You can be sure that entrepreneurs are thinking of creative new ways to plug more devices into the Internet of Things, and all of them will need the same thing: a safe, secure platform on which they can store the data they will gather and analyze it to deliver the key services they will provide. If that platform is highly configurable and open, so much the better to mesh the platform to their needs.

One of the many applications of OpenStack technology will be to build that platform, and be a part of a world that will talk to us and tell us how to take better care of ourselves.

And that’s a pretty good place to be.

AlanClark

OpenStack and Friends at SCALE 11x

The Foundation had a great time meeting friends old and new as a Silver sponsor of this year’s Southern California Linux Expo in Los Angeles February 22-24.

We thank Christopher MacGown, Founder and Chief Scientist at Piston Cloud, whose standing-room-only, The Tao of OpenStack, talk was one of the hottest of the show.  Thank you also to Community Hero, Stephon Striplin, from DreamHost and OpenStack LA User Group co-organizer, who helped staff our busy booth alongside Foundation Community Manager Stefano Maffulli and Event Planner Kathy Cacciatore.

One of our favorite activities was the in-booth happy hour with besties Opscode and Rackspace – 2 kegs of Sierra Nevada and 12 bowls of chips happily consumed!  Check out the pics….

Rackspace and other new friends

Rackspace and other new friends

OpenStack Opscode and Rackspace happy hour pours with Tim Bell on screen

OpenStack Opscode and Rackspace happy hour pours with Tim Bell on screen

Opscode staff at happy hour

Opscode staff at happy hour

Christopher MacGown after his SRO Tao of OpenStack session

Christopher MacGown after his SRO Tao of OpenStack session

Stefano Maffulli and SCALE Poker Player

Stefano Maffulli and SCALE Poker Player

Here are the stats:

  • Over 2,300 attendees, 100 exhibitors and 90 speakers
  • 68 VIP badges scanned – Very Interested People of OpenStack cloud software
  • 250 Free as in Beer, Speech & Love t-shirts, 200 pieces of collateral, 150 stickers and 200 OpenStack LA User Group handouts given away.
  • Honorable mention for award for Favorite Clothing Item – our t-shirt came with the beer!

OpenStack Community Weekly Newsletter (Mar 1 -8)

Highlights of the week

We Wrote an OpenStack Operations Guide in 5 Days!

OpenStack Operations GuideAnd here it is…the OpenStack Operations Guide. You can read it in EPUB, MOBI, PDF, or print. The electronic formats are free to download so click away. The print version costs $29.90 and all proceeds go to the OpenStack Foundation to support more book sprints like the one that produced this book. Read from the protagonists how they did it: Lorin Hochstein, Everett Toews, Anne Gentle.

Case Study on OpenStack Cloud in Health Care

Computing for health care and life sciences is rapidly adapting to opportunities created by new technologies for scalable data storage, elastic compute service resources, and data analytics. Given the powerful forces changing the economics of the overall health care marketplace, technologies that can address security and compliance at reduced cost are getting a lot of attention.

OpenStack Object Storage and quotas in upcoming 1.8.0 (Grizzly) release.

OpenStack Object Storage (codename Swift) will have two new middleware for managing quotas: container_quotas and account_quotas. The first, container_quotas, allows users to specify a limit on one of their containers. The account_quotas is more the typical quota implementation. A «super user» with the reselleradmin group/role can set a byte limit for an account and the account will not be able to have new objects/containers until someone cleanups his account to get under the limited quotas.

OpenStack Updated Individual Contributor License Agreements

Last weekend we updated the OpenStack Individual Contributor License Agreement and all OpenStack contributors needed to sign a new individual agreement. We graphed the data of the rate of signups: almost 400 developers signed in less than a week.

Tips and Tricks

Upcoming Events

Reports from past events

Other News

Welcome New Contributors

Celebrating the first patches submitted this week by:

  • Bob Ball, Citrix
  • Stas Maksimov, Dell
  • Fei Long Wang, IBM
  • Avinash Prasad, NTT Data
  • KC Wang, Bigswitch
  • Divakar Padiyar Nandavar, HP

The weekly newsletter is a way for the community to learn about all the various activities occurring on a weekly basis. If you would like to add content to a weekly update or have an idea about this newsletter, please leave a comment.

 

Tags:

We Did It: Zero to Book in Five Days

OpenStack Operations Guide Call me crazy, call me maybe, but we did it! We have a 50,000 word book, 230 pages long, now available for download in your favorite ebook format, or purchase a printed copy if you like the no-batteries-required version. Go to http://docs.openstack.org/ops/ to get a copy and watch a video we made on day four.

We started the week off with a cookout at Everett Toews’s house. I transposed the house numbers and walked up to the wrong house, rang the bell and everything! A nice Texan lady told us we looked like we were going to have fun but the party wasn’t at her house. Woops! We found the right house on the same street and had a great time. The evening was complete with peach cobbler as we nervously awaited our fate: Could we complete a book in a week?

On Monday we assembled to find out. In a room with an entire wall of white board, our facilitator Adam Hyde unwrapped packs and packs of sticky notes with fresh markers. He introduced the book sprint process and said that he has done about 55 of these. He also said we may not get the book we thought we would going in, but by the end we will get the book we need. Sure enough, we collectively wrote about 10,000 words a day, bringing in all the content we could, revising, reshaping, rewriting, until it all hung together as a real book.

Our hope is that it is the book we all need, that it fills a gap for an under-served audience, the operators of clouds. We want your input, so start reading!

 

 

OpenStack at Linux.conf.au 2013 – Videos posted and some pics

I know it was just over a month ago and I apologize for being dreadfully late to blog it, but I just thought I’d post up a few pics of LCA2013 and links to the videos of the talks.

Here’s the full list of talks from LCA2013:
… and all the OpenStack ones (that I can find in that list), from both the miniconf and the main track.
This is a significant tally of content given there was exactly 1 OpenStack talk at LCA2012! (go Monty!)My thanks to the many attendees and especially the speakers, many of whom travelled a long way to Canberra for the event. Thanks to the Foundation and especially Kathy Cacciatore for the main conf sponsorship that also allowed us to give out the best t-shirts ever from http://www.dopenstack.com. Thanks also to Susan Derrick at HP, Florian Haas at Hastexo, Tim Serong at SUSE and Katrina Clauscen at Aptira for sponsoring the “Cloud Hour” happy hour.With amazing keynotes from the likes of Sir Tim Berners-Lee, Radia Perlman and Bunnie Huang and a wealth of incredibly diverse and deep technical content, if you’ve not been to LCA before, go next year! We will be back with what will be sure to be an even bigger OpenStack miniconf.

Cheers
Tristan

Tags:

OpenStack Community Weekly Newsletter (Feb 22 – Mar 1)

Highlights of the week

Important CLA changes coming this weekend (2nd try)

Last weekend the change was blocked at the last minute by a request to amend the CLA text. We’re trying again this weekend.

Starting on March 3, 2013 1700UTC all contributors MUST review and agree to the new OpenStack Individual Contributor License Agreement and provide updated contact information at https://review.openstack.org/#/settings/agreements. On that day the Gerrit interface will be changing to present the new CLA text referring to the OpenStack Foundation, and will prompt you to agree to it there. Any previous agreement with OpenStack LLC will be marked expired at that time. The text of the new agreement is available for your convenience. You must also sign up for an OpenStack Foundation Individual Membership with the same E-mail address as used for your Gerrit contact information: http://openstack.org/register/.

What’s new in OpenStack Grizzly ?

Folsom brought us two new core projects : Quantum (Networking) and Cinder (Volumes). Two new projects have been incubated : Ceilometer (metering) led by Nicolas Barcet (VP Products at eNovance), and Heat (Cloud Orchestration). Oslo is a new incubated project which produces a set of python libraries containing infrastructure code shared by all OpenStack projects. More details for each OpenStack project on this post by eNovance.

OpenStack Ceilometer and Heat projects graduated

The OpenStack Technical Committee have voted these last weeks about graduation of Heat and Ceilometer, to change their status from incubation to integrated. Heat and Ceilometer will be released as part as OpenStack for the next release cycle Havana, due in October 2013.

Raspberry Pi as a transparent squid caching proxy

How does a hacker improve build times when he has a slow Internet connection? A fascinating use of a tiny sub-$50 computer (RaspberryPi) to build cloud images for OpenStack. By Steve Baker

Data placement in OpenStack Object Storage Swift

Technical deep dive in how OpenStack Object Storage solves one of the hard problems of all distributed storage system: how to effectively place the data within the storage cluster. Swift has a “unique-as-possible” placement algorithm which ensures that the data is placed efficiently and with as much protection from hardware failure as possible.

The life of an OpenStack libvirt image

The main stages of a Virtual Machine disk image as it transfers through OpenStack to be booted under libvirt explained by Pádraig Brady.

Security Advisories

Tips and Tricks

Upcoming Events

Reports from past events

Other News

Welcome New Contributors

Celebrating the first patches submitted this week by:

  • Jian Zhang, Intel
  • Dae S. Kim
  • Robert van Leeuwen, Spilgames

The weekly newsletter is a way for the community to learn about all the various activities occurring on a weekly basis. If you would like to add content to a weekly update or have an idea about this newsletter, please leave a comment.

Tags:

OpenStack Community Weekly Newsletter (Feb 15 – 22)

Highlights of the week

Important CLA changes coming this weekend

Starting on February 24, 2013 all contributors MUST review and agree to the new OpenStack Individual Contributor License Agreement and provide updated contact information at https://review.openstack.org/#/settings/agreements. On that day the Gerrit interface will be changing to present the new CLA text referring to the OpenStack Foundation, and will prompt you to agree to it there. Any previous agreement with OpenStack LLC will be marked expired at that time. The text of the new agreement is available for your convenience (changes “LLC” to “Foundation”, restores a sentence from ASF and corrects a few typographical errors). You must also sign up for an OpenStack Foundation Individual Membership with the same E-mail address as used for your Gerrit contact information: http://openstack.org/register/.

OpenStack outpaces Amazon, at least by hype

A study conducted by TrendKite shows that at this point OpenStack has more media mentions than Amazon EC2, Eucalyptus, CloudStack and OpenNebula combined. If that is not enough, another chart from the same study appears to show that “OpenStack” media mentions are on par with that of “Cloud Computing.”

CloudEnvy: Development in the cloud!

Jake Dahn is working on a project called CloudEnvy which has potential to change the development patterns of web developers everywhere: it allows you to configure and distribute reproducible development environments in the cloud.

Bring on the Crazy: Zero to Book in Five Days

Anne Gentle is leading a “crazy” project: a large, highly focused documentation sprint to write that operator’s guide — for operators by operators. Anne promised to pluck operators out of their day jobs, put them in a room, fuel them with coffee, BBQ, and TexMex, and get to writing. Sounds awesome to me: I’m voting for their panel On Writing the OpenStack Operations Manual in 5 Days at the upcoming OpenStack Summit.

The OpenStack Gate

The OpenStack project has a really impressive continuous integration system, which is one of its core strengths as a project. Every proposed change to our gerrit review system is subjected to a battery of tests on each commit, which has grown dramatically with time, and after formal review by core contributors, we run them all again before the merge. How can this gate merge hundreds of changes per day? Learn about Zuul, the OpenStack gatekeeper.

Security Advisories

Tips and Tricks

Upcoming Events

Reports from previous events

Other News

Welcome New Contributors

Celebrating the first patches submitted this week by:

  • Toshiyuki Hayashi, NTT
  • Joe Topjian
  • David Höppner
  • Erik Zaadi, IBM
  • Nathanael Burton
  • alatynskaya, Mirantis
  • David Peraza, IBM
  • Brant Knudson, IBM

The weekly newsletter is a way for the community to learn about all the various activities occurring on a weekly basis. If you would like to add content to a weekly update or have an idea about this newsletter, please leave a comment.

Tags:

Bring on the Crazy: Zero to Book in Five Days

Well, since my team mate Everett has outed me as crazy (yes, it’s true), I think I’ve got some ‘splaining to do! It’s true, we’re crazy, but we’ve got to write that operator’s guide — for operators by operators. And if you know ops folks like I do, you realize quickly that these guys have hardly any time to write down their notes! We’re going to pluck them from their day jobs, fuel them with coffee, BBQ, and TexMex, and get to writing.

I’ve run doc sprints for OpenStack at various times on different scales, but this one is highly focused. So highly focused in fact that I’ve turned down people who wanted to attend but hadn’t run clouds day-to-day. Sorry guys! Super hyper focus, blinders on.

I had originally put the proposal together for the Google Summer of Code Doc Summit, which I attended in 2011, but out of 30+ applying organizations, we weren’t selected. Undaunted, I revised the proposal for the OpenStack Foundation to consider, and they are funding it!
1-IMG_3304
We’re gathering in the Willie Nelson room at the Austin Rackspace office next week, and we’re going to try a tool called BookType, hosted by Source Fabric. It’s the same tool I’ve worked with for book sprints with FLOSS Manuals, and it’s made specifically for sprinting. We’re using a pre-installed instance of it at http://openstack.booktype.pro. If you look at it today, it’s empty except for a Test book sandbox. By next week it’ll be bursting with content! The outline we’re working with to start is at https://etherpad.openstack.org/EssexOperationsGuide. Thanks to everyone who has reviewed it and commented so far.

You all are welcome to watch our progress next week at at http://openstack.booktype.pro and offer editing and reviewing as we go. Please realize it’s a pressure cooker of a week, and a real challenge. I’m saying, be helpful and encouraging so we can be productive and accurate. Once we’re done, ship it! We’ll have copies of the book available (likely for a donation initially to cover costs) by the end of next week.

If you want to hear all the gory details, we have a few opportunities. Next week, if you’re in Austin, come see us at the Austin OpenStack Meetup on Wednesday night. Then, at the Summit, we want to bare our souls at a panel titled, On Writing the OpenStack Operations Manual in 5 Days. Come see if we are crazy, or driven there this week.

Tags: