OpenStack Developer Mailing List Digest Jan 23 – Feb 9

SuccessBot Says

  • odyssey4me: OpenStack Ansible Liberty 12.0.5 released.
  • stevemar: Devstack now defaults to v3 for Keystone.
  • boris-42: osprofiler functional job passed [1].
  • odyssey4me: OpenStack Ansible Kilo 11.2.9 released [2].
  • odyssey4me: OpenStack Ansible Liberty 12.0.6 released [3].
  • All: https://wiki.openstack.org/wiki/Successes

Cross-Project Specs

  • A common policy scenario across all projects [4].
  • Query config from web UI [5]

API Guidelines

  • Must not return service-side tracebacks [6].

Service Type vs. Project Name For Use In Headers

  • There’s a question of whether we should be using service type or project names in headers. Some reviews involving this [7][8][9][10].
  • We should be selecting things that better serve the API consumers and according to Dean Troyer the API working group is going in the right direction.
  • The service type as the primary identifier for endpoints and API services is well established, and is how the service catalog has and will always work. Service types therefore should be the way to go.
  • Full thread: http://lists.openstack.org/pipermail/openstack-dev/2016-January/085145.html

OpenStack Ansible Without Containers

  • Gyorgy annouces a new installer for OpenStack under GPLv3 using Ansible, but without containers.
    • Reasons for another installer since we already have the OpenStack Ansible project and Kolla:
      • Containers adding unnecessary complexity.
      • Packages: avoid mixing pip and distributor packages. Distributor packages includes things like init scripts, proper system users, upgrade possibilities, etc.
        • Kevin Carter mentions that these benefits are actually also included with the OpenStack Ansible project.
  • Without containers, upgrading a single controller can be tricky and disruptive since you have to upgrade every service at the same time. Rollbacks are also easier.
  • OpenStack Ansible project can already today do deployment without containers using the is_metal=true variable.
  • Full thread: http://lists.openstack.org/pipermail/openstack-dev/2016-January/084963.html

Release Countdown for Week R-8, Feb 8-12

  • Focus:
    • 2 more weeks before final releases for non-client libraries for this cycle.
    • 3 more weeks before the final releases for client libraries.
    • Projects should focus on wrapping up feature work in all libraries.
  • Release Actions:
    • The release team will be strictly enforcing library release freeze before M3 in 3 weeks.
  • Important Dates:
    • Final release for  non-client libraries: Feb 24
    • Final release for client libraries: Mar 2
    • Mitaka 3: Feb 29-Mar 4 (includes feature freeze and soft string freeze)
  • Full thread: http://lists.openstack.org/pipermail/openstack-dev/2016-February/085705.html

“No Open Core” in 2016

  • Before OpenStack had a name, the “four opens” principles were created to define how we operate as a community.
  • In 2010 when OpenStack started, it was different from other open source cloud platforms (Eucalyptus) which followed open core strategy of producing a crippled community edition and an “enterprise version”.
  • Today we have a non-profit independent foundation, which cannot do an “enterprise edition”.
    • Today member companies build “enterprise products” on top of the apache licensed upstream project. Some are drivers that expose functionality in proprietary components.
  • What does it mean to “not do open core” in 2016? What is acceptable and what’s not?
  • Thierry Carrez believes it’s time to refresh this of what is an acceptable official project in OpenStack.
    • It should have a fully-functional production grade open source implementation
    • If you need proprietary software of commercial entity to fully use the project, then it should not be accepted in OpenStack as an official project.
      • These projects can still be non-official projects and still be hosted by OpenStack infrastructure.
  • Doug Hellmann brings up Poppy [11] which is applying to be an official OpenStack project.
    • A wrapper to content delivery networks, but there is no open source solution.
    • Is this something that can be an official project, or is open core?
  • Full thread: http://lists.openstack.org/pipermail/openstack-dev/2016-February/085855.html

The Trouble with Names

  • A few issues have crept up recently with the service catalog, API headers, API end points, and even similarly named resources in different resources (e.g. backup), that are all circling around a key problem. Distributed teams and naming collision.
  • Each project has a unique name that is ensured by their git repository in the OpenStack namespace.
  • There’s a desire to replace project names with generic names like nova/compute in:
    • service catalog
    • api headers
  • Options we have are:
    • Use the code names we already have: nova, glance, swift, etc.
      • Upside: collision problem solved.
      • Downside: You need a secret decoder ring to know what a project does.
    • Have a registry of common names.
      • Upside: we can safely use common names everywhere and not fear collision down the road.
      • Downside: yet another contention point.
  • Approvals by the various people in the community to have a registry of the common names. Maybe in the governance projects.yaml file [12]?
    • This list does include only the official projects by the technical committee, therefore only those projects can reserve the common names.
  • OpenStack Client has already encoded some of these common names to projects [13].
  • Full thread: http://lists.openstack.org/pipermail/openstack-dev/2016-February/085748.html

Announcing Ekko – Scalable Block-Based Backup for OpenStack

  • The goal of Ekko is to provide incremental block-level backup and restore of Nova instances.
  • Two place with overlapping goals:
    • Cinder volume without having the incremental backups be dependent.
    • Nova instances
      • OpenStack Freezer today leverages Nova’s snapshot feature.
      • Ekko would leverage live incremental block-level backup of a nova instance.
  • Jay Pipes begins the discussion on the two projects (Freezer and Ekko) working together to make sure their REST API endpoints are not overlapping. Having two APIs for performing backups that are virtually identical is not good.
  • The creator of Ekko sees the reason for another backup project because of “actual implementation and end results are wildly different” even if they are the same API call.
  • Jay doesn’t like that today all the following endpoints exist:
    • Freezer’s /backups
    • Cinder’s /{tenant_id}/backups
  • Having these endpoints make for bad user experience and is just confusing.
  • The current governance model does not prevent competition of projects. So if two projects overlap in API endpoints, there should be an attempt in collaboration.
  • Full thread: http://lists.openstack.org/pipermail/openstack-dev/2016-January/084739.html

Technical Committee Highlights January 22, 2016

Upstream development track – please submit

We’ll have an “Upstream development” track at the Austin Summit. It will happen on the Monday, before the Design Summit starts. This is a classic Summit conference track with recorded videos, so we want polished proposals for this track. We expect these to include general communication about development process changes, new features in a central project that need adoption in other projects, corner use cases that may need support from development, developer-oriented infra talks and upstream development best practices. To propose a talk for this track, use the Summit proposal system, select Upstream development, and meet the February 1st deadline.

Our mission

The OpenStack overall mission has stood proudly for years now, and is due for an update to increase focus on cloud consumers rather than solely on cloud builders. So we have proposed an amendment to the original mission. You can read the current and proposed new mission on governance.openstack.org.

And now, even more doc

In a clarification effort, we pushed the definition of the 4 opens, as well as clarified OpenStack licensing requirements as reference documents under the governance repository. Previously those were maintained in oral tradition, the wiki, or left as an exercise to the reader of the Foundation bylaws. You can now find them published (like all Technical Committee resolutions and reference information) on the governance website.

The names are here! The names are here!

The N and O releases directly after Mitaka will be Newton and Ocata. For the Austin Summit, the tie-in is to the “Newton House”, located at 1013 E. Ninth Street in Austin, Texas. It’s listed on the National Register of Historic Places. For the Barcelona Summit, know that Ocata is a beach about 20 minutes north of Barcelona by train.

Newton House (Austin,TX)

Clarifying licensing requirements

A new governance page clarifies guidelines for licensing for projects in and around OpenStack.  We want to raise awareness and highlight that page for future reference. In the subset of OpenStack projects that may be included in a Defcore trademark program, the project must be licensed under Apache Software License v2, ASLv2. Libraries and software built in the OpenStack infrastructure system should use OSI-approved licenses that do not restrict distribution of the consuming project. Read more on the governance website.

 

OpenStack Developer Mailing List Digest January 16-22

Success Bot Says

  • mriedem: nova liberty 12.0.1 released [1].
  • OpenStack Ansible Kilo 11.2.7 has been released.
  • OpenStack-Ansible Liberty 12.0.4 has been released.
  • Tell us yours via IRC with a message “#success [insert success]”.
  • All: https://wiki.openstack.org/wiki/Successes

Governance

  • License requirement clarification for big tent projects [2].
  • Make constraints opt in at the test level [3].
  • OSprofiler is now an official OpenStack project [4].

Cross-Project

Release Count Down For Week R-10, Jan 25-29

  • Focus: with the second milestone behind us, project teams should be focusing on wrapping up new feature work and stabilizing recent additions.
  • Release actions:
    • Strictly enforcing library release freeze before M3 (5 weeks).
    • Review client/integration libraries and whatever other libraries managed by your team.
    • Ensure global requirements and constraints lists are up to date with accurate minimum versions and exclusions.
    • Quite a few projects with unreleased changes on stable/liberty branch. Check for your project [7].
  • Important dates:
    • Final release for non client libraries: February 24th
    • Final release for client libraries: March 2nd
    • Mitaka-3: Feb 29 through March 4 (includes feature freeze and soft string freeze).
    • Mitaka release schedule [8].
  • Full thread: http://lists.openstack.org/pipermail/openstack-dev/2016-January/084678.html

Stabilization Cycles: Elaborating on the Idea To Move It Forward

  • At the Tokyo summit, the OpenStack Development Theme session, in which people discuss overall focus in shared efforts, having cycles to stabilize projects was brought up.
  • A project could decide to spend some percentage of time of the cycle on focusing on bug fixing, review backlog, refactoring, instead of entirely on new features.
  • Projects are already empowered to do this, however, maybe the TC could work on formalizing this process so that teams have a reference when they want to.
  • Some contributors from the summit feel they need the Technical Committee to take leadership on this, so that they can sell it back to their companies.
  • Another side of discussion, healthy projects should naturally come up with bursts of feature additions and burst of repaying technical debt continuously.
    • Imposing specific periods of stabilization prevents reaching that ideal state.
  • Full thread: http://lists.openstack.org/pipermail/openstack-dev/2016-January/084564.html

OpenStack Developer Mailing List Digest January 9-15

Success Bot Says

  • stevemar: Latest python-neutronclient use keystoneauth, yay!
  • devkulkarni: Devstack plugin for Solum finally working as expected.
  • dulek: Initial tests show that our rolling upgrades stuff is working fine – I’m able to use Mitaka’s Cinder API service with Liberty’s cinder-scheduler and c-volume services.
  • Tell us yours via IRC with a message “#success [insert success]”.

Cross-Project Specs & API Guidelines

  • Add clouds.yaml support specification [1]
  • Deprecate individual CLIs in favor of OSC [2]
  • Add description of pagination parameters [3].

Release Models To Be Frozen Around Mitaka-2

  • Deadline: Mitaka-2, January 21st
  • Example, your release model is release:independent, and you want to switch to cycle-oriented models (e.g. release:cycle-with-intermediary or release:cycle-with-milestones). [4]
  • To change your project, propose an openstack/governance change in reference/projects.yaml file [5].

Vision and Changes For OpenStack API Guides

  • New tool: fairy-slipper [6]
    • Migrate files from WADL to Swagger.
    • Serve up reference info.
  • New build jobs to build API guides from project repos to developer.openstack.org
  • It was discussed in the last cross-project meeting [7] to answer questions.
  • There are a variety of specs [8][9] to go over this work.
  • See what’s happening this month [10].

“Upstream Development” Track At the Austin OpenStack Summit

  • Call for speakers [11] at the OpenStack conference in Austin will have a new track targeted towards upstream OpenStack developers.
    • Learn about new development processes
    • Tools that the infrastructure team gives us
    • New OSLO library features (or elsewhere)
    • Best practices
  • Probably Monday before the design summit tracks start.
  • Have a topic that fits this audience? Submit it! [12]

You Almost Never Need to Change Every Repository

  • There have been a lot of patches that tweak the same thing across many, many repositories.
  • Standardizations are great, but if you’re making the same change to more than a few repositories, we should be looking at another way to have that change applied.
  • If you find yourself making the same change over and over in a lot of projects, start a conversation on the dev mailing list first.

Release Countdown For Week R-11, Jan 18-22

  • Focus:
    • Next week is the second milestone for the Mitaka cycle.
    • Major feature work should be making good process, or be re-evaluated to see it really needs to land this cycle.
  • Release Actions:
    • Liaisons should submit tag requests to the openstack/releases repository for projects following the cycle-with-milestone before the end of the day on Jan 21.
    • Release liaison responsibility update should be reviewed [13].
  • Important Dates:
    • Mitaka 2: January 19-21
    • Deadline for Mitaka 2 tag: Jan 21
    • Release models to be frozen: Jan 21

What’s next for application developer guides?

Summary

This month, the developer.openstack.org site gets a new look and changes its source tooling. Read on for details about how these changes affect your project team.

Why are we changing the developer.openstack.org site?

You might know that the developer.openstack.org site documents over 900GET/PUT/POST/DELETE/PATCH calls for a dozen OpenStack services already on the developer.openstack.org site. As a couple of the keynote speakers in Tokyo so elegantly put it, the trustworthiness and consistency of the OpenStack APIs influenced their decision to run their business workloads in an OpenStack cloud.

Those interfaces need docs, lots of docs, and not only reference docs. While it takes a huge effort to maintain accurate references, we also need to document API usage and scenarios.

Now that we’ve written both an API Guide and a “Writing your first OpenStack application” tutorial, we want the site to be the go-to location for app devs, product developers, and anyone who needs to unlock the power of their OpenStack infrastructure.

In this release, the docs’ platform introduces tooling that lets you migrate from WADL to Swagger and integrate RST-sourced documentation with the API reference documentation. The “why” analysis is clear: we must community-source this info and make it easy to maintain and update so that users can trust it enough to bet their workloads on it.

Later on, this post answers the “how” questions.

Why all these changes at this point in the release?

Well, we haven’t had to release the API documentation like we release services documentation. We have done a lot of maintenance on the site, with bug fixing and so on. But it’s time to take the leap. Last release we made a proof-of-concept. This release we unleash a solution that helps us make incremental progress toward our goals.

How do you keep your API docs updated after January 16th?

On January 16th, we’ll migrate the Images API WADL and DocBook files to Swagger and RST files. Then we’ll test the build process and the content itself to validate the migration.

After testing, we will migrate the files for these services:

  • Identity
  • Compute
  • Images
  • Networks
  • Block Storage
  • Object Storage

Then, the remaining services can migrate their files by using the validated tooling.

If you do provide an OpenStack API service, read on.

For the nova project, place your how-to and conceptual articles in the api-guide folder in the nova repository. Other projects can mimic these patches that created an api-guide and build jobs for the Compute api-guide. You continue to update reference information in the openstack/api-site repo. However, the source files have changed.

With this release, you can embed annotations in your source code if you want to generate the reference information. Here’s an example patch from the nova project. Because we haven’t had a project do this yet completely, the build jobs still need to be written.

If your project already has WADL files, they will be migrated to Swagger files and stored in the api-site repository. The WADL files will be deleted; you can retrieve them from Git.

If your project does not have a WADL file, then you write Swagger plus RST to document your API calls, parameters, and reference information. You can generate Swagger from annotations or create Swagger from scratch that you store in the api-site repository. You should review, store, and build RST for conceptual or how-to information from within your project team’s repository.

All projects should use this set of API documentation guidelines from the OpenStack API working group any time their service has a REST API. This document tells you what and how much to write. If you follow the suggested outline, your API guide will be accurate and complete.

After the source files and build jobs exist, the docs are built to developer.openstack.org.

Where can I get help for my project’s API Guides?

These specifications describe the details: Application Developer Guides and Rework API Reference Information.

You can ask questions in #openstack-sdks or #openstack-docs on IRC. We await those magical API docs fairies who grant wishes, but in the meantime but we can point you in the right direction and give you the tools for your quest.

What if I still have questions?

Contact me, Anne Gentle, by email or IRC and I’ll get back to you as soon as possible.

I’m eager to enable our audience with great user-centric docs and hope that you’ll join us as we fulfill the vision.

Tags:

OpenStack Developer Mailing List Digest January 2-8

SuccessBot Says

  • russellb: Ported OVS to Python 3.
  • notmyname: Removed the beta tag on the swift erasure code docs.
  • AJaeger: OpenStack configuration reference has been migrated from DocBook XML to RST [1].
  • notmyname: This season’s Outreachy intern had her first patch merged.
  • odyssey4me: OpenStack-Ansible 12.0.3 tagged.
  • jordanP: My 3rd party CI correctly warned me that a patch would break my driver.
  • asselin: OpenStack Third Party CI documentation published [2].

Slightly Longer Delays for Gate CI Testing

  • When: January 31, 2016
  • Why:
    • Due to the sunsetting of a public cloud [3] used by the OpenStack Infra team, we will have less resources available.
    • Jeremy Stanley says that the Infra team is working on bringing in new providers to absorb the impact.

Release Countdown For Week R-12, Jan 11-25

  • Focus:
    • Second milestone is coming up. Finish up major features or reevaluate if it really needs to land in this cycle.
  • Actions:
    • Identify work that needs to be completed before the M-2 tag.
    • Release liaisons responsibilities are being updated. Provide comments for questions or concerns [4].
  • Important Dates:
    • Mitaka 2: Jan 19-21
    • Release schedule [5].

Re-introduce Twisted to global-requirements

  • A change in global-requirements introduces mimic, an http server that can mock various APIs.
  • Mimic depends on twisted, which in the past was removed in favor of Eventlet to avoid developers having to know multiple frameworks [5].
  • Jim Rollenhagen explains Ironic’s need for Mimic:
    • To do functional tests, not unit tests which they could do with requests-mock.
    • To bring up a less expensive Ironic environment for doing these tests.
  • Jay Pipes notes:
    • This is another way to introduce a larger surface area of bugs to creep in since you have keep Mimic up-to-date with the real interfaces.
    • There’s not a clear value in the advantage  of functional tests of a client that calls a fake HTTP API versus unit tests of a client that simply uses requests-mock.

OpenStack Community Newsletter (Dec. 12-22)

Running an OpenStack cloud? Come to the first European Mid-Cycle Operators meetup

The meetup, held February 15 – 16, 2016 in Manchester, is a chance for operators to share best practices and give feedback.

The keys to OpenStack startup success: Platform9 Systems

Platform9 Systems opens its startup playbook, highlighting ecosystem challenges and tips for fellow startups including how they differentiate their product from competitors with simplicity, choice and interoperability.

OpenStack operators connecting together

To make meetups more productive and useful, Edgar Magana at Workday proposes smaller, operator-focused gatherings.

The Road to Austin

Community feedback

OpenStack is always interested in feedback and community contributions, if you would like to see a new section in the OpenStack Weekly Community Newsletter or have ideas on how to present content please get in touch: [email protected].

CFP Deadlines

OpenStack Developer Mailing List Digest December 12-18

Tips ‘n Tricks 

Upcoming Events 

Other News

OpenStack Developer Mailing List Digest December 12-18

Success Bot Says

  • jroll: Ironic is using the devstack plugin.
  • Tell us yours via IRC with a message “#success [insert success]”.

Cross-Project Specs

  • Approved by the Technical Committee:
    • Chronicles Of A DLM [1].
      • OpenStack will adopt Tooz as an abstract layer to distributed lock managements solutions. This will allow projects to provide all the options Tooz provides for their distributed lock management needs.
      • Reference implementation is currently ZooKeeper for testing.
  • Close to approval:
    • Backwards compatibility for libraries and clients [2].
    • Deprecate individual CLIs in favor of OSC [3].

New Things In Gerrit 2.11 To Enjoy

  • New search strings:
    • Patch size
      • delta:<=10
      • show patches with <= 10 lines of change
    • Scoring by group
      • label:Code-Review<=-1,nova-core
      • filter by -1 from nova-core team
    • message:/comment:
      • Queries work with more than one word
  • UI Features:
    • is:amergable field
    • Column 3
      • Related changes: all changes in linear patch series.
      • Conflicts with: all open patches in the system that will conflict with this one.
      • Same topic: Grouping reviews.
    • Inline edit an entire patch.
      • Click the ‘edit’ button above the list of files. Once you’re done click ‘save’ and eventually ‘Publish Edit’ on the main page.
      • “Follow up” button: builds an entire follow up patch that you can inline edit to fix something like typos.

Stable Meeting Time Proposal

DocImpact vs. Reno

  • Sean notes a new job on all Nova changes that is processing commit messages [5].
  • UpgradeImpact got dropped for having an upgrade comment in Reno.
  • Instead of DocImpact, SecurityImpact in commit messages, new special sections using Reno will be created that trigger creating bugs and sending out emails like today. The hope is for better quality in these notifications.

Nominations open for the N and O names of OpenStack

  • For the N release, where the geographic region is “Texas Hill Country” [6]
  • For the O release, where the geographic region is “Catalonia” [7]
  • OpenStack foundation individual members should have received a vote link via email. [8]
  • Polls end of Dec 22nd UTC.

OpenStack Weekly Community Newsletter (Dec. 5-11)

Travel grants bring global community members to OpenStack Summit

Key contributors from 15 countries attended the recent OpenStack Summit in Tokyo. The program for Austin is growing – find out more about the Travel Support Program.

OpenStack Mitaka release: what’s next for Neutron, Cinder and Ceilometer

Meet the project team leads (PTLs) and find out how to get involved.

Check out this tool library for OpenStack operators

These tools – contributed by and for operators – offer quick solutions to streamline your work

OpenStack design contest T-shirts are now available!

Joline Buscemi won the third edition of the community T-shirt design contest, held annually by the OpenStack Foundation.

Community feedback

OpenStack is always interested in feedback and community contributions, if you would like to see a new section in the OpenStack Weekly Community Newsletter or have ideas on how to present content please get in touch: [email protected].

CFP Deadlines

OpenStack Developer Mailing List Digest December 5-11

Tips ‘n Tricks 

Upcoming Events 

Other News

OpenStack Developer Mailing List Digest December 5-11

Success Bot Says

  • AJaeger: With Juno End-of-Life, Python 2.6 testing has been removed from OpenStack CI.
  • Tell us yours via IRC with a message “#success [insert success]”.

Gerrit Update 12/16

  • Gerrit will be offline December 16th 17:00-21:00 UTC
  • Gerrit will be upgraded to 2.11
  • IP address will stay the same.
  • For questions, reply to the thread or on IRC #openstack-infra.

Release Countdown For Week R-17, Dec 7-11

  • Focus should be on cross-project themes identified a the summit [1], and whether any work needs to be done to address them in your project.
  • Doug Hellmann will be offline until December 21st. The rest of the release management team will able to assist via:
    • IRC #openstack-release
    • Mailing list with “[release]” in the subject.
  • Release actions:
    • If the Liberty release notes page [2] doesn’t have a link to your project’s release notes, submit a patch to your deliverable file in openstack/releases.

Bugs Will Now Close Automatically When Patches Merge

  • Part of deprecating our use of Launchpad for tracking completed work, when a patch with “Fixes-Bug” in the commit message merges, the bug will move to ‘Fixed Released” instead of “Fix Committed”.

Cleaning Up Deprecation Warnings in the Gate

  • We have a lot of logs to process in our elastic search cluster, which results in frequently having issues with the amount of RAM available in the cluster.
  • In general it would be good for projects to look at how verbose their logging is, and if that’s really necessary.
  • Using this query [3] to show deprecation warnings on jobs running on master the past 7 days shows 17576707 hits.
  • We need volunteers to to make changes in things like Devstack to have it avoid using deprecated options.
    • If you would like to help, please use the topic ‘stop-using-deprecated-options’ [4].

Stable Team PTL Elected

  • Close to 200 voters have elected Matt Riedmann as the first stable team PTL [5].
  • Matt will be looking into meeting times next week to begin discussions on transitioning from release team and work items.

Announcing the OpenStack Health Dashboard

  • A dashboard to visualize the state of tests running in gate [6].
  • Started last September, it’s still in early phases.
  • Biggest limitations:
    • The datastore: using subunit2sql as the backend for all data and only collecting results for tempest and grenade runs in the gate and periodic queues.
    • No results for runs that fail before tempest starts running.
  • Code [7]
  • Launchpad [8]
  • Work items that need to be done [9].