OpenStack Developer Mailing List Digest January 2-8

SuccessBot Says

  • russellb: Ported OVS to Python 3.
  • notmyname: Removed the beta tag on the swift erasure code docs.
  • AJaeger: OpenStack configuration reference has been migrated from DocBook XML to RST [1].
  • notmyname: This season’s Outreachy intern had her first patch merged.
  • odyssey4me: OpenStack-Ansible 12.0.3 tagged.
  • jordanP: My 3rd party CI correctly warned me that a patch would break my driver.
  • asselin: OpenStack Third Party CI documentation published [2].

Slightly Longer Delays for Gate CI Testing

  • When: January 31, 2016
  • Why:
    • Due to the sunsetting of a public cloud [3] used by the OpenStack Infra team, we will have less resources available.
    • Jeremy Stanley says that the Infra team is working on bringing in new providers to absorb the impact.

Release Countdown For Week R-12, Jan 11-25

  • Focus:
    • Second milestone is coming up. Finish up major features or reevaluate if it really needs to land in this cycle.
  • Actions:
    • Identify work that needs to be completed before the M-2 tag.
    • Release liaisons responsibilities are being updated. Provide comments for questions or concerns [4].
  • Important Dates:
    • Mitaka 2: Jan 19-21
    • Release schedule [5].

Re-introduce Twisted to global-requirements

  • A change in global-requirements introduces mimic, an http server that can mock various APIs.
  • Mimic depends on twisted, which in the past was removed in favor of Eventlet to avoid developers having to know multiple frameworks [5].
  • Jim Rollenhagen explains Ironic’s need for Mimic:
    • To do functional tests, not unit tests which they could do with requests-mock.
    • To bring up a less expensive Ironic environment for doing these tests.
  • Jay Pipes notes:
    • This is another way to introduce a larger surface area of bugs to creep in since you have keep Mimic up-to-date with the real interfaces.
    • There’s not a clear value in the advantage  of functional tests of a client that calls a fake HTTP API versus unit tests of a client that simply uses requests-mock.

OpenStack Community Newsletter (Dec. 12-22)

Running an OpenStack cloud? Come to the first European Mid-Cycle Operators meetup

The meetup, held February 15 – 16, 2016 in Manchester, is a chance for operators to share best practices and give feedback.

The keys to OpenStack startup success: Platform9 Systems

Platform9 Systems opens its startup playbook, highlighting ecosystem challenges and tips for fellow startups including how they differentiate their product from competitors with simplicity, choice and interoperability.

OpenStack operators connecting together

To make meetups more productive and useful, Edgar Magana at Workday proposes smaller, operator-focused gatherings.

The Road to Austin

Community feedback

OpenStack is always interested in feedback and community contributions, if you would like to see a new section in the OpenStack Weekly Community Newsletter or have ideas on how to present content please get in touch: [email protected].

CFP Deadlines

OpenStack Developer Mailing List Digest December 12-18

Tips ‘n Tricks 

Upcoming Events 

Other News

OpenStack Developer Mailing List Digest December 12-18

Success Bot Says

  • jroll: Ironic is using the devstack plugin.
  • Tell us yours via IRC with a message “#success [insert success]”.

Cross-Project Specs

  • Approved by the Technical Committee:
    • Chronicles Of A DLM [1].
      • OpenStack will adopt Tooz as an abstract layer to distributed lock managements solutions. This will allow projects to provide all the options Tooz provides for their distributed lock management needs.
      • Reference implementation is currently ZooKeeper for testing.
  • Close to approval:
    • Backwards compatibility for libraries and clients [2].
    • Deprecate individual CLIs in favor of OSC [3].

New Things In Gerrit 2.11 To Enjoy

  • New search strings:
    • Patch size
      • delta:<=10
      • show patches with <= 10 lines of change
    • Scoring by group
      • label:Code-Review<=-1,nova-core
      • filter by -1 from nova-core team
    • message:/comment:
      • Queries work with more than one word
  • UI Features:
    • is:amergable field
    • Column 3
      • Related changes: all changes in linear patch series.
      • Conflicts with: all open patches in the system that will conflict with this one.
      • Same topic: Grouping reviews.
    • Inline edit an entire patch.
      • Click the ‘edit’ button above the list of files. Once you’re done click ‘save’ and eventually ‘Publish Edit’ on the main page.
      • “Follow up” button: builds an entire follow up patch that you can inline edit to fix something like typos.

Stable Meeting Time Proposal

DocImpact vs. Reno

  • Sean notes a new job on all Nova changes that is processing commit messages [5].
  • UpgradeImpact got dropped for having an upgrade comment in Reno.
  • Instead of DocImpact, SecurityImpact in commit messages, new special sections using Reno will be created that trigger creating bugs and sending out emails like today. The hope is for better quality in these notifications.

Nominations open for the N and O names of OpenStack

  • For the N release, where the geographic region is “Texas Hill Country” [6]
  • For the O release, where the geographic region is “Catalonia” [7]
  • OpenStack foundation individual members should have received a vote link via email. [8]
  • Polls end of Dec 22nd UTC.

OpenStack Weekly Community Newsletter (Dec. 5-11)

Travel grants bring global community members to OpenStack Summit

Key contributors from 15 countries attended the recent OpenStack Summit in Tokyo. The program for Austin is growing – find out more about the Travel Support Program.

OpenStack Mitaka release: what’s next for Neutron, Cinder and Ceilometer

Meet the project team leads (PTLs) and find out how to get involved.

Check out this tool library for OpenStack operators

These tools – contributed by and for operators – offer quick solutions to streamline your work

OpenStack design contest T-shirts are now available!

Joline Buscemi won the third edition of the community T-shirt design contest, held annually by the OpenStack Foundation.

Community feedback

OpenStack is always interested in feedback and community contributions, if you would like to see a new section in the OpenStack Weekly Community Newsletter or have ideas on how to present content please get in touch: [email protected].

CFP Deadlines

OpenStack Developer Mailing List Digest December 5-11

Tips ‘n Tricks 

Upcoming Events 

Other News

OpenStack Developer Mailing List Digest December 5-11

Success Bot Says

  • AJaeger: With Juno End-of-Life, Python 2.6 testing has been removed from OpenStack CI.
  • Tell us yours via IRC with a message “#success [insert success]”.

Gerrit Update 12/16

  • Gerrit will be offline December 16th 17:00-21:00 UTC
  • Gerrit will be upgraded to 2.11
  • IP address will stay the same.
  • For questions, reply to the thread or on IRC #openstack-infra.

Release Countdown For Week R-17, Dec 7-11

  • Focus should be on cross-project themes identified a the summit [1], and whether any work needs to be done to address them in your project.
  • Doug Hellmann will be offline until December 21st. The rest of the release management team will able to assist via:
    • IRC #openstack-release
    • Mailing list with “[release]” in the subject.
  • Release actions:
    • If the Liberty release notes page [2] doesn’t have a link to your project’s release notes, submit a patch to your deliverable file in openstack/releases.

Bugs Will Now Close Automatically When Patches Merge

  • Part of deprecating our use of Launchpad for tracking completed work, when a patch with “Fixes-Bug” in the commit message merges, the bug will move to ‘Fixed Released” instead of “Fix Committed”.

Cleaning Up Deprecation Warnings in the Gate

  • We have a lot of logs to process in our elastic search cluster, which results in frequently having issues with the amount of RAM available in the cluster.
  • In general it would be good for projects to look at how verbose their logging is, and if that’s really necessary.
  • Using this query [3] to show deprecation warnings on jobs running on master the past 7 days shows 17576707 hits.
  • We need volunteers to to make changes in things like Devstack to have it avoid using deprecated options.
    • If you would like to help, please use the topic ‘stop-using-deprecated-options’ [4].

Stable Team PTL Elected

  • Close to 200 voters have elected Matt Riedmann as the first stable team PTL [5].
  • Matt will be looking into meeting times next week to begin discussions on transitioning from release team and work items.

Announcing the OpenStack Health Dashboard

  • A dashboard to visualize the state of tests running in gate [6].
  • Started last September, it’s still in early phases.
  • Biggest limitations:
    • The datastore: using subunit2sql as the backend for all data and only collecting results for tempest and grenade runs in the gate and periodic queues.
    • No results for runs that fail before tempest starts running.
  • Code [7]
  • Launchpad [8]
  • Work items that need to be done [9].

OpenStack Weekly Community Newsletter (Nov. 28 – Dec. 4)

Individual Member Director nominations are now open

The deadline for Individual Member Director nominations is next Friday, December 11, 2015

How to manage medical and health data with OpenStack, a wearable sensor and smart apps

“Our hope is to inspire you to look into open source, to think of ways to solve the data problems that you might have already,” says Anne Gentle of Rackspace.

Why no data center should be full of islands

OpenStack connects developers with new tools while leveraging a common foundation, says Foundation COO Mark Collier.

Powering one billion page views and 170 million unique visitors per month with OpenStack

NTT Resonant operates goo, one of Japan’s leading web portals and Internet search engines

Community feedback

OpenStack is always interested in feedback and community contributions, if you would like to see a new section in the OpenStack Weekly Community Newsletter or have ideas on how to present content please get in touch: [email protected].

Reports from Previous Events 

CFP and Election Deadlines

OpenStack Developer Mailing List Digest November 28 to December 4

Tips ‘n Tricks 

Upcoming Events 

Other News

OpenStack Developer Mailing List Digest November 28 to December 4

Success Bot Says

  • dims: cross-project, technical-debt-reduction effort pays dividends, no code left in oslo-incubator repo anymore.
  • dhellmann: horizn, searchlight, python-openstackclient, ironic-introspec, manila, barbican, aodh, and sahara tagged for mitaka-1 milestone!
  • odyssey4me: OpenStack-Ansible for Kilo and Liberty have been released.
  • ttx: Nova mitaka-1 (13.0.0.0b1) published!
  • flaper87: Glance m-1 is out
  • AJaegar: The Networking Guide has been translated into Japanese, read it [1].
  • stevemar: Got reno in all keystone projects.
  • Tell us yours via IRC with a message “#success [insert success]”.

Cross-Project Specs Close to Merging

  • Backwards compatibility for clients and libraries [2].
  • Deprecate Individual CLIs in favor of OpenStack Client [3].
  • Chronicles of Distributed Lock Manager [4].

Release Countdown For Week R-17, Dec 7-11

  • A list of all projects with tags [5].
  • There’s a known bug in Reno for release notes not appearing after merge [6].
  • Teams should have retrospectives to consider how the cycle is going so far.
  • Each project has a patch that removes the version field from setup.cfg file. Please review that as quickly as possible.
  • Mitaka 2 Release: Jan 19-21
  • Mitaka release schedule [7].

The Lock Files Saga (and Where We Can Go From Here)

  • Josh Harlow writes, different projects use file locks to ensure that no application on the same machine can manipulate a given resource.
  • Stale lock file bug happens [8] and there is no easy way to determine when a lock file can be deleted manually.
  • Proposed creative solutions [9][10].
  • Josh proposes having offset locks. Create a single file X size to store locks, instead of having a bunch of files representing locks.
    • Clint says this just makes the directory of lock files look clean, but still leaves each offset stale and has to cleaned anyway.
      • Idea: having a cron job which deletes lock files within a set reasonable time.
  • Ben Nemec feels this is largely cosmetic complaints about not cleaning up old files. The amount of trouble we’ve had with interprocess locking in the past, he has never felt that a cosmetic issue was sufficient reason to relook at the issue.
  • Clint feels what’s missing is metadata for cleaning up stale locks. That can be done with:
    • fcntl locks – You have the kernel telling you for sure if the locking process is still alive when you want to clean up and take the lock.
    • Creation locks – You need to write that information into the locks, and remove it, and then have a way to make sure the process is alive and know it has the lock, which isn’t simple.

Recording Milestones For Unmanaged Projects

  • Projects with non-release:managed tag should:
    • Prepare a patch to the deliverable file in openstack/releases repository recording the existing beta tag for your release. Include in the commit message that you are recording an existing milestone tag.
    • Prepare a patch to the project repository removing the version line from setup.cfg. This patch should depend on the release patch with the topic “remove-version-from-setup”.
    • Add a comment to the milestone tag request linking to the review from step 2.

Cross-Project Spec Liaisons

  • Mike Perez writes about problems we have with cross-project specs today:
    • Authors of specs can’t progress forward with specs because of lack of attention. Eventually getting frustrated and giving up.
    • Some projects could miss a cross-project spec being approved by the TC.
  • A proposal was given on the list and discussed at the cross-project meeting [11] to have cross-project spec liaisons from each project with the following responsibilities:
    • Watching the cross-project spec repo [12].
      • Comment on specs that involve your project. +1 to carry forward for TC approval.
        • If you’re not able to provide technical guidance on certain specs for your project, it’s up to you to get the right people involved.
        • Assuming you get someone else involved, it’s up to you to make sure they keep up with communication.
  • Communicate back to your project’s meeting on certain cross-project specs when necessary. This is also good for the previous bullet point of sourcing who would have technical knowledge for certain specs.
  • Attend the cross-project meeting when it’s called for [13].

Announcing a new cloud provider for OpenStack’s CI system: OVH

The OpenStack Project Infrastructure Team has added OpenStack Compute instances from OVH to the project’s continuous integration system.

There are a lot of ways to participate in an Open Source community like the OpenStack project and OVH is doing so in a way that best matches their strengths: contributing cloud resources to the project to drive our test and automation systems — resources which are critical to ongoing development.

OpenStack runs up to 30,000 automated jobs each day to test proposed changes and build documentation and software artifacts. The system responsible for this is itself a large multi-cloud application. It utilizes OpenStack Compute instances from Rackspace, HP Cloud, and now OVH to provide this service to developers. Each of those instances runs a single job and is then deleted in order to ensure the next job starts with a clean slate. This puts an extraordinarily high demand on our providers.

We are very excited by the addition of OVH, not only because it will help us scale to meet the needs of a growing project, but also because it demonstrates one of OpenStack’s key features: cross-cloud compatibility. When a developer uploads a proposed change to an OpenStack project, available instances from any of our contributing cloud providers will be used interchangeably to test it.

We are able to do this with the help of a number of OpenStack projects. First, we create a base image to use on all of our providers using Diskimage-builder. This ensures that all of our tests operate in an identical environment. Our test resource manager, Nodepool, uses the shade library to upload each of these images to our providers using the OpenStack Image service. Each of our providers is configured slightly differently, but shade handles the details so that Nodepool can focus on the business logic. Nodepool then identifies demand from our automation system, Zuul, to create OpenStack Compute instances as needed.

The Infrastructure Team thanks OVH for their contribution as well as our existing cloud providers, and we are excited about sharing our experience using this very demanding OpenStack application with all of them.

Technical Committee Highlights November 27, 2015

Welcome back from Tokyo. While there, I did not realize a three-dimensional subway map exists for Tokyo, but I sure loved traveling on the subway.

Welcoming the latest projects to OpenStack

Speaking of amazing cities and their subway maps, we should mention the growing list of OpenStack projects. We welcome these projects to OpenStack governance since the OpenStack Summit.

    • Monitoring – both OpenStack and its resources: monasca
    • Backups for file systems using OpenStack: freezer
    • Deployment for OpenStack: fuel
    • Cluster management service for Compute and Orchestration: senlin
    • Integrate Hyper-V, Windows and related components into OpenStack: winstackers

During these last weeks, the TC also had other project reviews requests that were put on hold for later once those projects and/or teams are more mature and ready to join the Big Tent.

Reports from TC Working Groups

Project Team Guide

The Project Team Guide team held a session back in Tokyo to discuss the next steps for this project. As a result of that session, more content will be created (or moved from the wiki): add community best practices, detail the benefits and trade-offs of the various release models, introduce deliverables and tags (as maintained in the governance repo’s projects.yaml), detail what common infrastructure projects can build on, and so on.

Communications Group

The communications working group (the one that brings these blog posts to you) will continue to operate under the same model. Announcements, summaries and communications will be sent out as they have been during the last cycle. Remember that feedback is always welcome and the group is looking for ways to improve. Talk back to us, we’re listening!

Project Tags

These are the latest new project tags created by the Technical Committee.

    • team:single-vendor: A new tag was added to communicate when a project team is currently driven by a single organization. We had some discussion about using the term “vendor” or “organization” but this intent is to show the opposite of a diversity in the team’s makeup.
    • assert:supports-upgrade: A new tag has been added to communicate when a project supports upgrades. Teams should apply this tag to their project if they assert they intend to support ongoing upgrades.
    • assert:supports-rolling-upgrade: A new tag has been added to communicate when a project supports rolling upgrades. Team should apply this tag to their project if they assert that operators can expect to perform rolling upgrades of their project, where the service can remain running while the upgrade is performed.

OpenStack Developer Mailing List Digest November 21-27

Success Bot Says

  • vkmc: We got 7 new interns for the Outreachy program December-March 2015 round.
  • bauzas: Reno in place for Nova release notes.
  • AJaeger: We now have Japanese Install Guides published for Liberty [1].
  • robcresswell: Horizon had a bug day! We made good progress on categorizing new bugs and removing old ones, with many members of the community stepping up to help.
  • AJaeger: The OpenStack Architecture Design Guide has been converted to RST [2].
  • AJaeger: The Virtual Machine Image guide has been converted to RST [3].
  • Ajaeger: Japanese Networking Guide is published as draft [4].
  • Tell us yours via IRC with a message “#success [insert success]”.

Release countdown for week R-18, Nov 30 – Dec 4

  • All projects following the cycle-with-milestones release model should be preparing for the milestone tag.
  • Release Actions:
    • All deliverables must have Reno configured before adding a Mitaka-1 milestone tag.
    • Use openstack/releases repository to manage the Mitaka-1 milestone tags.
    • One time change, we will be simplifying how we specify the versions for projects by moving to only using tags instead of the version entry for setup.cfg.
  • Stable release actions: Review stable/liberty branches for patches that have landed since the last release and determine if your deliverables need new tags.
  • Important dates:
    • Deadline for requesting a Mitaka-1 milestone tag: December 3rd
    • Mitaka-2: Jan 19-21
    • Mitaka release schedule [5]

Common OpenStack ‘Third-Party’ CI Solution – DONE!

  • Ramy Asselin who has been spearheading the work for a common third-party CI solution announces things being done!
    • This solution uses the same tools and scripts as the upstream Jenkins CI solution.
    • The documentation for setting up a 3rd party ci system on 2 VMs (1 private that runs the CI jobs, and 1 public that hosts the log files) is now available here [6] or [7].
    • There a number of companies today using this solution for their third party CI needs.

Process Change For Closing Bugs When Patches Merge

  • Today when a patch merges with ‘Closes-Bug’ in the commit message, that marks the associated bug as ‘Fix Committed’ to indicated fixed, but not in the release yet.
  • The release team uses automated tools to mark bugs from ‘Fix Committed’ to ‘Fix Released’, but they’re not reliable due to Launchpad issues.
  • Proposal for automated tools to improve reliability: Patches with ‘Closes-Bug’ in the commit message to have the bug status mark the associated bug as ‘Fix Released’ instead of ‘Fix Committed’.
  • Doug would like to have this be in effect next week.

Move From Active Distrusting Model to Trusting Model

  • Morgan Fainberg writes most projects have a distrusting policy that prevents the following scenario:
    • Employee from Company A writes code
    • Other Employee from Company A reviews code
    • Third Employee from Company A reviews and approves code.
  • Proposal for a trusting model:
    • Code reviews still need 2x Core Reviewers (no change)
    • Code can be developed by a member of the same company as both core reviewers (and approvers).
    • If the trust that is being given via this new policy is violated, the code can [if needed], be reverted (we are using git here) and the actors in question can lose core status (PTL discretion) and the policy can be changed back to the “distrustful” model described above.
  • Dolph Mathews provides scenarios where the “distrusting” model either did or would have helped:
    • Employee is reprimanded by management for not positively reviewing & approving a coworkers patch.
    • A team of employees is pressured to land a feature with as fast as possible. Minimal community involvement means a faster path to “merged,” right?
    • A large group of reviewers from the author’s organization repeatedly throwing *many* careless +1s at a single patch. (These happened to not be cores, but it’s a related organizational behavior taken to an extreme.)

Stable Team PTL Nominations Are Open

  • As discussed [8][9] of setting up a standalone stable maintenance team, we’ll be organizing PTL elections over the coming weeks.
  • Stable team’s mission:
    • Define and enforce the common stable branch policy
    • Educate and accompany projects as they use stable branches
    • Keep CI working on stable branches
    • Mentoring/growing the stable maintenance team
    • Create and improve stable tooling/automation
  • Anyone who successfully contributed to a stable branch back port over the last year can vote in the stable PTL election.
  • If interested, reply to thread with your self-nomination.
  • Deadline is 23:59 UTC Monday, November 30.
  • Election will be the week after.
  • Current candidates:
    • Matt Riedmann [10]
    • Erno Kuvaja [11]

Using Reno For Libraries

  • Libraries have two audiences for release notes:
    • Developers consuming the library.
    • Deployers pushing out new versions of the libraries.
  • To separate the notes from the two audiences and avoid doing manually something that we have been doing automatically, we can use Reno for just deployer release notes.
  • Library repositories that need Reno should have it configured like service projects, with separate jobs and a publishing location different from their developer documentation [12]

Releases VS Development Cycle

  • Thierry writes that as more projects enter the Big Tent, there have recently been questions about release models and development cycle.
  • Some projects want to be independent of the common release cycle and dates, but still keep some adherence to the development cycle. Examples:
    • Gnocchi wants to be completely independent of the common development cycle, but still maintain stable branches.
    • Fuel traditionally makes their releases a few months behind the OpenStack release to integration all the functionality there.
  • All projects above should current be release:independent, until they are able to (and willing) to coordinate their own releases with the projects following the common release cycle.
  • The release team currently supports 3 models:
    • release:cycle-with-milestones is the traditional Nova model with one release at the end of a 6-month dev cycle and a stable branch derived from that.
    • release:cycle-with-intermediary is the traditional Swift model, with releases as-needed during the 6-month development cycle, and a stable branch created from the last release in the cycle.
    • release:independent, for projects that don’t follow the release cycle at all.
      • Can make a release supporting the Mitaka release (including stable updates).
        • Can call the branch stable/mitaka even after the Mitaka release cycle, as long as the branch is stable and not development.
      • Should clearly document that their release is *compatible with* the OpenStack Mitaka release, rather than *part of* the Mitaka release.