Developer Mailing List Digest December 9-15th

Success Bot Says

  • mordred on #openstack-sdks: “Finished the shade transition to pure REST away from client libs” [0]
  • Tell us yours in OpenStack IRC channels using the command “#success <comment>”

Community Summaries

  • TC report 50 [0]
  • POST /api-sig/news [1]
  • Release countdown [2]
  • Technical Committee Status Update [3]
  • Resource Providers Update 45 [4]

Switching to Longer Development Cycles

Our self-imposed rhythm no longer matches our natural pace. Our elections, feature freezes feel like they’re around the corner, and due to that, we’re losing time from them, as multiple people have stated. The pace was designed around more people contributing full time to OpenStack, but lately, we have composite jobs, or were participating in multiple communities (which is great!).
It means:
  • One OpenStack coordinated release per year.
  • Maintain one stable branch per year.
  • Elect PTLs once a year.
  • One set of community goals per year.
  • One PTG per year.
Any project that still wants to release often can use the cycle-with-intermediary release model [0]. At a minimum, we’ll release once a year. Teams can choose to revive midcycle if they need to meet more than the one PTG each year. We’ll have more time for community-wide goals, so we will have more time to complete or even set more ambitious goals.
This doesn’t simplify upgrades. While upgrading every year is incrementally better than being forced to upgrade every 6 months. The real solution is better support for skipping releases.
It also doesn’t give us LTS. The cost of maintaining branches is not really due to the number of them in parallel, it’s due to the age of the oldest one. The real solution here is being discussed by the (still forming) LTS SIG. Extending stability periods is per-project at this point, and the proposal has yet to address a coordinated stability period.
We’re doing a year because of the way the events are organized. It’s suggested to start for the Rocky release and have the single PTG in 2018 in Dublin. The release team is opening the discussion to the community, and the TC will give a final decision.
Various cons were expressed, like this causing a rush to get in code at the end of a release to avoid having to wait an entire year. Projects are also forced to do compatibility support for versioned APIs, config files, etc since projects would be unable to drop compatibility in an intermediate release. Projects like Grenade will have to support the cases of some projects doing intermediate releases, and those doing yearly.
For the people that spend 20% of their time contributing to OpenStack, it has been voiced that it takes time to get a feature merged and the current cycle makes it impossible. This longer development cycle could help with allowing more time for those people. Various people expressed that we should be looking at the root cause of helping our part-time contributors, as the length of the cycle is unlikely the cause. People who can only contribute 20% of their time are also likely to deal with rebase conflicts, instead of focusing on their code.
Having a year could also impose having intermediate releases so specifications that can have multiple times a year to be approved. If that’s the idea, however, then it’s causing stress on the core team by having to match these precise schedules, in which the proposal was aiming to ease. It was a concern that this more solves the problem of giving more opportunities to people to get their spec/new feature considered.
This topic has also been brought up by various times in the past. Someone noted Daniel Berrange’s thread [1]

Zuul Dashboard Available

In addition to the Zuul dashboard [0] showing the “Status”, there are additional tabs added. “Jobs” page shows a list of all jobs in the system and their descriptions. The “Builds” page lists the most recent runs. You can query pipeline, job, and project.

Security SIG

Following previous mailing list discussions [0], the Security Project will be changing into a Special Interest Group (SIG). SIGs have shown to be a good match for the activity the group does around a topic or practice that spans around the community of developers, operators, and users. The group will continue to manage and care for the Security Guide, OSSNs, Bandit, Thread Reviews, Syntribos as well as encourage and incubate new security projects. The group will continue to work with the VMT and will keep a Sec-core group for launchpad that can work with the embargoes issues.

Cycle Highlights Reminder

As we get closer to Queens-3 and our final RCs, a reminder is given about the new ‘cycle-highlights’ that has been added to deliverable info. Background of why this was added, some PTLs were being pinged multiple times every release cycle by various people like journalists, product manager and other to compile the same information. To mitigate that, we have a way to capture highlights as part of the release. This will give basic information, but not as a complete marketing message.
This is done in the openstack/releases repository in the deliverables/queens/$PROJECT.yaml file formatted like so:
    cycle-highlights:
        – Introduced new service to use the unsed host to mine bitcoin.
We have three different places that document activities for three different audiences:
Commit messages: Developer documentation
Release notes: End-user and deployer documentation
Cycle highlights: Journalists, product manager, and others.

Community Goals For Rocky

Some questions to ask ourselves: What common challenges do we have, and who is willing to drive that community-wide goal (aka champion).
A champion is someone who drives a goal but doesn’t commit to writing code necessarily. The champion will communicate with project PTLs about the goal, and make the liaison if needed.
The list of ideas for community-wide goals is collected on this etherpad [0]. Propose some ideas now!

Developer Mailing List Digest November 25 to December 1st

News

  • Project Team Gather (PTG) in Dublin registration is live [0]

Community Summaries

  • TC report by Chris Dent [0]
  • Release countdown [1]
  • Technical Committee Status updated [2]
  • POST /api-sig/news [3]
  • Nova notification update [4]
  • Nova placement resource providers update [5]

Dublin PTG Format

We will continue themes as we did in Denver (Monday-Tuesday), but shorter times like half days. Flexibility is added for other groups to book the remaining available rooms in 90-min slots on-demand driven by the PTG Bot (Wednesday-Friday.

First Contact SIG

A wiki has been created for the group [0]. The group is looking for intersted people being points of contact for newcomers and what specified time zones. Resource links like contributor portal, mentoring wiki, Upstream Institute, outreachy are being collected on the wiki page. A representative from the operators side to chair and represent would be good.

Policy Goal Queens-2 Update

Queens-2 coming to a close, we recap our community wide goal for policies [0]. If you want your status changed, contact Lance Bragstad. Use the topic policy-and-docs-in-code for tracking related code changes.

Not Started

  • openstack/ceilometer
  • openstack/congress
  • openstack/networking-bgpvpn
  • openstack/networking-midonet
  • openstack/networking-odl
  • openstack/neutron-dynamic-routing
  • openstack/neutron-fwaas
  • openstack/neutron-lib
  • openstack/solum
  • openstack/swift

In Progress

  • openstack/barbican
  • openstack/cinder
  • openstack/cloudkitty
  • openstack/glance
  • openstack/heat
  • openstack/manila
  • openstack/mistral
  • openstack/neutron
  • openstack/panko
  • openstack/python-heatclient
  • openstack/tacker
  • openstack/tricircle
  • openstack/trove
  • openstack/vitrage
  • openstack/watcher
  • openstack/zaqar

Completed

  • openstack/designate
  • openstack/freezer
  • openstack/ironic
  • openstack/keystone
  • openstack/magnum
  • openstack/murano
  • openstack/nova
  • openstack/octavia
  • openstack/sahara
  • openstack/searchlight
  • openstack/senlin
  • openstack/zun

Tempest Plugin Split Goal

A list of open reviews [0] is available for the Tempest plugin split goal [1].

Not Started

  • Congress
  • ec2-api
  • freezer
  • mistral
  • monasca
  • senlin
  • tacker
  • Telemetry
  • Trove
  • Vitrage

In Progress

  • Cinder
  • Heat
  • Ironic
  • magnum
  • manila
  • Neutron
  • murano
  • networking-l2gw
  • octavia

Completed

  • Barbican
  • CloudKitty
  • Designate
  • Horizon
  • Keystone
  • Kuryr
  • Sahara
  • Solum
  • Tripleo
  • Watcher
  • Winstackers
  • Zaqar
  • Zun

Developer Mailing List Digest November 18-27

Community Summaries

  • Glance priorities [0]
  • Nova placement resource provider update [1]
  • Keystone Upcoming Deadlines [2]
  • Ironic priorities and subteam reports [3]
  • Keystone office hours [4]
  • Nova notification update [5]
  • Release countdown [6]
  • Technical committee status update [7]

Self-healing SIG created

Adam Spiers announced the formation of a SIG around self-healing. Its scope is to coordinate the use and development of several OpenStack projects which can be combined in various ways to manage OpenStack infrastructure in a policy-driven fashion, reacting to failures and other events by automatically healing and optimising services.

Proposal for a QA SIG

A proposal to to have a co-existing QA special interest group (SIG) that would be a place for downstream efforts to have a common place in collaborating and sharing tests. Example today the OPNFV performs QA on OpenStack releases today and are actively looking for opportunities to share tools and test cases. While a SIG can exist to do some code, the QA team will remain for now since there are around 15 QA projects existing like Tempest and Grenade.

Improving the Process for Release Marketing

Collecting and summarizing “top features” during release time is difficult for both PTL’s and Foundation marketing. A system is now in place for PTL’s to highlight release notes [0]. Foundation marketing will work with the various teams if needed to understand and make things more press friendly.

Developer Mailing List Digest November 11-17

Summaries

  • POST /api-sig/news [0]
  • Release countdown for week R-14, November 18-24 [1]

[0] – http://lists.openstack.org/pipermail/openstack-dev/2017-November/124633.html
[1] – http://lists.openstack.org/pipermail/openstack-dev/2017-November/124631.html

 

Upstream Long Term Support Releases

The Sydney Summit had a very well attended and productive session [0] about to go about keeping a selection of past releases available and maintained for long term support (LTS).

In the past the community has asked for people who are interested in old branches being maintained for a long time to join the Stable Maintenance team with the premise that if the stable team grew, it could support more branches for longer periods. This has been repeated for about years and is not working.

This discussion is about allowing collaboration on patches beyond end of life (EOL) and enable whoever steps up to maintain longer lived branches to come up with a set of tests that actually match their needs with tests that would be less likely to bitrot due to changing OS/PYPI substrate. We need to lower expectations of what we’re likely to produce will get more brittle the older the branch gets. Any burden created by taking on more work is absorbed by people doing the work, as does not unduly impact the folks not interested in doing the work.

The idea is to continue the current stable policy more or less as it is. Development teams take responsibility of a couple of stable branches. At some point what we now call an EOL branch, instead of deleting it we would leave it open and establish a new team of people who want to continue to main that branch. It’s anticipated the members of those new teams are coming mostly from users and distributors. Not all branches are going to attract teams to maintain them, and that’s OK.

We will stop tagging these branches so the level of support they provide are understood. Backports and other fixes can be shared, but to consume them, a user will have to build their own packages.

Test jobs will run as they are, and the team that maintain the branch could decide how to deal with them. Fixing the jobs upstream where possible is preferred, but depending on who is maintaining the branch, the level of support they are actually providing and the nature of the breakage, removing individual tests or whole jobs is another option. Using third-party testing came up but is not required.

Policies for the new team being formed to maintain these older branches is being discussed in an etherpad [2].

Some feedback in the room expressed they to start one release a year who’s branch isn’t deleted after a year. Do one release a year and still keep N-2 stable releases around. We still do backports to all open stable branches. Basically do what we’re doing now, but once a year.

Discussion on this suggestion extended to the OpenStack SIG mailing list [1] suggesting that skip-release upgrades are a much better way to deal with upgrade pain than extending cycles. Releasing every year instead of a 6 months means our releases will contain more changes, and the upgrade could become more painful. We should be release often as we can and makes the upgrades less painful so versions can be skipped.

We have so far been able to find people to maintain stable branches for 12-18 months. Keep N-2 branches for annual releases open would mean extending that support period to 2+ years. If we’re going to do that, we need to address how we are going to retain contributors.

When you don’t release often enough, the pressure to get a patch “in” increases. Missing the boat and waiting for another year is not bearable.

[0] – https://etherpad.openstack.org/p/SYD-forum-upstream-lts-releases
[1] – http://lists.openstack.org/pipermail/openstack-sigs/2017-November/000149.html
[2] – https://etherpad.openstack.org/p/LTS-proposal

Developer Mailing List Digest October 28th – November 3rd

News

  • Sydney Summit Etherpads [0]

Summaries

  • Nova Placements Resource Provider Update by Eric Fried [0]
  • Nova Notification Update by Balazs Gibizer [1]
  • Technical Committee Status update by Thierry Carrez [2]
  • Technical Committee Report by Chris Dent [3]
  • Release Countdown by Sean McGinnis [4]
  • POST /api-sig/news by Chris Dent [5]

TC Election Results (continued)

Congrats to our 6 newly elected Technical Committee members:
  • Colleen Murphy (cmurphy)
  • Doug Hellmann (dhellmann)
  • Emilien Macchi (emilienm)
  • Jeremy Stanley (fungi)
  • Julia Kreger (TheJulia)
  • Paul Belanger (pabelanger)
Full results are available [0]. The process and results are also available [1]. 420 voted out of 2430 electorate, giving us a 17.28% turn out with a delta of 29.16% [2].
Reasons for the low turnout is hard to tell without knowing who is voting and what their activity is in the community. More people are beginning to understand the point of the TC activities, being more around duties than rights (e.g. stewardship and leadership). People could care a bit less about specific individuals and are less motivated by the vote itself. If the activity of the TC was a lot more conflict and a lot less consensus, people might care about it more.

Security SIG

Our governance used to only have project teams to recognize activity in OpenStack, so we created a security team. Introduction of sigs provide a new construct for recognizing activity around a group that share interest around a topic or practice that are not mainly around software bits.
Security is a great example of a topic that could benefit from this construct to gather all security-conscious people in our community. SIGs can have software by-products and own git repositories, and the software is more about security in general than a piece of OpenStack itself.
It’s important to consider the Vulnerability Management Team (VMT) under the new model, which acts as an independent task force.
The Security team discussed the idea of a SIG in their meeting, and overall think it’s worth exploring by having the SIG and team exist in parallel to see if there is traction.

Developer Mailing List Digest October 21-27

News

  • TC election results [0]
  • Next PTG will be in Dublin, the week of February 26, 2018. More details will be posted on openstack.org/ptg as soon as we have them. [1]

SuccessBot Says

  • gothamr_  [0]:changes to the manila driverfixes branches can finally be merged xD Thanks infra folks for ZuulV3!
  • andreaf [1]: Tempest test base class is now a stable API for plugins
  • More [2]
[1] – http://eavesdrop.openstack.org/irclogs/%23openstack-release/%23openstack-release.2017-10-24.log.html

Community Summaries

  • TC Report 43 by Chris Dent [0]
  • Nova Notification Update Week 43 by Balazs Gibizer [1]
  • POST /api-sig/news by Chris Dent [2]
  • Technical Committee Status Update by Thierry Carrez [3]
  • Nova Placements Resource Provider Update [4]

Time to Remove the Ceilometer API?

Summarized by Jeremy Stanley

The Ceilometer REST API was deprecated in Ocata, a year ago, and the User Survey indicates more than half its users have switched to the non-OpenStack Gnocchi service’s API instead (using Ceilometer as a backend). The Ceilometer install guide has also been recommending Gnocchi at least as long ago as Newton. The old API has become an attractive nuisance from the Telemetry team’s perspective, and they’d like to go ahead and drop it altogether in Queens.

Keystone v2.0 API Removal

Summarized by Thierry Carrez

Keystone Queen’s PTL Lance Bragstad gives notice that the Queen’s release will not be included v2.0, except the ec2-api. This is being done after a lengthy given deprecation period.

Developer Mailing List Digest September 30 – October 6

Summaries

Sydney Forum Schedule Available

TC Nomination Period Is Now Over

Prepping for the Stable/Newton EOL

  • The published timeline is:
    • Sep 29 : Final newton library releases
    • Oct 09 : stable/newton branches enter Phase III
    • Oct 11 : stable/newton branches get tagged EOL
  • Given that those key dates were a little disrupted, Tony Breeds is proposing adding a week to each so the new timeline looks like:
    • Oct 08 : Final newton library releases
    • Oct 16 : stable/newton branches enter Phase III
    • Oct 18 : stable/newton branches get tagged EOL
  • Thread

Policy Community Wide Goal Progress

Tempest Plugin Split Community Wide Goal Progress

  • The goal
  • The reviews
  • List of projects which have already completed the goal:
    • Barbican
    • Designate
    • Horizon
    • Keystone
    • Kuryr
    • Os-win
    • Sahara
    • Solum
    • Watcher
  • List of projects which are working on the goal:
    • Aodh
    • Cinder
    • Magnum
    • Manila
    • Murano
    • Neutron
    • Neutron L2GW
    • Octavia
    • Senlin
    • Zaqar
    • Zun
  • Message

User Group Newsletter – September 2017

Sydney Summit news

Let’s get excited! The Sydney Summit is getting close, and is less than 30 days away!!

Get all your important information in this Summit guide. It includes suggestions about where to stay, featured speakers, a Summit timeline and much more.

The schedule is LIVE! Plan it via the website or on the go with the Summit app. Stuck on where to start with your schedule? See the best of what the Sydney Summit has to offer with this Superuser article.

A SUPER IMPORTANT NOTE REGARDING TRAVEL!!

*All* non-Australian residents will need a visa to travel to Australia (including United States citizens). Click here for more information

Forum

The Forum is where  (users and developers) gather to brainstorm the requirements for the next release, gather feedback on the past version and have strategic discussions that go beyond just one release cycle.

The Forum schedule brainstorming is well underway! Check out the link below for key dates.

You can read more about the Forum here.

#HacktheStack – Cloud App Hackathon

Join us for Australia’s first OpenStack Application Hackathon Nov 3-5, 2017 at Doltone House in Australia’s Technology park, the weekend prior to the OpenStack Summit Sydney.

This 3-day event is organized by the local OpenStack community and welcomes students and professionals to hack the stack using the most popular open infrastructure platforms and application orchestration tools such as OpenShift (Kubernetes and Docker container orchestrator), Cloudify (TOSCA and Docker app orchestrator) and Agave (Science-as-a-Service gateway), in addition to the premier Open Source Infrastructure-as-a-Service: OpenStack!

There are great opportunities to get involved! You can sign up as a participant or share your expertise as a mentor. There are also some fantastic sponsorship opportunities available.

Click for more information here

 

2018 Summit news – Save the dates!

Back by popular demand, Vancouver is our first Summit destination in 2018. Mark your calendar for May 21-24, 2018.

Our second summit for 2018 will be heading to Berlin! Save the dates for November 13-15th!

New User Groups

We welcome our newest User Groups!

Looking for your local user group or want to start one in your area? Head to the groups portal.

OpenDev

Did you miss OpenDev? Read a great event summary here.

Catch up on all the talks with the event videos here.

 

OpenStack Days  

OpenStack Days bring together hundreds of IT executives, cloud operators and technology providers to discuss cloud computing and learn about OpenStack. The regional events are organized and hosted annually by local OpenStack user groups and companies in the ecosystem, and are often one or two-day events with keynotes, breakout sessions and even workshops. It’s a great opportunity to hear directly from prominent OpenStack leaders, learn from user stories, network and get plugged into your local community.

See when and where the upcoming OpenStack Days are happening.

 

OpenStack Marketing Portal

There is some fantastic OpenStack Foundation content available on the Marketing Portal.

This includes materials like:

  • OpenStack 101 slide deck
  • 2017 OpenStack Highlights & Stats presentation
  • Collateral for events (Sticker and T-Shirt designs)

Latest from Superuser

How to install the OpenStack Horizon Dashboard

How to get more involved in OpenStack

How to make your Summit talk a big success

How to deliver network services at the edge

Kickstart your OpenStack skills with an Outreachy Internship

 

Have you got a story for Superuser? Write to editor[at]openstack.org.

 

On a sad note…a farewell

Last week, an extremely valued member of our community, Tom Fifield, announced his departure as Community Manager from the OpenStack Foundation.

Tom, we thank you for your amazing industrious efforts over the last five years! Your work has contributed to culminating in the healthy community we have today, across more than 160 countries, where users and developers collaborate to make clouds better for the work that matters.

Thank you Tom!!

Read his full announcement here

 

Contributing to the User Group Newsletter.

If you’d like to contribute a news item for next edition, please submit to this etherpad.

Items submitted may be edited down for length, style and suitability.

 

 

 

 

 

 

 

“Dear Boss, I want to attend OpenStack Summit Sydney”

Want to attend the OpenStack Summit Sydney but need help with the right words for getting your trip approved? While we won’t write the whole thing for you, here’s a template to get you going. It’s up to you to decide how the Summit will help your team, but with free workshops and trainings, technical sessions, strategy talks and the opportunity to meet thousands of likeminded Stackers, we don’t think you’ll have a hard time finding an answer.

 

Dear [Boss],

I would like to attend the OpenStack Summit in Sydney, November 6-8, 2017. The OpenStack Summit is the largest conference for the open source cloud platform OpenStack, and the only one where I can get free OpenStack training, learn how to contribute code upstream to the project, and meet with other users to learn how they’ve been using OpenStack in production. The Summit is an opportunity for me to bring back knowledge about [Why you want to attend! What are you hoping to learn? What would benefit your team?] and share it with our team, while helping us get to know similar OpenStack-minded teams from around the world.

Companies like Commonwealth Bank, Tencent and American Airlines will be presenting, and technical sessions will demonstrate how teams are integrating other open source projects, like Kubernetes with OpenStack, to optimize their infrastructure. I’ll also be able to give Project Teams feedback about OpenStack so our user needs can be incorporated into upcoming software releases.

You can browse past Summit content at openstack.org/videos to see a sample of the conference talks.

The OpenStack Summit is the opportunity for me to expand my OpenStack knowledge, network and skills. Thanks for considering my request.

[Your Name]

 

Learn more about the Summit and register at openstack.org/summit/sydney-2017/

Tags:

Developer Mailing List Digest September 23-29 2017

Summaries

Sydney Forum

General Links

Etherpads (copied from Sydney Forum wiki)

Catch-alls

If you want to post an idea, but aren’t working with a specific team or working group, you can use these:

Etherpads from Teams and Working Groups

Garbage Patches for Simple Typos Fixes

  • There is some agreement that we as a community have to do something beyond mentoring new developers.
    • Others have mentioned that some companies are doing this to game the system in other communities besides OpenStack.
      • Gain: show a high contribution level was “low quality” patches.
    • Some people in the community want to put a stop to this figuratively with a stop sign, otherwise, things will never improve. If we don’t do something now we are hurting everyone, including those developers who could have done more meaningful contributions.
    • Others would like before we go into creating harsh processes, we need to collect the data to show other times to provide guidance I Have not worked.
    • We have a lot of anecdotal information right now that we need to collect and summarize.
    • If the results show that there are clear abuses, rather than misunderstandings, then we can use that data to design effective blocks without hurting other contributors or creating a reputation that our community is not welcoming.
  • Some are unclear why there is so much outrage about these patches, to begin with. They are fixing real things.
    • Maybe there is a CI cost, but the faster they are merged the less likely someone is to propose it in the future which keeps the CI cost down.
    • If people are deeply concerned about CI resources, step one is to give us a better accounting into their existing system to see where resources are currently spent.
  • Thread

Status of the Stewardship Working Group

  • The stewardship working group was created after the first session of leadership training that the Technical Committee, User Committee, Board and other community members were invited to participate in 2016.
  • Follow-up on what we learned at ZingTrain and push adoption of the tools we discovered there.
  • While we did (and continue)
    • The activity of the workgroup mostly died when we decided to experiment getting rid of weekly meetings for greater inclusion.
    • Lost original leadership.
  • The workgroup is dormant until someone steps up and leads it again.
  • Join us on IRC Freenode in channel openstack-swg if interested.
  • Message

Improving the Process for Release Marketing

  • Release marketing is a critical heart for sharing what’s new with each release.
  • Let’s work together on reworking how the marketing community and projects work together to make the release communications happen.
  • Having multiple, repetitive demands to summarize” top features” during release time can be a pester, and having to recollect the information each time isn’t an effective use of time.
    • Being asked to make a polished, “Press–Friendly” message out of release can feel far outside of the PTL’s Focus areas or skills.
    • Technical content marketers, attempting to find the key features from release notes, mailing lists, specifications, roadmaps, whatever means interesting features are sometimes overlooked.
  • To address this gap, the release team and foundation marketing team proposed collecting information as part of the release tagging process.
    • We will collect from deliverable files to provide highlights for the series (about three items).
    • The text will be used to build a landing page on release.openstack.org that shows the”Key features” flagged by PTL’s that the marketing teams should be looking at during release communication times.
      • This page will link to the release notes so marketers can gather additional information.
  • Message

Simplification in OpenStack

  • Two camps appear: people that want to see OpenStack as a product with a way of doing deployments and the people who want to focus on configuration management tools.
    • One person gives an example of using both Ubuntu MAAS and Puppet. The puppet solution allowed for using existing deployment methodologies unlike the former.
  • We should start promoting and using a single solution for the bulk of the community efforts. Right now we do that with Devstack as a reference implementation that nobody should use for anything but dev/test.
    • This sort of idea could make other deployment efforts relevant.
  • Kolla came up at the PTG: scenario-based testing and documentation based on different  “constellations” or use cases.
    • Puppet has been doing this and Triple-o has been doing this.
    • If you break down actual use cases, most people want nova (qemu+KVM), neutron (vxlan, potentially VLAN), Cinder (ceph).
      • If we agreed to cover 90% of users, that’ll boil down to 4 to 5 different “constellations.”
    • Someone has been working on a local testing environment, and it boils down to this.
  • Thread