{"id":8098,"date":"2017-05-30T20:10:33","date_gmt":"2017-05-31T01:10:33","guid":{"rendered":"http:\/\/www.openstack.org\/blog\/?p=8098"},"modified":"2017-05-30T20:10:33","modified_gmt":"2017-05-31T01:10:33","slug":"openstack-developer-mailing-list-digest-20170526","status":"publish","type":"post","link":"https:\/\/www.openstack.org\/blog\/openstack-developer-mailing-list-digest-20170526\/","title":{"rendered":"OpenStack Developer Mailing List Digest May 20-26"},"content":{"rendered":"<h1 class=\"p1\"><b>SuccessBot Says<\/b><\/h1>\n<ul>\n<li class=\"li4\"><span class=\"s1\">clarkb <a href=\"http:\/\/eavesdrop.openstack.org\/irclogs\/%23openstack-infra\/%23openstack-infra.2017-05-24.log.html\"><span class=\"s2\">1<\/span><\/a> : infra added city cloud to the pool of test nodes.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">pabelanger <a href=\"http:\/\/eavesdrop.openstack.org\/irclogs\/%23openstack-infra\/%23openstack-infra.2017-05-24.log.html\"><span class=\"s2\">2<\/span><\/a> : opensuse-422-infracloud-chocolate-8977043 launched by nodepool.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">All: <a href=\"https:\/\/wiki.openstack.org\/wiki\/Successes\"><span class=\"s2\">3<\/span><\/a><\/span><\/li>\n<\/ul>\n<h1 class=\"p3\"><b>etcd 3.x as a Base Service<\/b><\/h1>\n<ul>\n<li class=\"li4\"><span class=\"s1\">A devstack review <a href=\"https:\/\/review.openstack.org\/#\/c\/445432\/\"><span class=\"s2\">4<\/span><\/a> that adds a new etcd3 service.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">Two options to enable the DLM use case with Tooz (for eventless based services) <a href=\"https:\/\/review.openstack.org\/#\/c\/466098\/\"><span class=\"s2\">5<\/span><\/a> <a href=\"https:\/\/review.openstack.org\/#\/c\/466109\/\"><span class=\"s2\">6<\/span><\/a><\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">Full thread: <a href=\"http:\/\/lists.openstack.org\/pipermail\/openstack-dev\/2017-May\/thread.html#117370\"><span class=\"s2\">7<\/span><\/a><\/span><\/li>\n<\/ul>\n<h1 class=\"p3\"><b>Do We Want to be Publishing Binary Container Images?<\/b><\/h1>\n<ul>\n<li class=\"li4\"><span class=\"s1\">During the Forum, the discussion on collaboration between various teams building or consuming container images.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">Decide how to publish images from the various teams to docker hub or other container registries.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">The community has refrained from publishing binary packages in other formats such as debs and RPMs. Instead we have left this to the responsibility of the downstream consumers to build production packages.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">This would require more tracking of upstream issues (bugs, CVEs, etc) to ensure the images are updated as needed.<\/span>\n<ul>\n<li class=\"li4\"><span class=\"s1\">Given our security and stable team resources, this might not be a good idea at this time.<\/span><\/li>\n<\/ul>\n<\/li>\n<li class=\"li4\"><span class=\"s1\">Kolla is interested in doing this for daily builds. Everything is licensed with ASL which gives no guarantees.<\/span>\n<ul>\n<li class=\"li4\"><span class=\"s1\">Even if you mark something to not be used in production, people still use it. Take the recent user survey with DevStack being used in production.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">Kolla today publishes build instructions. Manually every release they provide built containers.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">Built containers would run through our CI gate, so others don\u2019t have to have a local CI build pipeline.<\/span><\/li>\n<\/ul>\n<\/li>\n<li class=\"li4\"><span class=\"s1\">Things we publish to Pypi are different from this proposal:<\/span>\n<ul>\n<li class=\"li4\"><span class=\"s1\">The formats published by pypi are source formats (sdist) and developer friend but production ready format (wheel).<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">Most of our services are not packaged and published to PyPi. The libraries are to make them easy to consume in our CI.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">The artifacts in PyPi contain references to dependencies, the dependencies are not built into the packages themselves.<\/span><\/li>\n<\/ul>\n<\/li>\n<li class=\"li4\"><span class=\"s1\">Iterating on the infra-spec review for publishing to DockerHub has started <a href=\"https:\/\/review.openstack.org\/447524\"><span class=\"s2\">8<\/span><\/a><\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">Full thread: <a href=\"http:\/\/lists.openstack.org\/pipermail\/openstack-dev\/2017-May\/thread.html#116677\"><span class=\"s2\">9<\/span><\/a><\/span><\/li>\n<\/ul>\n<h1 class=\"p3\"><b>RFC Cross Project Request ID Tracking<\/b><\/h1>\n<ul>\n<li class=\"li4\"><span class=\"s1\">In the logging Forum session, it was brought up how much effort operators are having to put into reconstructing flows for things like server boot when they go wrong.<\/span>\n<ul>\n<li class=\"li4\"><span class=\"s1\">Jumping from service to service, the request-id is reset to something new.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">Being able to query in elastic search for the same request-id in communication between services would be useful.<\/span><\/li>\n<\/ul>\n<\/li>\n<li class=\"li4\"><span class=\"s1\">There is a concern of trusting the request-id on the wire, because it\u2019s coming from a random user.<\/span>\n<ul>\n<li class=\"li4\"><span class=\"s1\">We have a new concept of \u201cservice users\u201d which are set of higher privilege services that we are using to wrap user requests.<\/span><\/li>\n<\/ul>\n<\/li>\n<li class=\"li4\"><span class=\"s1\">Basic idea is:<\/span>\n<ul>\n<li class=\"li4\"><span class=\"s1\">services will optionally take an inbound X-OpenStack-Request-ID which we\u2019ll strongly validate req-$uuid format. <\/span>\n<ul>\n<li class=\"li4\"><span class=\"s1\">They will continue to generate one as well.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">When the context is built we\u2019ll check the service user was involved, and if not, reset the request-id to the local generated one.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">Both request-ids will be logged.<\/span><\/li>\n<\/ul>\n<\/li>\n<li class=\"li4\"><span class=\"s1\">Python clients and callers will need to be augmented to pass the request-id in on requests.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">Servers will opt into calling other services this way.<\/span><\/li>\n<\/ul>\n<\/li>\n<li class=\"li4\"><span class=\"s1\">Oslo spec for this has been merged <a href=\"https:\/\/review.openstack.org\/#\/c\/464746\/\">1<span class=\"s2\">0<\/span><\/a>.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">Full thread: <a href=\"http:\/\/lists.openstack.org\/pipermail\/openstack-dev\/2017-May\/thread.html#116619\">1<span class=\"s2\">1<\/span><\/a><\/span><\/li>\n<\/ul>\n<h1 class=\"p3\"><b>Can We Stop Global Requirements Update (Cont.)<\/b><\/h1>\n<ul>\n<li class=\"li4\"><span class=\"s1\">Gnocchi has gate issues with Babel this time. Julien plans to remove all oslo dependencies over the next few months.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">The project Cotyledon was presented at some summit ago as an alternative to oslo.service and getting rid of eventless. The library lives under the telemetry umbrella for now.<\/span>\n<ul>\n<li class=\"li4\"><span class=\"s1\">The project doesn\u2019t live under oslo so that it\u2019s encouraged for the greater python ecosystem to adopt and help maintain it.<\/span><\/li>\n<\/ul>\n<\/li>\n<li class=\"li4\"><span class=\"s1\">Octavia is also using Cotyledon.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">Full thread: <a href=\"http:\/\/lists.openstack.org\/pipermail\/openstack-dev\/2017-May\/thread.html#116736\">1<span class=\"s2\">2<\/span><\/a><\/span><\/li>\n<\/ul>\n<h1 class=\"p3\"><b>Revised Postgresql Deprecation Patch for Governance<\/b><\/h1>\n<ul>\n<li class=\"li4\"><span class=\"s1\">In the Forum session we agreed to the following:<\/span>\n<ul>\n<li class=\"li4\"><span class=\"s1\">Explicitly warn in operator facing documentation Postresql is less supported than MySQL.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">Sure is the process of investigating migration from Postgresql to Gallera for future versions of OpenStack products.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">TC governance patch is updated <a href=\"https:\/\/review.openstack.org\/#\/c\/427880\/\">1<span class=\"s2\">3<\/span><\/a>.<\/span><\/li>\n<\/ul>\n<\/li>\n<li class=\"li4\"><span class=\"s1\">Current sticking points:<\/span>\n<ul>\n<li class=\"li4\"><span class=\"s1\">It&#8217;s important that the operator community largely is already in one camp or not.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">Future items listed that are harder are important enough to justify a strict trade off here.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">It&#8217;s ok to have the proposal have a firm lean in tone, even though it&#8217;s set of concrete actions are pretty reversible and don&#8217;t commit to future removal of Postgresql.<\/span><\/li>\n<\/ul>\n<\/li>\n<li class=\"li4\"><span class=\"s1\">What has been raised as being hard by an abstraction layer like SQLAlchemy:<\/span>\n<ul>\n<li class=\"li4\"><span class=\"s1\">OpenStack services taking a more active role in managing DBMS.<\/span>\n<ul>\n<li class=\"li4\"><span class=\"s1\">See Active or passive role with our database layer summary below for this discussion.<\/span><\/li>\n<\/ul>\n<\/li>\n<li class=\"li4\"><span class=\"s1\">The ability to have zero down time upgrade for services such as Keystone.<\/span>\n<ul>\n<li class=\"li4\"><span class=\"s1\">Expand\/contract with code and carefully dancing around the existence of two schema concepts simultaneously (e.g. Nova and Neutron).<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">This shouldn\u2019t be a problem because we use alembic or sqlalchemy-migrate to abstract away ALTER TABLE types.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">Expand\/contract using server side triggers to reconcile the two schema. This is more difficult because there is no abstraction layer that exists in SQLAlchemy. It could be feasible to build one specific to OpenStack.<\/span><\/li>\n<\/ul>\n<\/li>\n<li class=\"li4\"><span class=\"s1\">Consistent UTF-8 4 &amp; 5 byte support in our APIs<\/span>\n<ul>\n<li class=\"li4\"><span class=\"s1\">Unicode itself only needs 4 bytes and that is as far as any database supports right now. This problem has been solved by SQLAlchemy well before Python 3 existed.<\/span><\/li>\n<\/ul>\n<\/li>\n<li class=\"li4\"><span class=\"s1\">The requirement that Postgresql libraries are compiled for new users trying to just run unit tests.<\/span>\n<ul>\n<li class=\"li4\"><span class=\"s1\">New developers who aren\u2019t concerned with Postgresql don\u2019t have to run these tests.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">OpenStack went all the way with Kilo using the native python-MySQL driver which required compiling.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">This is OpenStack. We are the glue to thousands of c-compiled libraries and packages.<\/span><\/li>\n<\/ul>\n<\/li>\n<li class=\"li4\"><span class=\"s1\">Consistency around case sensitivity collation.<\/span>\n<ul>\n<li class=\"li4\"><span class=\"s1\">MySQL is defaulting to case-insensitive.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">Postgresql almost has no support for case-insensitive.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">SQLAlchemy supports things like ilike().<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">String datatype in SQLAlchemy guarantees case-insensitive.<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<li class=\"li4\"><span class=\"s1\">Top concerns that remain:<\/span>\n<ul>\n<li class=\"li4\"><span class=\"s1\">A1) Do not surprise users late by them only finding out they are on less traveled once they are so deeply committed. It\u2019s fine for users to choose the path, as long as they are informed they are going to need to be more self reliant.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">A2) Do not prevent features like zero downtime in Keystone making forward progress with a MySQL only solution.<\/span><\/li>\n<\/ul>\n<\/li>\n<li class=\"li4\"><span class=\"s1\">Orthogonal concerns:<\/span>\n<ul>\n<li class=\"li4\"><span class=\"s1\">B1) Postgresql was chosen by people in the past, maybe more than we realized, that\u2019s real users we don\u2019t want to throw under the bus. Whole sale delete is off the table. There\u2019s no clear path off and missing data of who\u2019s on it.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">B2) The upstream code isn\u2019t so irreparably changed (e.g. delete the SQLAlchemy layer) that it\u2019s not possible to have alternative database backends.<\/span><\/li>\n<\/ul>\n<\/li>\n<li class=\"li4\"><span class=\"s1\">The current proposal <a href=\"https:\/\/review.openstack.org\/#\/c\/427880\/\">1<span class=\"s2\">3<\/span><\/a> addresses A1 and B1.<\/span><\/li>\n<li class=\"li4\"><span class=\"s1\">Full thread: <a href=\"http:\/\/lists.openstack.org\/pipermail\/openstack-dev\/2017-May\/thread.html#116642\">1<span class=\"s2\">4<\/span><\/a><\/span><\/li>\n<\/ul>\n<p class=\"p4\" class=\"lead\"><span class=\"s1\">[1] &#8211; <a href=\"http:\/\/eavesdrop.openstack.org\/irclogs\/%23openstack-infra\/%23openstack-infra.2017-05-24.log.html\">http:\/\/eavesdrop.openstack.org\/irclogs\/%23openstack-infra\/%23openstack-infra.2017-05-24.log.htm<span class=\"s2\">l<\/span><\/a><\/span><\/p>\n<p class=\"p4\"><span class=\"s1\">[2] &#8211; <a href=\"http:\/\/eavesdrop.openstack.org\/irclogs\/%23openstack-infra\/%23openstack-infra.2017-05-24.log.html\">http:\/\/eavesdrop.openstack.org\/irclogs\/%23openstack-infra\/%23openstack-infra.2017-05-24.log.htm<span class=\"s2\">l<\/span><\/a><\/span><\/p>\n<p class=\"p4\"><span class=\"s1\">[3] &#8211; <a href=\"https:\/\/wiki.openstack.org\/wiki\/Successes\">https:\/\/wiki.openstack.org\/wiki\/Successe<span class=\"s2\">s<\/span><\/a><\/span><\/p>\n<p class=\"p4\"><span class=\"s1\">[4] &#8211; <a href=\"https:\/\/review.openstack.org\/#\/c\/445432\/\">https:\/\/review.openstack.org\/#\/c\/445432<span class=\"s2\">\/<\/span><\/a><\/span><\/p>\n<p class=\"p4\"><span class=\"s1\">[5] &#8211; <a href=\"https:\/\/review.openstack.org\/#\/c\/466098\/\">https:\/\/review.openstack.org\/#\/c\/466098<span class=\"s2\">\/<\/span><\/a><\/span><\/p>\n<p class=\"p4\"><span class=\"s1\">[6] &#8211; <a href=\"https:\/\/review.openstack.org\/#\/c\/466109\/\">https:\/\/review.openstack.org\/#\/c\/466109<span class=\"s2\">\/<\/span><\/a><\/span><\/p>\n<p class=\"p4\"><span class=\"s1\">[7] &#8211; <a href=\"http:\/\/lists.openstack.org\/pipermail\/openstack-dev\/2017-May\/thread.html#117370\">http:\/\/lists.openstack.org\/pipermail\/openstack-dev\/2017-May\/thread.html#11737<span class=\"s2\">0<\/span><\/a><\/span><\/p>\n<p class=\"p4\"><span class=\"s1\">[8] &#8211; <a href=\"https:\/\/review.openstack.org\/447524\">https:\/\/review.openstack.org\/44752<span class=\"s2\">4<\/span><\/a><\/span><\/p>\n<p class=\"p4\"><span class=\"s1\">[9] &#8211; <a href=\"http:\/\/lists.openstack.org\/pipermail\/openstack-dev\/2017-May\/thread.html#116677\">http:\/\/lists.openstack.org\/pipermail\/openstack-dev\/2017-May\/thread.html#11667<span class=\"s2\">7<\/span><\/a><\/span><\/p>\n<p class=\"p4\"><span class=\"s1\">[10] &#8211; <a href=\"https:\/\/review.openstack.org\/#\/c\/464746\/\">https:\/\/review.openstack.org\/#\/c\/464746<span class=\"s2\">\/<\/span><\/a><\/span><\/p>\n<p class=\"p4\"><span class=\"s1\">[11] &#8211; <a href=\"http:\/\/lists.openstack.org\/pipermail\/openstack-dev\/2017-May\/thread.html#116619\">http:\/\/lists.openstack.org\/pipermail\/openstack-dev\/2017-May\/thread.html#11661<span class=\"s2\">9<\/span><\/a><\/span><\/p>\n<p class=\"p4\"><span class=\"s1\">[12] &#8211; <a href=\"http:\/\/lists.openstack.org\/pipermail\/openstack-dev\/2017-May\/thread.html#116736\">http:\/\/lists.openstack.org\/pipermail\/openstack-dev\/2017-May\/thread.html#11673<span class=\"s2\">6<\/span><\/a><\/span><\/p>\n<p class=\"p4\"><span class=\"s1\">[13] &#8211; <a href=\"https:\/\/review.openstack.org\/#\/c\/427880\/\">https:\/\/review.openstack.org\/#\/c\/427880<span class=\"s2\">\/<\/span><\/a><\/span><\/p>\n<p class=\"p4\"><span class=\"s1\">[14] &#8211; <a href=\"http:\/\/lists.openstack.org\/pipermail\/openstack-dev\/2017-May\/thread.html#116642\">http:\/\/lists.openstack.org\/pipermail\/openstack-dev\/2017-May\/thread.html#11664<span class=\"s2\">2<\/span><\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>SuccessBot Says clarkb 1 : infra added city cloud to the pool of test nodes. pabelanger 2 : opensuse-422-infracloud-chocolate-8977043 launched by nodepool. All: 3 etcd 3.x as a Base Service A devstack review 4 that adds a new etcd3 service. Two options to enable the DLM use case with Tooz (for eventless based services) 5&#8230;  <a href=\"https:\/\/www.openstack.org\/blog\/openstack-developer-mailing-list-digest-20170526\/\" class=\"more-link\" title=\"Read OpenStack Developer Mailing List Digest May 20-26\">Read more &raquo;<\/a><\/p>\n","protected":false},"author":82,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[],"_links":{"self":[{"href":"https:\/\/www.openstack.org\/blog\/wp-json\/wp\/v2\/posts\/8098"}],"collection":[{"href":"https:\/\/www.openstack.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.openstack.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.openstack.org\/blog\/wp-json\/wp\/v2\/users\/82"}],"replies":[{"embeddable":true,"href":"https:\/\/www.openstack.org\/blog\/wp-json\/wp\/v2\/comments?post=8098"}],"version-history":[{"count":1,"href":"https:\/\/www.openstack.org\/blog\/wp-json\/wp\/v2\/posts\/8098\/revisions"}],"predecessor-version":[{"id":8099,"href":"https:\/\/www.openstack.org\/blog\/wp-json\/wp\/v2\/posts\/8098\/revisions\/8099"}],"wp:attachment":[{"href":"https:\/\/www.openstack.org\/blog\/wp-json\/wp\/v2\/media?parent=8098"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.openstack.org\/blog\/wp-json\/wp\/v2\/categories?post=8098"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.openstack.org\/blog\/wp-json\/wp\/v2\/tags?post=8098"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}