Barcelona, Spain
October 25-28, 2016

Event Details


The Race Conditions of Neutron L3 HA's Scheduler Under Scale Performace

Neutron's Router presents a L3 interface that connects VMs to an external network. In its legacy mode, a router can be scheduled only to one agent at a time, presenting a single point of failure - if that node fails, all communications are halted. L3 HA mitigates this issue by presenting an Active/Passive configuration where a router can be scheduled to multiple agents which recover the datapath automatically during failures using a dedicated network that hosts keepalived processes.

Using L3 HA presents an orchestration problem: since a router now appears in more than one agent and another dedicated network is required for the keepalived processes, there are a lot more resources involved. We performed L3 HA scale testing for Liberty and Mitaka, which showed benefits of L3 HA and uncovered a number of race conditions in Neutron code both on L3 agent and on server side. We'll present results of this scale testing and showcase some of these race conditions.


What can I expect to learn?

Attendees could expect to learn:

  1. Basics of router rescheduling mechanisms and L3 HA specifics.
  2. Known issues with routing and what have been done to fix them.
  3. Performance data for L3 HA for Liberty and Mitaka.
  4. Details of how ALLOCATING state works and fixes race conditions.
  5. What is planned for next cycles in this area.
Wednesday, October 26, 11:25am-12:05pm
Difficulty Level: Intermediate
RedHat
Working for Red Hat and on Neutron for the past two years, soldier at the IDF beforhand. Coding enthusiast, review fanatic and speaking hobbiest. FULL PROFILE
Mirantis
Ann is a Software engineer at Mirantis IT, Neutron core-reviewer with a focus on databases, contributing to Neutron since Havana. FULL PROFILE
Mirantis
Kevin Benton is a software engineer at Mirantis and the Neutron Pike PTL. He has been contributing to and supporting deployments of Neutron since 2013. His background is in security and networking. FULL PROFILE