One of the challenges that comes from deploying multi-tiered distributed systems, or microservices, atop a dynamic scheduler is the introduction of new problems surrounding load balancing. There are some inherent challenges in building a load balancer that's meant to operate in a highly available way, without any single points of failure. In this talk, Sargun Dhillon will walk through the distributed load balancing mechanism that he built for Mesos. This service discovery mechanism is meant to have the same kinds of features, api, and availability that existed in legacy, statically partitioned environments. The purpose of this is to ease the transition, and remove some of the largest road blocks in moving applications over to modern datacenters. In addition, he will speak to why he built it as opposed to other alternatives for service discovery and load balancing such as using Zookeeper, and the challenges that came from it. We built a library called Lashup that has a membership protocol, a multicast layer, failure detector, and CRDT key/value store. This has allowed us to build applications that orchestrate Mesos clusters with great ease.
