One of the central components of the open source Gestalt project is the lambda engine known as Gestalt Laser. Laser, short for Lambda Application Server, was built on top of mesos. In the development of laser we encountered a number of issues dealing with both mesos performance and latency which motivated novel engineering solutions. We also had to solve issues around scaling and porting the laser engine across multiple clusters and this required the creation of a number of components for federated clusters and network communication enhancements including libraries for working with the Mesos HTTP API.
The areas of our talk will include: What are the core components of a lambda service? * API Gateways * Message Queues * Runtime Engines Overview of the Gestalt Laser solution. * Discussion of the engineering issues encountered. * Mesos Latency * Caching and Routing Optimizations * Learnings and Libraries for working with mesos http api. * Language Runtime Packaging and Optimizations * State of the current open source project, and what people can do to help!
My name is Brad Futch. I work for Galactic Fog and we're focused on helping enterprises embrace cloud native architectures and development practices. I've had experience in many fields related to application development, and have been working on cloud native architectures for 5 years... Read More →
For ArangoDB 3.0 we have implemented the Raft consensus protocol in ArangoDB, thereby creating a high-performance resilient K/V store “Agency” based on Raft’s replicated log. This makes it possible to use ArangoDB instead of Zookeeper, Consul or etcd to create resilient distributed applications. Mesos and thus DC/OS are the ideal ecosystem to develop and run distributed applications, and all such applications need fault tolerant management and synchronization between the various tasks. Features and their benefits:
- transactions involving multiple keys: simpler application logic - complex redis-like data manipulations: less roundtrips, higher performance - powerful preconditions for write transactions: less roundtrips, higher performance - JSON-based HTTP/REST API: accessible from nearly all languages - subscription to HTTP callbacks: fast reaction on changes - user-defined supervision tasks JavaScript running in leader: easier implementation of resilience logic
As known from ArangoDB, deployment on Mesos can be done with a single click or command. By the beginning of the conference, we will have tortured ArangoDB Agency with Jepsen tests and I will announce the results in the talk, whatever they are.
Max Neunhöffer is a mathematician turned database developer. In his academic career he has worked for 16 years on the development and implementation of new algorithms in computer algebra. During this time he has juggled a lot with mathematical big data like group orbits containing... Read More →
For an enterprise solution, you need to support multi-tenancy with pool allocation guarantees. The infrastructure needs to scale up and tear down with the requirements of the users. Users may be using many tools/technologies/frameworks like spark, HDFS or non-distributed jobs or complex workflows.
Mandeep Gandhi has been working for Adobe Systems for quite some time now where he has worked upon on many products and cloud platform initiative. Last year, he presented a talk in Mesos Con "Handling Multi-tenancy & Scaling with Apache Mesos". Currently, he is working on a very interesting... Read More →
With the Mesos persistent volumes primitives developers can get started on creating frameworks to bring storage systems such as Cassandra onto Mesos. However in working with the framework developers we found that more features need to be put in place to replace existing toolings that run these systems in production.
In this talk we'll cover the new features that we are working on to address these needs.
- Sharing the same volume among multiple frameworks simultaneously. - Allowing frameworks to set filesystem permissions explicitly. - Security and performance based isolation around volumes.
We'll also talk about the continuing effort to make persistent volumes workflow more reliable and usable.
Anindya Sinha is a senior software engineer on the Infrastructure Team at Apple, where he works on the Apache Mesos project. Prior to Apple, he worked for Cisco on a realtime video delivery platform.
As a mobile gaming company, we need to cater for unpredictable spikes in users, and scale each part of our application individually. We also need to support fast development iterations, which might need a new short-lived environment.
This talk will highlight how we go from a blank canvas AWS account, to a fully functional PaaS, without making any compromises on the infrastructure. We will talk about High Availability across AWS zones, to ensure Mesos is always available to meet the harsh demands of our environment.
This talk will show how we design our infrastructure to keep Mesos running all the time, how we can launch a new cluster in 5 minutes, and how we deploy our environment on top of it, using Infrastructure-as-code tools like Terraform and Packer.
Build engineer interested in automating workflows, autonomous systems, creating a culture of awesome, and building infrastructure and tools that meet the needs of the company.
Emmanuel is a DevOps at DIGIT games Studio, he is the one building and automating the whole infrastructure at DIGIT and make sure everything is working autonomously . He as been interested by Mesos since couple year but only start using it effectively last year.
Technology organizations are complex environments that bind deeply technical tools and infrastructure with strong cultural threads. Software like Mesos and Docker offer compelling stories around scalability, fault tolerance, and deterministic environments. Nonetheless, many challenges emerge as one attempts to weave these components into an existing engineering culture.
A successful transition to a service-oriented architecture has left Groupon with numerous services deployed to a variety of hardware, physical and virtual, via a diverse set of tools. Along the way the organization has picked up an array of operational readiness requirements and deploy controls. All of these help to maintain stability and performance but come at a cost: ease of deployment.
Conveyor aims to maintain and improve Groupon’s high bar for production software quality, while streamlining the process of getting code into production. Powerful tools like Apache Mesos, Marathon, and Docker form the foundation of this approach. This talk covers how Conveyor uses these tools together and the various technical and cultural hurdles encountered in the process of rolling out this infrastructure within the enterprise.
Tristan Blease is a principal software engineer at Groupon. He focuses on leading teams that are building tooling and infrastructure to ensure the success of our cloud migration.
Shipping quality software always involves in-depth testing. In order to minimize time spent on repetitive actions, this task should be fully automated. Unfortunately this creates many problems which have to be solved to ensure stability of the whole process. During this talk Paweł will discuss key problems faced while building automatic testing infrastructure for Tizen operating system images. He will also present how Tizen release team dealt with such an uneasy task. Both hardware and software presented in this talk is open (https://git.tizen.org/cgit/tools/testlab/sd-mux.git and https://git.tizen.org/cgit/tools/testlab/major.git) and can be easily used to build embedded software testing lab.
Paweł Wieczorek is a consultant software engineer in the Collabora's Core team. He works mainly in testing automation area with strong focus on system security and performance. Speaker on ELC and ELCE, FOSDEM, FrOSCon and a few others.
Apple Engineers give an overview of the Mesos HTTP API client that they developed to interact with Mesos. It is built to scale and leverages akka-streams with full support for back pressure. The talk covers the architecture and different stages of the client, challenges faced developing it and concrete usage examples.
