MesosCon EU 2016

Micro service architectures result in up to 20 times larger environments than their monolithic counterparts. In such big and interconnected environments container metrics will tell you about infrastructure health but not service health. Even if you have implemented service health checks to quickly react on service failures, in a resilient system (like built on top of Mesos/Marathon or DC/OS) you will see intermediary mushroom cloud effects of a large number of services being affected temporarily. The mushroom cloud shows you all services, containers and hosts being affected by a failing component. How do you find out what really caused the problem and how to distinguish effect vs. cause?

In this session Alois will do post-mortem analysis by walking through different cases of failures we've observed in a real-world large e-commerce production environment running on Apache Mesos and show you how to figure out what actually caused the failures.

A CI/CD pipeline running on Mesos needs a dedicated component to store, serve the artefacts and their metadata. It is now possible to run JFrog's Artifactory, a universal binary repository manager, in a highly available configuration on Apache Mesos that can support many hundreds of users. Once usage of Artifactory scales beyond the demands of a single node, a highly available configuration uses a primary/secondary architecture to scale to several nodes. These nodes also require access to a relational database and shared filesystem.

In this presentation, Alexis Tual, a Solution Engineer at JFrog, will review the challenges faced when adapting the existing highly available architecture of JFrog to the world of Mesos: from storage to scheduling. He will also show how you can use Artifactory to push and pull Docker images with Marathon to create CI/CD pipelines for containerized projects. He will also discuss future work for both Mesos and Artifactory that will improve this integration.

Mesos is never the only OSS you need to run your production datacenter. And just like all of us hanging out together at MesosCon, if you surround Mesos with its OSS friends, you get a happy, highly productive Mesos. But you have to be careful, not all OSS plays well with others.

In this talk, we’ll start by looking at a handful of production Mesos datacenters from major users. We’ll use these real-world examples to abstract a standard Mesos datacenter architecture that contains all of the components needed to run today’s modern, containerized apps with big data and analytics frameworks.

With that harmonized Mesos datacenter architecture, we’ll look at each of the abstract components and discuss the leading OSS projects that fit each piece. We’ll discuss the characteristics of what makes some solutions work well with Mesos, and call out the projects that don’t meet the standard.

We’ll wrap up the talk by showing the complete architecture diagram, and show the single- command trick for bringing all of the best OSS components together and getting them up and running in a production cluster.

Process migration (also known as process checkpointing) is the ability to move a group of related running processes from one set of nodes to another. It involves three phases: take a snapshot of all processes, copy the snapshot data over to the target nodes, and restart processes from that snapshot. Process migration has traditionally been used for fault-tolerance in the context of long-running stateful applications. Without it, the application developers need to modify the stateful application to periodically save the state to disk in order to restart in case of a failure. This is inefficient and error-prone!

In this talk, we demonstrate process migration within a Mesos cluster for a more enjoyable way to schedule your containers. Apart from fault-tolerance, process migration within Mesos opens up new avenues for implementing better scheduling policies. Some of the other use cases include improved maintenance primitives, debugging, speculative execution and managing “tails” for multi-threaded applications.

Walk through Marathon’s architecture.
Describe how to secure the communication with Mesos
Describe the authentication/authorization plugin interface
Show an example plugin

DC/OS is a powerful platform to run containers and resilient microservices architectures at scale. But releasing or upgrading software to production often is a stressful moment due to the risk of performance issues or even downtime. Applying canary-patterns to ContinuousDelivery pipelines provides a safety-net which makes releasing containers less risky and stressful. By publishing new software versions to only a small percentage of visitors with specific criteria, it enables you to test, optimise and scale in a controlled and gradual way, without negatively impacting the majority of users. In this presentation we’re going to talk about how VAMP adds powerful opensource canary-releasing features to the DC/OS stack, and how to setup a smart ContinuousDelivery pipeline.