MesosCon EU 2016: Full Schedule

Register Now or Learn More

8:00am PDT

Registration

Wednesday August 31, 2016 8:00am - 9:00am PDT
Hotel Lobby

Breaks

9:00am PDT

Keynote: Welcome Address - Benjamin Hindman, Founder and Chief Architect, Mesosphere

Speakers

Ben Hindman

Mesosphere Founder - Apache Mesos Co-Creator, Mesosphere

Ben is one of the creators of Apache Mesos, a platform for building and running resource-efficient distributed systems at scale. Ben started working on Mesos as a PhD student at Berkeley before he brought it to Twitter where it runs on thousands of machines. An academic at heart... Read More →

benh mesos mesoscon eu 08 31 16 pdf

Wednesday August 31, 2016 9:00am - 9:10am PDT
Ballroom ABC

Keynotes

9:10am PDT

Keynote: Breaking Barriers: Creatively and Courageously - Jelena Lucin, International Project Manager, Greenlight for Girls

In a society often diffused with preconceptions and norms, Jelena Lucin, International Project Manager & Science Advisor for greenlight for girls, will tell the story of how an initiative can grow from one email to a global organization in a few short years by breaking barriers, creatively and courageously.

Greenlight for girls, asbl is an international organisation dedicated to inspiring girls of all ages and backgrounds by demonstrating just how fun & interactive the world of Science and Technology can be. With hands-on workshops and events led by role-models from different industries around the globe, we inspire and build the future talent pipeline of STEM leaders by encouraging young students to think about how their futures are full of possibilities and to realise their own individual and creative potential.

Speakers

Jelena Lucin

International Project Manager & Science Adviser, Greenlight for Girls

As a scientist, educator and global Project Manager, her multi-faceted role with greenlight for girls, an organization that instills passion and creativity, takes on many dimensions - from coordinating events, leading project teams around the world to building relationships with major... Read More →

Wednesday August 31, 2016 9:10am - 9:30am PDT
Ballroom ABC

Keynotes

9:35am PDT

Keynote: Elastic Resource Scheduling with Apache Mesos - Sharma Podila, Sr. Software Engineer, Netflix

Netflix customers worldwide streamed over Forty Two Billion hours of content last year. Service style applications, batch jobs, and stream processing alike, from a variety of use cases across Netflix rely on executing container based applications in multi-tenant clusters powered by Mesos and Fenzo. This talk will highlight the motivations for using Apache Mesos and focus on elastic resource scheduling to handle heterogeneous mix of tasks and resources in an autoscaling cluster. Based on experiences from Netflix projects Mantis and Titus, the talk will additionally discuss thoughts on capacity guarantees (SLAs) for various workloads and insights from integrating Docker execution in AWS EC2.

Speakers

Sharma Podila

Senior Software Engineer, Netflix

Sharma Podila is a Senior Software Enginner in the Edge Engineering team at Netflix, Inc. His current work includes Fenzo, an open source generic scheduler with plugin based optimizations, developing cloud native Mesos frameworks, and evolving the microservices platform at Netfli... Read More →

Podila MesosCon Europe Keynote Aug Sep 2016 pdf

Wednesday August 31, 2016 9:35am - 10:00am PDT
Ballroom ABC

Keynotes

10:00am PDT

Coffee Break and Sponsor Showcase

Wednesday August 31, 2016 10:00am - 10:30am PDT
Diamond + Staten

Breaks

10:30am PDT

Yammer Datacenter Story - OnPremise, GFS, Azure - Tobias Haag & Kyle Gordon, Microsoft

Abstract:

All things Docker seem to be the defining slogan for Yammer these days. And what other way to run Docker at scale than Mesos?
Yammer wrote an in-house datacenter automation platform which was fraught with problems and restrictions the Mesos landscape promised to solve, and in 2015 engineers at Yammer started exploring it as a possible replacement.
This talk will focus on two core points of the implementation run at Yammer: Docker & Security

The big gotcha: Docker. Everything has to run inside Docker. From Zookeeper to the Mesos agents and even the executors. Dedication, mixed with some despair, eventually resulted in the platform for production today.

Security is the second biggest piece to honor the commitment made to Yammer customers as part of the Office365 suite. The path to provide a good solution to security aspects like compartmentalization, trust bootstrap, access control and transport encryption proved a balancing act between customer commitment and developer friendliness.

In the end Mesos helped us deliver Docker's promise of Build, Ship, Run as the foundation of environments Yammer can trust to run from Dev all the way to Production at scale and anywhere on the planet.

Speakers

Kyle Gordon

Software Engineer 2, Yammer / Microsoft

I have been an engineer on the operations team for Yammer for almost 3 years. Currently I am in the process of creating a stable environment to migrate Yammer to Azure. We are primarily a micro service company and are an ideal fit to live on a mesos architectured environment. We’ve... Read More →

Tobias Haag

Senior Software Engineer, Microsoft

4 years at Yammer from Junior to Senior Software Engineer at Yammer / Microsoft. Since late 2013 part of the Infrastructure team with focus on Datacenter automation and software lifecycle support. As of mid 2015 leading developer on Azure migration effort with focus on Docker and... Read More →

MesosCon 2016 Yammer pptx

Wednesday August 31, 2016 10:30am - 11:20am PDT
Ballroom B

Developer, Beginner

10:30am PDT

Mesos 1.0 - Vinod Kone & Anand Mazumdar, Mesosphere

Mesos will soon reach the 1.0 milestone. In addition to new features, this would mean a more stable user-facing API and stricter support/release guarantees for operators/framework developers. The aim of this talk is to apprise the operators/framework developers/users about the new API and also discuss the support/compatibility guarantees offered by Mesos going forward.

This talk is a sequel to the last year’s MesosCon Seattle talk on “Mesos HTTP API” and continues from where it left off.

This talk will cover the following specific topics:
- Discuss the newly introduced Operator APl.
- Update on the recent improvements to the Framework API.
- Update on client libraries for the new Framework API.
- Release cadence for Mesos going forward.
- Support/Compatibility guarantees for operators/framework developers e.g., backporting of patches etc.
- Master->Agent renaming in the 1.0 API.

Speakers

Vinod Kone

Apache Mesos PMC, Mesosphere

Vinod Kone is a committer and PMC member of the Apache Mesos project. He is currently a Tech Lead and Engineering Manager @ Mesosphere. Previously, he was a Tech Lead and Manager of the Mesos team @Twitter. Vinod completed his PhD in Computer Science from UC Santa Barbara.

Anand Mazumdar

Distributed System Engineer, Mesosphere

Anand Mazumdar is a software engineer at Mesosphere where he works on the Apache Mesos project. Prior to that, he used to work at a Quantitative Hedge Fund and Amazon Web Services on scalable data stream processing. He holds a Masters in Computer Science from The University of Texas... Read More →

Mesos 1.0 new pdf

Wednesday August 31, 2016 10:30am - 11:20am PDT
Ballroom C

Operator, Intermediate

10:30am PDT

Lamba Application Servers on Mesos - Brad Futch, Galactic Fog

One of the central components of the open source Gestalt project is the lambda engine known as Gestalt Laser. Laser, short for Lambda Application Server, was built on top of mesos. In the development of laser we encountered a number of issues dealing with both mesos performance and latency which motivated novel engineering solutions. We also had to solve issues around scaling and porting the laser engine across multiple clusters and this required the creation of a number of components for federated clusters and network communication enhancements including libraries for working with the Mesos HTTP API.

The areas of our talk will include:
What are the core components of a lambda service?
* API Gateways
* Message Queues
* Runtime Engines
Overview of the Gestalt Laser solution.
* Discussion of the engineering issues encountered.
* Mesos Latency
* Caching and Routing Optimizations
* Learnings and Libraries for working with mesos http api.
* Language Runtime Packaging and Optimizations
* State of the current open source project, and what people can do to help!

Speakers

Brad Futch

Principal Engineer, Galactic Fog

My name is Brad Futch. I work for Galactic Fog and we're focused on helping enterprises embrace cloud native architectures and development practices. I've had experience in many fields related to application development, and have been working on cloud native architectures for 5 years... Read More →

Lamba Application Servers on Mesos with Notes pdf

Lamba Application Servers on Mesos pdf

Wednesday August 31, 2016 10:30am - 11:20am PDT
Ballroom A

User

11:30am PDT

Marathon and Chronos - State of the Art - Matthias Eichstedt & Matthias Veit, Mesosphere

Marathon and Chronos are the core and essential frameworks for most Apache Mesos clusters.
Marathon ensures long running tasks are running in the datacenter and is commonly used to maintain high availability of other frameworks. Chronos is the fault tolerant cron of the datacenter.
There have been significant advances in both of these frameworks to leverage the latest features of Apache Mesos including persistent storage in addition to extending the capabilities of the system such as networking and security.
This session will provide an overview of all the new features along with experience from the field and best practices for using these features in production. The session will conclude with a look at the roadmap for each of these projects providing a glimpse into what to expect over the next year.

Speakers

Matthias Eichstedt

Engineering Manager, Mesosphere

I'm Engineering Manager of the Orchestration team at Mesosphere, based in Hamburg, Germany.

Matthias Veit

Mesosphere, Engineering Manager

Marathon+Metronome pdf

Wednesday August 31, 2016 11:30am - 12:20pm PDT
Ballroom B

Developer, Intermediate

11:30am PDT

The Evolution of Deploy Tooling at Twitter - David McLaughlin, Twitter

Apache Mesos and Apache Aurora have been crucial in growing a microservices culture at Twitter. The initial phase of adoption at Aurora at Twitter saw many teams abandon their own deploy tooling and frameworks in favor of concise Aurora configuration files.

But over time a whole new class of bespoke deploy tooling has emerged as service owners had to target large deployment matrices consisting of multiple pre-production environments (for things like performance testing, integration testing, regression testing, canary testing, etc.) across multiple availability zones (i.e. multiple Mesos clusters). As the matrix grows, manual orchestration quickly becomes untenable.

The custom tooling also had a strong emphasis on supporting rolling back services to a previous good state, which only becomes more complicated as your deployment matrix grows.

In this talk I’ll outline the ways in which Mesos and Aurora helped our engineers to implement more sophisticated DevOps processes by making it easy to grow their deployment matrix. I’ll outline how that process surfaced some of the holes in currently available tooling and how it led to a huge amount of duplicate effort for our service owners. Finally I’ll describe a system we’ve built at Twitter to support our CI/CD pipeline that we feel closes those gaps.

Speakers

David McLaughlin

Wednesday August 31, 2016 11:30am - 12:20pm PDT
Ballroom C

Operator, Intermediate

11:30am PDT

ArangoDB Agency: A New Resilient, Transactional K/V Store - Max Neunhöffer, ArangoDB

For ArangoDB 3.0 we have implemented the Raft consensus protocol in ArangoDB, thereby creating a high-performance resilient K/V store “Agency” based on Raft’s replicated log. This makes it possible to use ArangoDB instead of Zookeeper, Consul or etcd to create resilient distributed applications. Mesos and thus DC/OS are the ideal ecosystem to develop and run distributed applications, and all such applications need fault tolerant management and synchronization between the various tasks.
Features and their benefits:

- transactions involving multiple keys: simpler application logic
- complex redis-like data manipulations: less roundtrips, higher performance
- powerful preconditions for write transactions: less roundtrips, higher performance
- JSON-based HTTP/REST API: accessible from nearly all languages
- subscription to HTTP callbacks: fast reaction on changes
- user-defined supervision tasks JavaScript running in leader: easier implementation of resilience logic

As known from ArangoDB, deployment on Mesos can be done with a single click or command. By the beginning of the conference, we will have tortured ArangoDB Agency with Jepsen tests and I will announce the results in the talk, whatever they are.

Speakers

Max Neunhöffer

Database Developer, ArangoDB

Max Neunhöffer is a mathematician turned database developer. In his academic career he has worked for 16 years on the development and implementation of new algorithms in computer algebra. During this time he has juggled a lot with mathematical big data like group orbits containing... Read More →

ArangoDBAgency pdf

Wednesday August 31, 2016 11:30am - 12:20pm PDT
Ballroom A

User, Beginner

12:20pm PDT

Lunch and Sponsor Showcase

Wednesday August 31, 2016 12:20pm - 2:00pm PDT
Diamond + Staten

Breaks

2:00pm PDT

Fault Tolerance in Mesos - Vinod Kone & Anand Mazumdar, Mesosphere

Mesos is a widely used cluster resource manager that has been used in production at scale for some time. At its core, Mesos is a sophisticated distributed system with lots of components and failure scenarios. In datacenters, hardware/network failures are the norm rather than the exception. Being a distributed systems kernel, Mesos needs to be resilient to all of these failures, degrading gracefully wherever applicable. We believe that everyone running a Mesos cluster in production/building distributed systems would benefit from learning how Mesos does fault tolerance to reason better about failures in their cluster/adopt these battle-hardened best practices in their project.

This talk focuses on how various Mesos components like Master, Agent, Scheduler, Executor react to common failures like node failures/process crash, network partitions, Zookeeper failures etc.
This talk will cover the following specific topics:
- How Mesos masters/agents react to node failures and network partitions.
- Agent recovery after a process restart.
- Master health checks
- Executor health checks.
- Scheduler Disconnections/Partitions.
- Zookeeper Failures.

Speakers

Vinod Kone

Apache Mesos PMC, Mesosphere

Anand Mazumdar

Distributed System Engineer, Mesosphere

Fault Tolerance In Mesos pdf

Wednesday August 31, 2016 2:00pm - 2:50pm PDT
Ballroom B

Developer, Beginner

2:00pm PDT

The Mushroom Cloud Effect or What Happens When Containers Fail? - Alois Mayr, Dynatrace

Micro service architectures result in up to 20 times larger environments than their monolithic counterparts. In such big and interconnected environments container metrics will tell you about infrastructure health but not service health. Even if you have implemented service health checks to quickly react on service failures, in a resilient system (like built on top of Mesos/Marathon or DC/OS) you will see intermediary mushroom cloud effects of a large number of services being affected temporarily. The mushroom cloud shows you all services, containers and hosts being affected by a failing component. How do you find out what really caused the problem and how to distinguish effect vs. cause?

In this session Alois will do post-mortem analysis by walking through different cases of failures we've observed in a real-world large e-commerce production environment running on Apache Mesos and show you how to figure out what actually caused the failures.

Speakers

Alois Mayr

Technology Lead Cloud & Containers, Dynatrace

Alois is technology lead for Cloud Foundry at Dynatrace and working on bringing full-stack monitoring to cloud native platforms.

20160831 MesosCon WhenContainersFail pdf

Wednesday August 31, 2016 2:00pm - 2:50pm PDT
Ballroom C

Operator, Beginner

2:00pm PDT

Lessons Learned from Running Serverless Workloads on Mesos - Diana Arroyo & Alek Slominsk, IBM

Serverless computing platforms promise new capabilities that make writing scalable micro-services easier and more cost effective. These platforms provide a distributed compute service to execute application logic in response to events. Workload demands of serverless platforms can require thousands of concurrent short lived containers to be created and destroyed in milli-seconds.

In this talk we will share the lessons learned when running these workloads in a Mesos environment to meet the performance demands of OpenWhisk, a serverless open-source computing platform. We will present our experience running workload experiments in Mesos, share Mesos tuning tips and workload generation code to make Mesos an ideal platform for serverless workloads.

Speakers

Diana Arroyo

Research Senior Software Engineer, IBM

Diana is a Senior Software Engineer at IBM T. J. Watson Research Center focusing on developing cutting-edge resource management solutions for a large set of system platforms ranging from web server applications to virtual machines orchestration and now cloud container platforms. She... Read More →

Alek Slominski

Research Staff Member, IBM

Aleksander Slominski is Research Staff Member in the Serverless Group at IBM Research, a member of Knative Eventing. He has worked on business workflows and eventing systems since 2000. He is interested in applying emerging technologies to build next-generation of business applications... Read More →

lessons learned serverless v2f pdf

Wednesday August 31, 2016 2:00pm - 2:50pm PDT
Ballroom A

Wildcard, Intermediate

3:00pm PDT

JEE on DC/OS 101 and Fun - Josef Adersberger, Qaware

Cloud native applications are popular these days – applications that run in the cloud reliably und scale almost arbitrarily. They follow three key principles: They are built and composed as microservices, they are packaged and distributed in containers and the containers are executed dynamically in the cloud. In this hands-on session we will show how to build, package and deploy cloud native Java EE applications on top of DC/OS - fully automated with Gradle. And for the fun of it we will be using an off-the-shelf DJ pad, programmed with nothing else than the Java Sound API, to demonstrate the core concepts and to visualize and remote control DC/OS.

Speakers

Josef Adersberger

CTO, QAware

Josef Adersberger is co-founder & CTO of QAware, a German custom software development company and CNCF silver member. He studied computer science in Rosenheim and Munich and holds a doctoral degree in software engineering. He is currently responsible for a large-scale cloud migration... Read More →

adersberger mesoscon jee on dcos pdf

Wednesday August 31, 2016 3:00pm - 3:50pm PDT
Ballroom B

Developer, Beginner

3:00pm PDT

Flexibility Across the Cloud - Managing and Scaling your High Availability DC/OS Cluster using Juju - Thomas Barber, Spicule Ltd

DC/OS, the recently open sourced Mesos based operating system allows system administrators and devops departments to run entire data centers as a single compute unit. But what about managing your servers and scaling your infrastructure?

With the advent of cloud computing and vastly reduced infrastructure costs, the compute resource available to businesses is virtually limitless and Juju, created by Canonical, allows us to manage our applications flexibility and across bothcloud and physical infrastructure as if it were the same thing. In this presentation Tom Barber will take you into the new world of application modelling.

Already used to drive many Open Stack deployments we will take a look at how Juju can help you model your infrastructure in a way that vastly simplifies managing your DC/OS or Mesos installation and networking, whilst avoiding single cloud lock in or managing various services across different vendors and APIs.

We'll also delve into testing failure, enterprise monitoring and logging of your cloud agnostic DC/OS stack.

Speakers

Tom Barber

Technical Director, Spicule LTD

Tom Barber is the director of Meteorite BI and Spicule BI. A member of the Apache Software Foundation and regular speaker at ApacheCon, Tom has a passion for simplifying technology. The creator of Saiku Analytics and open source stalwart, when not working for NASA, Tom currently deals... Read More →

Wednesday August 31, 2016 3:00pm - 3:50pm PDT
Ballroom C

Operator, Intermediate

3:00pm PDT

Handling Multi-tenancy & Scaling with Apache Mesos - Mandeep Gandhi, Adobe Systems

For an enterprise solution, you need to support multi-tenancy with pool allocation guarantees. The infrastructure needs to scale up and tear down with the requirements of the users. Users may be using many tools/technologies/frameworks like spark, HDFS or non-distributed jobs or complex workflows.

Speakers

Mandeep Gandhi

Computer Scientist, Adobe

Mandeep Gandhi has been working for Adobe Systems for quite some time now where he has worked upon on many products and cloud platform initiative. Last year, he presented a talk in Mesos Con "Handling Multi-tenancy & Scaling with Apache Mesos". Currently, he is working on a very interesting... Read More →

compute final mesosConEur2016 pdf

Wednesday August 31, 2016 3:00pm - 3:50pm PDT
Ballroom A

User, Intermediate

3:50pm PDT

Coffee Break and Sponsor Showcase

Wednesday August 31, 2016 3:50pm - 4:20pm PDT
Diamond + Staten

Breaks

4:20pm PDT

DC/OS: Considerations and Best Practices for Production Deployments - Ben Lin, Mesosphere

DC/OS makes it easy to quickly setup and run a Mesos cluster but the default configuration may not be as battle-hardened as one would like or uniquely tailored to meet your requirements and constraints. In this session, Mesosphere consultants will discuss key customer requirements and the architecture decisions that lead to a production ready, resilient, and scalable DC/OS design. This session will cover Capacity Planning, Monitoring, Physical Host Considerations, Security, Container Data Management, and Backup and Recovery. The session will also cover the unique elements that need to be considered when deploying DC/OS in Production vs. Dev/Test configurations.

Speakers

Ben Lin

Solutions Architect, Mesosphere

Ben Lin is a Solutions Architect at Mesosphere. Ben works closely with customers to develop strategies for architecting, deploying, and operationalizing DC/OS environments. Previously he worked in the VMware NSBU (Networking & Security Business Unit), focusing on advanced solutions... Read More →

MesosConEU16 DCOS Deployment Considerations pdf

Wednesday August 31, 2016 4:20pm - 5:10pm PDT
Ballroom B

Developer, Intermediate

4:20pm PDT

Bringing JFrog Artifactory to Apache Mesos - Alexis Tual, Jfrog

A CI/CD pipeline running on Mesos needs a dedicated component to store, serve the artefacts and their metadata. It is now possible to run JFrog's Artifactory, a universal binary repository manager, in a highly available configuration on Apache Mesos that can support many hundreds of users. Once usage of Artifactory scales beyond the demands of a single node, a highly available configuration uses a primary/secondary architecture to scale to several nodes. These nodes also require access to a relational database and shared filesystem.

In this presentation, Alexis Tual, a Solution Engineer at JFrog, will review the challenges faced when adapting the existing highly available architecture of JFrog to the world of Mesos: from storage to scheduling. He will also show how you can use Artifactory to push and pull Docker images with Marathon to create CI/CD pipelines for containerized projects. He will also discuss future work for both Mesos and Artifactory that will improve this integration.

Speakers

Alexis Tual

Solution Engineer, JFrog

Alexis is a versatile Solution Engineer working at JFrog in Toulouse (France). He has a strong Java (Groovy !) / Web Developer background and in the past few years dedicated his time to automation and CI/CD.

Wednesday August 31, 2016 4:20pm - 5:10pm PDT
Ballroom C

Operator, Beginner

4:20pm PDT

Practical Persistent Volumes: New Features for Supporting Storage Frameworks - Anindya Sinha, Apple

With the Mesos persistent volumes primitives developers can get started on creating frameworks to bring storage systems such as Cassandra onto Mesos. However in working with the framework developers we found that more features need to be put in place to replace existing toolings that run these systems in production.

In this talk we'll cover the new features that we are working on to address these needs.

- Sharing the same volume among multiple frameworks simultaneously.
- Allowing frameworks to set filesystem permissions explicitly.
- Security and performance based isolation around volumes.

We'll also talk about the continuing effort to make persistent volumes workflow more reliable and usable.

Speakers

Anindya Sinha

Senior Software Engineer, Apple, Inc.

Anindya Sinha is a senior software engineer on the Infrastructure Team at Apple, where he works on the Apache Mesos project. Prior to Apple, he worked for Cisco on a realtime video delivery platform.

MesosConEurope2016PPVv1.0 pdf

Wednesday August 31, 2016 4:20pm - 5:10pm PDT
Ballroom A

User, Intermediate

5:20pm PDT

Building Highly Available Mesos Frameworks, 2.0 - Neil Conway, Mesosphere

Production-quality Mesos frameworks must be able to continue managing tasks despite unreliable networks and faulty computers. Mesos provides tools to help developers do fault-tolerant task management, but putting these tools together effectively remains something of a black art. This talk will offer practical guidance to framework developers to help them understand how Mesos deals with failures and the tools it provides to enable fault tolerant frameworks. The talk will also cover new Mesos features that allow framework developers to control how partitioned tasks should be handled. Mesos operators will also benefit from a discussion of exactly how Mesos behaves during network partitions and other failure scenarios.

Speakers

Neil Conway

Neil Conway is an engineer at Mesosphere, and an Apache Mesos Committer and PMC Member. At Mesosphere, he has worked on a variety of projects, including partition-aware Mesos frameworks and improving support for stateful Mesos frameworks. Prior to joining Mesosphere, he completed... Read More →

Wednesday August 31, 2016 5:20pm - 6:10pm PDT
Ballroom A

Developer, Intermediate

5:20pm PDT

Taming Apache Mesos: Tools to Make Scalable Software Easier, Quicker and More Resilient - Pini Reznik, Container Solutions

Cloud-based infrastructure is providing businesses with enhanced resiliency, improved performance and reduced costs. Development against such infrastructure, however, has become turgid and fragile. Even with the revolution of “infrastructure as code”, we still see hardcoded IP Addresses, sleeps and randomly failing unit tests. Test’s themselves are bloated with boilerplate connection code and massive delays as test infrastructure is provisioned. The rapid feedback loops promoted in the previous decade are fast becoming impossible to achieve.

The presentation will introduce the audience to Apache Mesos and several Mesos-related projects that we hope solve some of these problems. Minimesos is an in-memory development environment to allow engineers to develop scalable applications on their own machine. Mesos-Starter is a Spring component that enables users to create a Mesos framework in a matter of minutes. And our bleeding edge project, Mesos-Monkey, provides a tool to harden, test and verify an application's resiliency to failure.

The audience will also be able to try online versions of MiniMesos throughout and after the presentation.

Speakers

Pini Reznik

CTO, Container Solutions

Pini Reznik is a Co-founder and CTO of Container Solutions, Pini oversees the architecture and implementation of Cloud Native migrations for organisations from every sector. His work spans two decades in the configuration management field, with current emphasis on DevOps, automation... Read More →

Wednesday August 31, 2016 5:20pm - 6:10pm PDT
Ballroom B

Developer, Advanced

5:20pm PDT

It’s Complicated, Okay (or Let’s Talk Openly about Mesos’ OSS Neighbors, Friends, and Rivals) - Aaron Williams, Mesosphere

Mesos is never the only OSS you need to run your production datacenter. And just like all of us hanging out together at MesosCon, if you surround Mesos with its OSS friends, you get a happy, highly productive Mesos. But you have to be careful, not all OSS plays well with others.

In this talk, we’ll start by looking at a handful of production Mesos datacenters from major users. We’ll use these real-world examples to abstract a standard Mesos datacenter architecture that contains all of the components needed to run today’s modern, containerized apps with big data and analytics frameworks.

With that harmonized Mesos datacenter architecture, we’ll look at each of the abstract components and discuss the leading OSS projects that fit each piece. We’ll discuss the characteristics of what makes some solutions work well with Mesos, and call out the projects that don’t meet the standard.

We’ll wrap up the talk by showing the complete architecture diagram, and show the single- command trick for bringing all of the best OSS components together and getting them up and running in a production cluster.

Speakers

Aaron Williams

Engineering Leader, Mesosphere

Accomplished engineering manager with a passion and drive for building and scaling infrastructure, and utilizing data to solve complex issues. Strong believer in collaborative teamwork -- the sum is greater than its parts. Demonstrated track record of directing fast-paced, high-performing... Read More →

Wednesday August 31, 2016 5:20pm - 6:10pm PDT
Ballroom C

Operator, Beginner

6:15pm PDT

Onsite Attendee Reception & Sponsor Showcase

Join fellow attendees and sponsor companies for conversation, snacks and beverages.

Wednesday August 31, 2016 6:15pm - 7:30pm PDT
Diamond + Staten

Breaks

8:00am PDT

Registration

Thursday September 1, 2016 8:00am - 9:00am PDT
Hotel Lobby

Breaks

9:00am PDT

Welcome and Opening Remarks - Artem Harutyunyan, Mesosphere

Speakers

Artem Harutyunyan

Director of Engineering, Mesosphere

Artem Harutyunyan is a Director of Engineering at Mesosphere where he works on Apache Mesos and Marathon projects. Prior to joining Mesosphere he spent several years at Qualys where he led the design and development of a distributed system for storing and analyzing large volumes of... Read More →

Thursday September 1, 2016 9:00am - 9:05am PDT
Ballroom ABC

Keynotes

9:05am PDT

Keynote: Bringing Mesos to the World, A Panel Discussion - William Freiberg, COO, Mesosphere

One of the main concerns with Open Source software is adoption acceleration and innovation within the open source community. More contributors, collaborating with less friction. All friction points need to be removed to accelerate the Apache Mesos community and create exponential growth in the ecosystem. Commercial Mesos fuels adoption, apps and services drive solutions, and adopters create demand, which all ensure the Mesos community becomes more powerful and robust. In this talk, William Freiberg, COO of Mesosphere will take a deeper look into all the components that will continue to catalyze the Apache Mesos ecosystem.

Moderators

William Freiberg

William Freiberg is COO role of Mesosphere, after serving as the company’s Chief Business Officer and head of Strategic Development since 2014. During that time, Freiberg oversaw development of Mesosphere’s enterprise sales team and helped the company navigate key partnerships... Read More →

Speakers

Kaushik Bhattacharya

Cloud Solutions Architect, Microsoft

Cole Crawford

CEO, Vapor.io

Adam Mollenkopf

Real-Time & Big Data GIS Capability Lead, Esri

Adam Mollenkopf is responsible for the strategic direction Esri takes towards enabling real-time and big data capabilities in the ArcGIS platform. This includes having the ability to ingest real-time data streams from a wide variety of sources, performing continuous and recurring... Read More →

Thursday September 1, 2016 9:05am - 9:15am PDT
Ballroom ABC

Keynotes

9:15am PDT

Keynote: The Impact of Distributed Computing on the Evolution of Devops - Kris Buytaert, Devops, Linux and Open Source Expert

What do openMosix, Corosync, Mesos and Kubernetes have in common and how are they all crucial in the evolution of devops and software delivery in general? The devops movement started with #devopsdays Gent in november 2009, but the people starting it were already delivering software for well over a decade. Lessons learned while using some of these distributed platforms formed the foundations of best practices we now often group under the term #devops.

Speakers

Kris Buytaert

Chief Yak Shaver, Inuits.eu

Kris Buytaert is a long time Linux and Open Source Consultant. He's one of instigators of the devops movement, currently working for Inuits He is frequently speaking at, or organizing different international conferences He spends most of his time working on bridging the gap between... Read More →

DevopsandDistributedComputing pdf

Thursday September 1, 2016 9:15am - 9:45am PDT
Ballroom ABC

Keynotes

9:45am PDT

Coffee Break and Sponsor Showcase

Thursday September 1, 2016 9:45am - 10:15am PDT
Diamond + Staten

Breaks

10:15am PDT

Ambrose: Automatic Metric Collection and Correlation in DC/OS - Nick Parker, Mesosphere

DC/OS provides easy deployment and management of clustered applications. In both production and test environments, a deep understanding of the state of those applications is equally if not more important. This talk describes a new suite of components for collecting and tagging metrics in DC/OS systems, both system-level information at the agent nodes, as well as arbitrary metrics from individual applications. Applications are presented with a simple interface for passing metrics upstream, and Ambrose does the rest, tagging all data with its origin as its processed.

Speakers

Nicholas Parker

Mesosphere

Metrics on DC OS Enterprise (Mesoscon) pdf

Thursday September 1, 2016 10:15am - 11:05am PDT
Ballroom B

Developer, Beginner

10:15am PDT

Performing Process Migration with Mesos - Kapil Arya & Isabel Jimenez, Mesosphere

Process migration (also known as process checkpointing) is the ability to move a group of related running processes from one set of nodes to another. It involves three phases: take a snapshot of all processes, copy the snapshot data over to the target nodes, and restart processes from that snapshot. Process migration has traditionally been used for fault-tolerance in the context of long-running stateful applications. Without it, the application developers need to modify the stateful application to periodically save the state to disk in order to restart in case of a failure. This is inefficient and error-prone!

In this talk, we demonstrate process migration within a Mesos cluster for a more enjoyable way to schedule your containers. Apart from fault-tolerance, process migration within Mesos opens up new avenues for implementing better scheduling policies. Some of the other use cases include improved maintenance primitives, debugging, speculative execution and managing “tails” for multi-threaded applications.

Speakers

Kapil Arya

Kapil Arya is an Engineer at Mesosphere focussing on the core team. He recently finished his PhD at Northeastern University where he was the lead developer of the open source distributed checkpointing project DMTCP, and contributed to the reversible debugger FReD. Interning at VMware... Read More →

Isabel Jimenez

Distributed systems engineer, Mesosphere

Isabel Jimenez is a Software Engineer at Mesosphere.

MesosConEU2016 ProcessMigration pdf

Thursday September 1, 2016 10:15am - 11:05am PDT
Ballroom C

Operator, Beginner

10:15am PDT

Building an Immutable MMO Gaming Platform - Emmanuel Rieg & Ross McKinley, DIGIT Game Studios

As a mobile gaming company, we need to cater for unpredictable spikes in users, and scale each part of our application individually. We also need to support fast development iterations, which might need a new short-lived environment.

This talk will highlight how we go from a blank canvas AWS account, to a fully functional PaaS, without making any compromises on the infrastructure. We will talk about High Availability across AWS zones, to ensure Mesos is always available to meet the harsh demands of our environment.

This talk will show how we design our infrastructure to keep Mesos running all the time, how we can launch a new cluster in 5 minutes, and how we deploy our environment on top of it, using Infrastructure-as-code tools like Terraform and Packer.

Speakers

Ross McKinley

Build & Release Engineer, DIGIT Game Studios

Build engineer interested in automating workflows, autonomous systems, creating a culture of awesome, and building infrastructure and tools that meet the needs of the company.

Emmanuel Rieg

DevOps Engineer, DIGIT Game Studios

Emmanuel is a DevOps at DIGIT games Studio, he is the one building and automating the whole infrastructure at DIGIT and make sure everything is working autonomously . He as been interested by Mesos since couple year but only start using it effectively last year.

DIGIT MesosCon EU 2016 Building An Immutable MMO Gaming Platform pdf

Thursday September 1, 2016 10:15am - 11:05am PDT
Ballroom A

User, Intermediate

11:15am PDT

Designing and Evaluating a Distributed Computing Language Runtime - Christopher Meiklejohn, Université catholique de Louvain

Consistency is hard and coordination is expensive. As we move into the world of connected 'Internet of Things' style applications, or large-scale mobile applications, devices have less power, periods of limited connectivity, and operate over unreliable asynchronous networks. This poses a problem with shared state: how do we handle concurrent operations over shared state, while clients are offline, and ensure that values converge to a desirable result without making the system unavailable?

We look at a new programming model, called Lasp. This programming model combines distributed convergent data structures with a dataflow execution model designed for distribution over large-scale applications. This model supports arbitrary placement of processing
node: this enables the user to author applications that can be distributed across data centers and pushed to the edge.

In this talk, we will focus on the design and evaluation of the Lasp runtime system: a system written in Erlang with a target scale of 10,000 - 20,000 nodes. We will look at the supporting algorithms that assist in achieving this scale, and how we’ve leveraged Mesos and DC/OS to build a system for performing experiments and identifying bottlenecks as we develop the language.

Speakers

Christopher Meiklejohn

Ph.D. candidate, Université catholique de Louvain

Thursday September 1, 2016 11:15am - 12:05pm PDT
Ballroom B

Developer, Intermediate

11:15am PDT

Docker At Scale - Pitfalls And How To Debug Them - Jan Schlicht, Mesosphere

Using Mesos to run Docker containers at scale is a common practice for many users.
In this talk we will give an overview of the different ways on how to run docker on top of Mesos including discussing the differences between running docker with different (i.e., Mesos, Docker, or universal) containerizers.

As running Docker at large scale offers its own challenges (e.g., how to start up 1000 containers as quickly as possible), we present best practices and common pitfalls we encountered over the last years. We also discuss approaches for debugging Docker related problems.

MesosCon 2016 Docker At Scale pdf

Thursday September 1, 2016 11:15am - 12:05pm PDT
Ballroom C

Operator, Intermediate

11:15am PDT

Conveyor: Enabling Low-friction Continuous Deployment via Workflows and Containers with Marathon and Apache Mesos - Jonathan Pliske & Tristan Blease, Groupon

Technology organizations are complex environments that bind deeply technical tools and infrastructure with strong cultural threads. Software like Mesos and Docker offer compelling stories around scalability, fault tolerance, and deterministic environments. Nonetheless, many challenges emerge as one attempts to weave these components into an existing engineering culture.

A successful transition to a service-oriented architecture has left Groupon with numerous services deployed to a variety of hardware, physical and virtual, via a diverse set of tools. Along the way the organization has picked up an array of operational readiness requirements and deploy controls. All of these help to maintain stability and performance but come at a cost: ease of deployment.

Conveyor aims to maintain and improve Groupon’s high bar for production software quality, while streamlining the process of getting code into production. Powerful tools like Apache Mesos, Marathon, and Docker form the foundation of this approach. This talk covers how Conveyor uses these tools together and the various technical and cultural hurdles encountered in the process of rolling out this infrastructure within the enterprise.

Speakers

Tristan J Blease

Principal Software Engineer, Groupon

Tristan Blease is a principal software engineer at Groupon. He focuses on leading teams that are building tooling and infrastructure to ensure the success of our cloud migration.

Jonathan Pliske

Groupon

Conveyor MesosCon EU (1) pdf

Thursday September 1, 2016 11:15am - 12:05pm PDT
Ballroom A

User, Beginner

12:05pm PDT

Lunch and Sponsor Showcase

Thursday September 1, 2016 12:05pm - 1:30pm PDT
Diamond + Staten

Breaks

1:30pm PDT

One is not enough - Using Multiple Disk Support - Joris Van Remoortere, Jörg Schad & Felix Hupfeld, Mesosphere

Multiple Disk (MESOS-191) is an age old feature request that was implemented at last year's MesosCon EU hackathon. It allows operators to expose multiple disk resources. This enables frameworks to a) utilize more than disk per agent and b) gain exclusive access to a complete disk device (including disk errors when they attempt to exceed the capacity of the volume instead of having Mesos kill the Task when exceeding its disk resource limit).
This talk will give an overview of the different types of disk resources now available (Root, Path, and Mount Disk) and guidelines on when to use which. Secondly we will discuss how multiple disk resources can be utilized by users who are running their workloads using Marathon. Last we will provide best practices for framework developer who are looking for utilizing this feature.

This will be done on the example of the Mesos Quobyte framework which was one of the first frameworks to take advantage of multiple disk support.

Speakers

Felix Hupfeld

Founder, Quobyte

Felix manages Quobyte’s technology and pushes development forward. Before that, he worked as a tech lead and capacity planner in Google’s infrastructure team (2009–2013). He was the architect and project manager for XtreemFS (2006–2009). Felix’s PhD was on distributed s... Read More →

Joris Van Remoortere

Joris Van Remoortere is a Senior Software Engineer at Mesosphere. He focuses on large-scale, low-latency, high-throughput systems. He alleviates scaling issues and makes performance and reliability improvements on many projects, including Orly, messaging and payment systems, web tiers... Read More →

Jörg Schad

CTO, ArangoDB

Jörg Schad is the CTO at ArangoDB. In a previous life, he has worked on or built machine learning pipelines in healthcare, distributed systems, including early Kubernetes code at Mesosphere, and in-memory databases. He received his Ph.D. for research about distributed databases and... Read More →

MesosCon16 Multiple Disk pdf

Thursday September 1, 2016 1:30pm - 2:20pm PDT
Ballroom B

Developer, Intermediate

1:30pm PDT

Your Containers did WHAT??? Securing and Monitoring Containerized Apps - Alessandro Gallotta, Sysdig

The next generation distributed data center architecture is making applications more powerful and more responsive. But as many teams are starting to find out, the complexity of securing these applications and monitoring their behavior can be impractical, painful, and sometimes plain impossible.

In this demo-driven presentation, Luca Marturana will take you through the underlying challenges of container operations, cover the current state of the art of container and microservice monitoring, and discuss new techniques such as behavioral monitoring to secure your infrastructure. Using open source tools running in live environments, he will demonstrate how to effectively monitor, troubleshoot, and secure Mesos deployments.

The presentation will feature live interaction with container environments and live demos of all tools and techniques discussed. Special emphasis will be put on using the Mesos portfolio of scheduling and management tools as well as sysdig, an open source container and system troubleshooting tool developed by the presenter, and the open source behavioral security monitor falco.

GitHub link: https://github.com/draios/sysdig

Specific topics will include:
* visualizing the physical vs logical architecture of Mesos & DCOS deployments
* monitoring performance at the holistic microservice/application level for orchestrated systems
* Leveraging Mesos metadata such as Master, Slave, Marathon and labels for more intelligent troubleshooting
* identifying and surfacing anomalous system activity of individual Docker containers

Speakers

Alessandro Gallotta

Software Engineer, Sysdig

Alessandro Gallotta is a software engineer at Sysdig. He is a core developer where he focuses on backend services dealing with big data and high availability issues. He holds a M.Sc. in Computer Engineering from University of Catania, Italy. Prior to Sysdig he worked as web developer... Read More →

Thursday September 1, 2016 1:30pm - 2:20pm PDT
Ballroom C

Operator, Intermediate

1:30pm PDT

Automated Testing Laboratory for Embedded Linux Distributions - Paweł Wieczorek, Samsung Electronics

Shipping quality software always involves in-depth testing. In order to minimize time spent on repetitive actions, this task should be fully automated. Unfortunately this creates many problems which have to be solved to ensure stability of the whole process. During this talk Paweł will discuss key problems faced while building automatic testing infrastructure for Tizen operating system images. He will also present how Tizen release team dealt with such an uneasy task. Both hardware and software presented in this talk is open (https://git.tizen.org/cgit/tools/testlab/sd-mux.git and https://git.tizen.org/cgit/tools/testlab/major.git) and can be easily used to build embedded software testing lab.

Speakers

Paweł Wieczorek

Samsung R&D Institute Poland

Paweł Wieczorek works at Samsung R&D Institute Poland since 2014. Starting as an access control developer, Paweł contributed to the security framework of Tizen operating system. At that time, he introduced testing automation practices to Tizen and still actively develops automated... Read More →

AutomatedTestingLaboratory pdf

Thursday September 1, 2016 1:30pm - 2:20pm PDT
Ballroom A

User, Beginner

2:30pm PDT

Practical, Resilient Software Defined Networking: A Field Report - Sargun Dhillon & Avinash Sridharan, Mesosphere

Mesos, as it comes out of the box, has some limitations in its network infrastructure. The mechanism out of the box does not work with legacy applications that expect stable ports, and IPs. We built a general purpose, solution using off-the-shelf components to provide overlay networking for Mesos clusters. This overlay network is meant to become the default way of doing networking in Mesosphere's DC/OS. In this talk, Avinash, Jie, and Sargun will talk about the design for the Mesos modules, isolators, and integration into Docker. This mechanism sits along side an external distributed system to orchestrate it, and glue it all together. This system is meant to be a flexible, highly available system that is meant to run in the stable state without any masters. We'll talk about why we decided to go with this design and how it works as deployed in the real DC/OS installations.

Speakers

Sargun Dhillon

Avinash Sridharan

Software Engineer, Mesosphere

Love containers, especially networking them !! Have worked (and studied) in the field of networking for the past decade or so. Very interested in designing and developing new network technologies. Want everything to be fast, distributed, and extremely efficient.

MesosCon 2016 Avinash pdf

Thursday September 1, 2016 2:30pm - 3:20pm PDT
Ballroom B

Developer, Intermediate

2:30pm PDT

Securing Marathon Installation - Gaston Kleiman, Mesosphere

Walk through Marathon’s architecture.
Describe how to secure the communication with Mesos
Describe the authentication/authorization plugin interface
Show an example plugin

Speakers

Gastón Kleiman

Staff Software Engineer, Mesosphere

Gastón Kleiman, Apache Mesos PMC/Committer, is a Staff Software Engineer at Mesosphere. He fell in love with distributed systems and infrastructure automation while contracting for Google, where he got to use Borg, MapReduce and other cool technology. That led him to work at Amazon... Read More →

Securing a Marathon Installation pdf

Thursday September 1, 2016 2:30pm - 3:20pm PDT
Ballroom C

Operator, Beginner

2:30pm PDT

Apple's Mesos API Client - Dario Rexin, Apple

Apple Engineers give an overview of the Mesos HTTP API client that they developed to interact with Mesos. It is built to scale and leverages akka-streams with full support for back pressure. The talk covers the architecture and different stages of the client, challenges faced developing it and concrete usage examples.

Speakers

Dario Rexin

Apple Inc., Software Engineer

mesoscon eu 2016 pdf

Thursday September 1, 2016 2:30pm - 3:20pm PDT
Ballroom A

User, Advanced

3:20pm PDT

Coffee Break and Sponsor Showcase

Thursday September 1, 2016 3:20pm - 4:00pm PDT
Diamond + Staten

Breaks

4:00pm PDT

Supporting GPUs in Docker Containers on Apache Mesos - Kevin Klues, Mesosphere & Yubo Li, IBM

Gaining access to GPUs inside a Docker container is a non-trivial task. Indeed, Nvidia provides both a Docker volume-plugin, as well as a full blown wrapper around Docker itself in order to allow seamless access to GPUs inside a running Docker container. With the advent of the new unified containerizer in Mesos (which runs docker containers natively in Mesos) as well as continued support for the Docker containerizer (which shells out the task of launching docker containers to Docker itself), Mesos needs a way to to seamlessly support GPUs inside running docker containers as well.

This talk covers the details of why gaining access to GPUs inside a running docker container is hard, how Nvidia has solved this problem for standalone docker itself, and how Mesos has now solved this problem in both its unified containerizer and Docker containerizer technologies together.

Speakers

Kevin Klues

Principal Software Engineer, NVIDIA

Kevin Klues is a Principal Software Engineer on the Cloud Native team at NVIDIA. Since joining NVIDIA, Kevin has been involved in the design and implementation of a number of Kubernetes related technologies, including the TopologyManager and NVIDIA's Kubernetes device plugin. Kevin... Read More →

Yubo Li

Research Stuff Member, IBM Research

Dr. Yubo Li is from IBM Research China, Beijing. He is working with GPU enablement on Kubernetes/Mesos/OpenStack cloud and architecture design for deep learning (DL). He is the chief architect for GPU acceleration service on SuperVessel, an open-access cloud with OpenStack and OpenPOWER... Read More →

Supporting GPUs in Docker Containers on Apache Mesos pdf

Thursday September 1, 2016 4:00pm - 4:50pm PDT
Ballroom B

Developer, Beginner

4:00pm PDT

Building the Glue for Service Discovery & Load Balancing Microservices on Apache Mesos - Sargun Dhillon, Mesosphere

One of the challenges that comes from deploying multi-tiered distributed systems, or microservices, atop a dynamic scheduler is the introduction of new problems surrounding load balancing. There are some inherent challenges in building a load balancer that's meant to operate in a highly available way, without any single points of failure. In this talk, Sargun Dhillon will walk through the distributed load balancing mechanism that he built for Mesos. This service discovery mechanism is meant to have the same kinds of features, api, and availability that existed in legacy, statically partitioned environments. The purpose of this is to ease the transition, and remove some of the largest road blocks in moving applications over to modern datacenters. In addition, he will speak to why he built it as opposed to other alternatives for service discovery and load balancing such as using Zookeeper, and the challenges that came from it. We built a library called Lashup that has a membership protocol, a multicast layer, failure detector, and CRDT key/value store. This has allowed us to build applications that orchestrate Mesos clusters with great ease.

Speakers

Sargun Dhillon

MesosCon 2016 pdf

Thursday September 1, 2016 4:00pm - 4:50pm PDT
Ballroom A

Wildcard, Beginner

4:00pm PDT

Continuous Delivery without Stress: Open Source Canary-Releasing with DC/OS and VAMP - Olaf Molenveld & Dragoslav Pavkovic, Magnetic.io/VAMP

DC/OS is a powerful platform to run containers and resilient microservices architectures at scale. But releasing or upgrading software to production often is a stressful moment due to the risk of performance issues or even downtime. Applying canary-patterns to ContinuousDelivery pipelines provides a safety-net which makes releasing containers less risky and stressful. By publishing new software versions to only a small percentage of visitors with specific criteria, it enables you to test, optimise and scale in a controlled and gradual way, without negatively impacting the majority of users. In this presentation we’re going to talk about how VAMP adds powerful opensource canary-releasing features to the DC/OS stack, and how to setup a smart ContinuousDelivery pipeline.

Speakers

Olaf Molenveld

CTO/founder, Magnetic.io/Vamp

Founder and CTO of Vamp.io (formerly Magnetic.io), builders of Vamp. Vamp is a modern cloudnative solution to continuously release new micro services into production without downtime using advanced AI based Canary testing and releasing features, and delivering smart right-scaling... Read More →

Dragoslav Pavkovic

MesosCon2016 MagneticIO Vamp pdf

Thursday September 1, 2016 4:00pm - 5:00pm PDT
Ballroom C

Operator, Beginner

9:00am PDT

Apache Mesos Hackathon sponsored by DC/OS on Azure

Join other members of the Apache Mesos community for a day of hacking - sponsored by DC/OS on Azure.

RSVP required. If you are interested in attending, please add the Hackathon option to your registration.

Categories:
Participants will be asked to divide into teams to focus on Apache Mesos related categories. Topics include Mesos frameworks, core Mesos, documentation or demos for the ecosystem, or other ecosystem tooling.

Judging:
At this year's hackathon, everyone will judge and vote for two winning teams.

Community need

"cool" factor

Prizes:
There will be one winning team from each category, and each winning team member will receive a Hubsan x4 Quadcopter.

Friday September 2, 2016 9:00am - 3:00pm PDT
Ballroom A

Hackathon