DC/OS is a distributed operating system – powered by Apache Mesos – that treats collections of CPUs, RAM, networking and so on as a distributed kernel and then implements core distributed system components that handle system-wide tasks such as scheduling, dns, service discovery, and others without regard to the underlying infrastructure. The Azure Container Service is a reference implementation of DC/OS optimized to take advantage of the features of Microsoft Azure infrastructure. If you already have an Azure account, you can try out a reference implementation of DC/OS built on Microsoft Azure by creating an Azure Container Service cluster. (Grab a free Azure trial account first if you don’t have one.)
This document describes:
- Advantages of using DC/OS
- Azure infrastructure and the implementation architecture
- Bill of materials used to construct DC/OS and the ACS clusters
Benefits of DC/OS
DC/OS is powered by Apache Mesos used as the distributed kernel of a set of computers that you can treat as one unit, though you retain control of each and every one. In DC/OS, the kernel of the system is in fact any number of Mesos masters and agents both publicly available and private; failed Mesos masters are replaced by a standby master transparently, and handles leader election. Masters, of course, handle failed agents and processes.
DC/OS applications function as system components in its distributed user space. The most obvious is the system marathon component, which is the distributed
init for DC/OS; but this also includes the Admin Router service, the Mesos-DNS service, Exhibitor, and other system-wide components that are used by user processes and manage the masters and agents.
Why DC/OS and not Mesos?
Using Mesos directly is in fact what many companies do successfully, so there’s every chance you might want or need to deploy Mesos yourself. DC/OS, however, has several important features above and beyond the set of open-source Mesos features that you can use yourself:
- Deploying and managing Mesos can be complex, precisely because it can manage very complex environments; DC/OS makes that complexity straightforward to use and supported by the community.
- DC/OS implements fault tolerance across not just industrial but internet scale.
- The DC/OS universe of packages for easy installation supports developers, data scientists, and system administrators alike with all their favorite open source packages.
- The real-time metrics “firehouse” is open for use by your favorite diagnostics and analytics packages.
- DC/OS has three ways to automate your distributed OS: a powerful CLI, a lovely UI, and a rich API.
The Universe of packages
It’s worth nothing that point 3, above, are the tools you need to use. Here’s a list of what was in the universe on Day 1 of DC/OS, categorized by the type of license.
Apache License V2
- Apache Cassandra
- Elastic Search
- Apache Hadoop
- Apache Kafka
- Mr Redis
- Spark Notebook
- Apache Spark
- Apache Storm
- Docker Swarm
- Apache Zeppelin
- OpenVPN Admin
And while it’s easy to think of using the universe of packages available with the DC/OS packaging system, you can also publish there to give your skills back to the community, too.
Deploying DC/OS yourself may also be what you want to do, but you can start with a reference implementation of DC/OS in “the cloud” with the Azure Container Service.
Azure Container Service infrastructure and optimizations
The Azure Container Service is built with DC/OS as one of the critical orchestration options and the DC/OS implementation is optimized for easy creation and usage on Microsoft Azure and on-premise, eventually with Azure Stack, and is one of the best ways to get started with DC/OS, Mesos, and a distributed cluster that can be used and managed like one large system whether in your datacenter or in Azure.
The Azure Container Service implementation brings several more benefits to you:
- Pretty much the easiest way to get started with DC/OS. Click a few buttons, provide a few parameters and you are ready to deploy your apps. If you have an Azure account, try it.
- DC/OS deployments are vetted and optimize specifically for Azure: all VMs, storage, networks, load balancers, and so on are created and configured for a highly available DC/OS cluster: it just “happens”.
- Potential to increase integration with Azure services if you decide it would benefit your deployments as the system moves forward.
- Complete support: Brings Microsoft support to the infrastructure to complement Mesosphere’s support of DC/OS.
The default ACS architecture looks like this:
DC/OS component list
The following list shows the components used by DC/OS itself. You’ll note that the core components center around Mesos, Marathon, Python, and so on.