DC/OS – California Sunshine

Learn DC/OS with minidcos (4) – Use Marathon LB

In Part II, we deployed a service on the private node and in this part, let me talk about how to use marathon-lb to expose it onto the public node. For more details of Marathon and Marathon-LB, please check here.

Install Marathon-LB

With CLI, it’s easy to install marathon-lb, which is a package of DC/OS.

> dcos package install marathon-lb
By Deploying, you agree to the Terms and Conditions https://mesosphere.com/catalog-terms-conditions/#community-services
We recommend at least 2 CPUs and 1GiB of RAM for each Marathon-LB instance. 

*NOTE*: For additional ```Enterprise Edition``` DC/OS instructions, see https://docs.mesosphere.com/administration/id-and-access-mgt/service-auth/mlb-auth/
Continue installing? [yes/no] yes
Installing Marathon app for package [marathon-lb] version [1.15.0]
Marathon-lb DC/OS Service has been successfully installed!
See https://github.com/mesosphere/marathon-lb for documentation.

> dcos package install marathon-lb

By Deploying, you agree to the Terms and Conditions https://mesosphere.com/catalog-terms-conditions/#community-services

We recommend at least 2 CPUs and 1GiB of RAM for each Marathon-LB instance.

*NOTE*: For additional ```Enterprise Edition``` DC/OS instructions, see https://docs.mesosphere.com/administration/id-and-access-mgt/service-auth/mlb-auth/

Continue installing? [yes/no] yes

Installing Marathon app for package [marathon-lb] version [1.15.0]

Marathon-lb DC/OS Service has been successfully installed!

See https://github.com/mesosphere/marathon-lb for documentation.

As you can see, the marathon-lb 1.15.0 is installed successfully. By default, it requires 2 CPU with 1GB memory. You can also adjust it from DC/OS GUI. In my testing, I need to remove “net.ipv4.tcp_max_orphans=60000” from Marathon-LB’s “Sysctl Params“. Otherwise, the service will be brought up.

You will find the marathon-lb service is up and it’s deployed on the public node.

> dcos marathon app list
ID               MEM  CPUS  TASKS  HEALTH  DEPLOYMENT  WAITING  CONTAINER  CMD  
/marathon-lb     256   1     1/1    1/1       ---      False      DOCKER   N/A  
/nginx-service   128  0.1    1/1    N/A       ---      False      DOCKER   N/A  
/nginx-service2  128  0.1    1/1    N/A       ---      False      DOCKER   N/A  

> dcos task 
NAME            HOST        USER  STATE  ID                                                   MESOS ID                                 REGION  ZONE  
marathon-lb     <strong>172.17.0.4</strong>  root    R    marathon-lb.8bcefb33-5a3b-11eb-8525-70b3d5800002     41f320ba-c2f6-44db-a454-d31621b1c57c-S0   ---    ---   
nginx-service   172.17.0.3  root    R    nginx-service.97a7d02f-5a21-11eb-8525-70b3d5800002   41f320ba-c2f6-44db-a454-d31621b1c57c-S0   ---    ---   
nginx-service2  172.17.0.3  root    R    nginx-service2.d327bd30-5a37-11eb-8525-70b3d5800002  41f320ba-c2f6-44db-a454-d31621b1c57c-S1   ---    ---

> dcos marathon app list

ID MEM CPUS TASKS HEALTH DEPLOYMENT WAITING CONTAINER CMD

/marathon-lb 256 1 1/1 1/1 --- False DOCKER N/A

/nginx-service 128 0.1 1/1 N/A --- False DOCKER N/A

/nginx-service2 128 0.1 1/1 N/A --- False DOCKER N/A

> dcos task

NAME HOST USER STATE ID MESOS ID REGION ZONE

marathon-lb 172.17.0.4 root R marathon-lb.8bcefb33-5a3b-11eb-8525-70b3d5800002 41f320ba-c2f6-44db-a454-d31621b1c57c-S0 --- ---

nginx-service 172.17.0.3 root R nginx-service.97a7d02f-5a21-11eb-8525-70b3d5800002 41f320ba-c2f6-44db-a454-d31621b1c57c-S0 --- ---

nginx-service2 172.17.0.3 root R nginx-service2.d327bd30-5a37-11eb-8525-70b3d5800002 41f320ba-c2f6-44db-a454-d31621b1c57c-S1 --- ---

Since marathon-lb uses HAProxy as its backend, we can access the HAProxy statistic panel through http://172.17.0.4:9090/haproxy?stats.

Expose the Nginx Service with Marathon-LB

To expose the nginx service, we can simply add following label “HAPROXY_GROUP: external” tn the configuration.

{
  ...
   "container": {
   "portMappings": [
   {
      "containerPort": 80,
      "hostPort": 0,
      "labels": {
          "VIP_0": "/nginx-service:80"
      },
      "protocol": "tcp",
      "servicePort": 10000,
      "name": "nginx-service"
    }
  ],
  "labels": {
      "HAPROXY_GROUP":"external"
  },
  ...
}

{

...

"container": {

"portMappings": [

{

"containerPort": 80,

"hostPort": 0,

"labels": {

"VIP_0": "/nginx-service:80"

"protocol": "tcp",

"servicePort": 10000,

"name": "nginx-service"

}

"labels": {

"HAPROXY_GROUP":"external"

...

}

After relaunching the nginx service, you can access the nginx from public node (172.17.0.4) on service port 10000, which the real service runs on the private node (172.17.0.3).

> curl http://172.17.0.4:10000
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

> curl http://172.17.0.4:10000

<!DOCTYPE html>

<html>

<head>

<title>Welcome to nginx!</title>

<style>

body {

width: 35em;

margin: 0 auto;

font-family: Tahoma, Verdana, Arial, sans-serif;

}

</style>

</head>

<body>

<h1>Welcome to nginx!</h1>

If you see this page, the nginx web server is successfully installed and

working. Further configuration is required.

For online documentation and support please refer to

<a href="http://nginx.org/">nginx.org</a>.

Commercial support is available at

<a href="http://nginx.com/">nginx.com</a>.

Thank you for using nginx.

</body>

</html>

Reference

Marathon-LB Documentation

Learn DC/OS with minidcos (3) – Install and Use DC/OS CLI

After Part I and Part II, you have a workable DC/OS cluster and a nginx service is running. In this chapter, let’s talk about how to use DC/OS CLI, which is a helpful tool.

Installation

The DC/OS CLI’s version need to be the same as the DC/OS cluster runs. With minidcos, I installed the 1.12.5 so CLI’s version needs to be 1.12 as well. Use the following commands to download the CLI binary.

> curl https://downloads.dcos.io/binaries/cli/linux/x86-64/dcos-1.12/dcos -o dcos

1	> curl https://downloads.dcos.io/binaries/cli/linux/x86-64/dcos-1.12/dcos -o dcos

Then setup it

> ./dcos cluster setup http://172.17.0.2
If your browser didn't open, please follow this link:

    http://172.17.0.2/login?redirect_uri=urn:ietf:wg:oauth:2.0:oob

Enter token from the browser:

> ./dcos cluster setup http://172.17.0.2

If your browser didn't open, please follow this link:

http://172.17.0.2/login?redirect_uri=urn:ietf:wg:oauth:2.0:oob

Enter token from the browser:

The DC/OS page will be open and log into to get the token. When token is typed in, your CLI is already attached to the running cluster. It’s ready to use it.

Commands

DC/OS CLI provides a lot of useful commands. With it, you can do the same as the GUI does. Here are some examples.

> ./dcos node 
   HOSTNAME        IP                         ID                    TYPE             REGION  ZONE  
  172.17.0.3   172.17.0.3  41f320ba-c2f6-44db-a454-d31621b1c57c-S1  agent             None   None  
  172.17.0.4   172.17.0.4  41f320ba-c2f6-44db-a454-d31621b1c57c-S0  agent             None   None  
master.mesos.  172.17.0.2    41f320ba-c2f6-44db-a454-d31621b1c57c   master (leader)   None   None  

> ./dcos service
NAME          HOST     ACTIVE  TASKS  CPU   MEM   DISK  ID                                         
marathon   172.17.0.2   True     1    0.1  128.0  0.0   41f320ba-c2f6-44db-a454-d31621b1c57c-0001  
metronome  172.17.0.2   True     0    0.0   0.0   0.0   41f320ba-c2f6-44db-a454-d31621b1c57c-0000

> ./dcos task
NAME           HOST        USER  STATE  ID                                                  MESOS ID                                 REGION  ZONE  
nginx-service  172.17.0.4  root    R    nginx-service.97a7d02f-5a21-11eb-8525-70b3d5800002  41f320ba-c2f6-44db-a454-d31621b1c57c-S0   ---    ---   

> ./dcos task log nginx-service
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Configuration complete; ready for start up

> dcos marathon app list
ID              MEM  CPUS  TASKS  HEALTH  DEPLOYMENT  WAITING  CONTAINER  CMD  
/nginx-service  128  0.1    1/1    N/A       ---      False      DOCKER   N/A

> ./dcos node

HOSTNAME IP ID TYPE REGION ZONE

172.17.0.3 172.17.0.3 41f320ba-c2f6-44db-a454-d31621b1c57c-S1 agent None None

172.17.0.4 172.17.0.4 41f320ba-c2f6-44db-a454-d31621b1c57c-S0 agent None None

master.mesos. 172.17.0.2 41f320ba-c2f6-44db-a454-d31621b1c57c master (leader) None None

> ./dcos service

NAME HOST ACTIVE TASKS CPU MEM DISK ID

marathon 172.17.0.2 True 1 0.1 128.0 0.0 41f320ba-c2f6-44db-a454-d31621b1c57c-0001

metronome 172.17.0.2 True 0 0.0 0.0 0.0 41f320ba-c2f6-44db-a454-d31621b1c57c-0000

> ./dcos task

NAME HOST USER STATE ID MESOS ID REGION ZONE

nginx-service 172.17.0.4 root R nginx-service.97a7d02f-5a21-11eb-8525-70b3d5800002 41f320ba-c2f6-44db-a454-d31621b1c57c-S0 --- ---

> ./dcos task log nginx-service

/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration

/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/

/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh

10-listen-on-ipv6-by-default.sh: Getting the checksum of /etc/nginx/conf.d/default.conf

10-listen-on-ipv6-by-default.sh: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf

/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh

/docker-entrypoint.sh: Configuration complete; ready for start up

> dcos marathon app list

ID MEM CPUS TASKS HEALTH DEPLOYMENT WAITING CONTAINER CMD

/nginx-service 128 0.1 1/1 N/A --- False DOCKER N/A

For more details, please use argument “-h” to check.

Let’s use CLI to run a service. I extracted the Nginx service configuration into the JSON file and the id was changed to “/nginx-service2”.

{
  "id": "/nginx-service2",
  "backoffFactor": 1.15,
  "backoffSeconds": 1,
  "container": {
    "portMappings": [
      {
        "containerPort": 80,
        "hostPort": 0,
        "labels": {
          "VIP_0": "/nginx-service:80"
        },
        "protocol": "tcp",
        "servicePort": 10000,
        "name": "nginx-service"
      }
    ],
    "type": "DOCKER",
    "volumes": [],
    "docker": {
      "image": "nginx:stable",
      "forcePullImage": false,
      "privileged": false,
      "parameters": []
    }
  },
  "cpus": 0.1,
  "disk": 0,
  "instances": 1,
  "maxLaunchDelaySeconds": 3600,
  "mem": 128,
  "gpus": 0,
  "networks": [
    {
      "mode": "container/bridge"
    }
  ],
  "requirePorts": false,
  "upgradeStrategy": {
    "maximumOverCapacity": 1,
    "minimumHealthCapacity": 1
  },
  "killSelection": "YOUNGEST_FIRST",
  "unreachableStrategy": {
    "inactiveAfterSeconds": 0,
    "expungeAfterSeconds": 0
  },
  "healthChecks": [],
  "fetch": [],
  "constraints": []
}

{

"id": "/nginx-service2",

"backoffFactor": 1.15,

"backoffSeconds": 1,

"container": {

"portMappings": [

{

"containerPort": 80,

"hostPort": 0,

"labels": {

"VIP_0": "/nginx-service:80"

"protocol": "tcp",

"servicePort": 10000,

"name": "nginx-service"

}

"type": "DOCKER",

"volumes": [],

"docker": {

"image": "nginx:stable",

"forcePullImage": false,

"privileged": false,

"parameters": []

}

"cpus": 0.1,

"disk": 0,

"instances": 1,

"maxLaunchDelaySeconds": 3600,

"mem": 128,

"gpus": 0,

"networks": [

{

"mode": "container/bridge"

}

"requirePorts": false,

"upgradeStrategy": {

"maximumOverCapacity": 1,

"minimumHealthCapacity": 1

"killSelection": "YOUNGEST_FIRST",

"unreachableStrategy": {

"inactiveAfterSeconds": 0,

"expungeAfterSeconds": 0

"healthChecks": [],

"fetch": [],

"constraints": []

}

Then use “dcos marathon app add” to create a new service “/nginx-service2”

> dcos marathon app add nginx.json 
Created deployment 149eaeee-2c69-4102-99ca-a7415cd67c55

> dcos marathon app list
ID               MEM  CPUS  TASKS  HEALTH  DEPLOYMENT  WAITING  CONTAINER  CMD  
/nginx-service   128  0.1    1/1    N/A       ---      False      DOCKER   N/A  
/nginx-service2  128  0.1    1/1    N/A       ---      False      DOCKER   N/A 

> dcos marathon app show /nginx-service2
{
  "backoffFactor": 1.15,
  "backoffSeconds": 1,
  "container": {
    "docker": {
      "forcePullImage": false,
      "image": "nginx:stable",
      "parameters": [],
      "privileged": false
    },
    "portMappings": [
      {
        "containerPort": 80,
        "hostPort": 0,
        "labels": {
          "VIP_0": "/nginx-service:80"
        },
        "name": "nginx-service",
        "protocol": "tcp",
        "servicePort": 10000
      }
    ],
    "type": "DOCKER",
    "volumes": []
  },
  "cpus": 0.1,
  "deployments": [],
  "disk": 0,
  "executor": "",
  "gpus": 0,
  "id": "/nginx-service2",
  "instances": 1,
  "killSelection": "YOUNGEST_FIRST",
  "labels": {},
  "maxLaunchDelaySeconds": 3600,
  "mem": 128,
  "networks": [
    {
      "mode": "container/bridge"
    }
  ],
  "requirePorts": false,
  "tasks": [
    {
      "appId": "/nginx-service2",
      "healthCheckResults": [],
      "host": "172.17.0.3",
      "id": "nginx-service2.d327bd30-5a37-11eb-8525-70b3d5800002",
      "ipAddresses": [
        {
          "ipAddress": "172.18.0.2",
          "protocol": "IPv4"
        }
      ],
      "localVolumes": [],
      "ports": [
        29349
      ],
      "servicePorts": [],
      "slaveId": "41f320ba-c2f6-44db-a454-d31621b1c57c-S1",
      "stagedAt": "2021-01-19T09:22:17.727Z",
      "startedAt": "2021-01-19T09:22:33.342Z",
      "state": "TASK_RUNNING",
      "version": "2021-01-19T09:22:17.599Z"
    }
  ],
  "tasksHealthy": 0,
  "tasksRunning": 1,
  "tasksStaged": 0,
  "tasksUnhealthy": 0,
  "unreachableStrategy": {
    "expungeAfterSeconds": 0,
    "inactiveAfterSeconds": 0
  },
  "upgradeStrategy": {
    "maximumOverCapacity": 1,
    "minimumHealthCapacity": 1
  },
  "version": "2021-01-19T09:22:17.599Z",
  "versionInfo": {
    "lastConfigChangeAt": "2021-01-19T09:22:17.599Z",
    "lastScalingAt": "2021-01-19T09:22:17.599Z"
  }
}

> dcos marathon app add nginx.json

Created deployment 149eaeee-2c69-4102-99ca-a7415cd67c55

> dcos marathon app list

ID MEM CPUS TASKS HEALTH DEPLOYMENT WAITING CONTAINER CMD

/nginx-service 128 0.1 1/1 N/A --- False DOCKER N/A

/nginx-service2 128 0.1 1/1 N/A --- False DOCKER N/A

> dcos marathon app show /nginx-service2

{

"backoffFactor": 1.15,

"backoffSeconds": 1,

"container": {

"docker": {

"forcePullImage": false,

"image": "nginx:stable",

"parameters": [],

"privileged": false

"portMappings": [

{

"containerPort": 80,

"hostPort": 0,

"labels": {

"VIP_0": "/nginx-service:80"

"name": "nginx-service",

"protocol": "tcp",

"servicePort": 10000

}

"type": "DOCKER",

"volumes": []

"cpus": 0.1,

"deployments": [],

"disk": 0,

"executor": "",

"gpus": 0,

"id": "/nginx-service2",

"instances": 1,

"killSelection": "YOUNGEST_FIRST",

"labels": {},

"maxLaunchDelaySeconds": 3600,

"mem": 128,

"networks": [

{

"mode": "container/bridge"

}

"requirePorts": false,

"tasks": [

{

"appId": "/nginx-service2",

"healthCheckResults": [],

"host": "172.17.0.3",

"id": "nginx-service2.d327bd30-5a37-11eb-8525-70b3d5800002",

"ipAddresses": [

{

"ipAddress": "172.18.0.2",

"protocol": "IPv4"

}

"localVolumes": [],

"ports": [

29349

"servicePorts": [],

"slaveId": "41f320ba-c2f6-44db-a454-d31621b1c57c-S1",

"stagedAt": "2021-01-19T09:22:17.727Z",

"startedAt": "2021-01-19T09:22:33.342Z",

"state": "TASK_RUNNING",

"version": "2021-01-19T09:22:17.599Z"

}

"tasksHealthy": 0,

"tasksRunning": 1,

"tasksStaged": 0,

"tasksUnhealthy": 0,

"unreachableStrategy": {

"expungeAfterSeconds": 0,

"inactiveAfterSeconds": 0

"upgradeStrategy": {

"maximumOverCapacity": 1,

"minimumHealthCapacity": 1

"version": "2021-01-19T09:22:17.599Z",

"versionInfo": {

"lastConfigChangeAt": "2021-01-19T09:22:17.599Z",

"lastScalingAt": "2021-01-19T09:22:17.599Z"

}

From GUI, you can see the similar status.

All right. Now you know how to install CLI and some basic commands to check the status of cluster/node/services.

References

DC/OS CLI Documentation

Learn DC/OS with minidcos (2) – Deploy the First Service

After reading Part I, you have a running DC/OS “cluster”, the dashboard UI ready for exploration. Now let’s talk about how to deploy service there. DC/OS management component runs on master nodes, so to-be-deployed services will run on agent nodes.

Agent Nodes – Public/Private Nodes

If you click “Nodes” on dashboard UI, you will see there are two agent nodes.

One is a private node, the other is an public one. A DC/OS agent node is a node on which user tasks are run. Agent nodes contain a few DC/OS components, including a Mesos agent process. Agent nodes can be public or private, depending on agent and network configuration.

A private agent node is an agent node that is on a network that does not allow access from outside of the cluster via the cluster’s infrastructure networking.
A public agent node is an agent node that is on a network that allows access from outside of the cluster via the cluster’s infrastructure networking.

More details about node types can be found here.

Deploy a Nginx Service with GUI

There two ways to deploy service – using GUI and using CLI. Let’s try GUI first and DC/OS CLI will be in later chapter.

Click “Services” on navigation bar, click “Run a service”, then click “Single Container”.

Now the container configuration UI shows up. Let’s do minimum configuration and bright up a nginx server.

On Service page

You need to define the service name, the container type (better to use “docker” with minidcos docker. With real cluster, you can use UCR), the image, how many instances, how much CPU/memory/GPU/disk and etc.

On Networking Page

Select “Bridge”, type the container’s port and service endpoint name (for service discovery), and enable the “Load Balanced Port” (Will talk about it in later chapters).

More details about DC/OS networking can be found here.

Other Pages

Placements – Specify more constraints and tell DC/OS (underlying, it’s Marathon) to deploy the service on which node.
Volumes – Specify a volume to mount to your service.
Health Checks – Specify endpoints for health checking. DC/OS will periodically check the health of services. Details are here.
Environment – Specify the environment variables (used by launching your services) or labels (expose more information to other services).

JSON Editor

You can enable JSON editor to review all the configuration in JSON format. The JSON file can also be used by DC/OS CLI, which allows us to version the configuration.

Run the service

After configuration, click “Review & Run“; After reviewing, click “Run service“. On “Services” panel, you will see the new service is being deployed and running.

Now you will find the service runs on the private node (172.17.0.3) and the container’s 80 port was mapped to port 6165.

> docker exec -it dcos-e2e-default-f0bd1-agent-0 bash
[root@dcos-e2e-default-f0bd1-agent-0 /]# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                  NAMES
22cad2bd70f9        nginx:stable        "/docker-entrypoint.…"   12 minutes ago      Up 12 minutes       0.0.0.0:6165->80/tcp   mesos-6b61698d-b50b-4dda-8251-e275515d285d

> docker exec -it dcos-e2e-default-f0bd1-agent-0 bash

[root@dcos-e2e-default-f0bd1-agent-0 /]# docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

22cad2bd70f9 nginx:stable "/docker-entrypoint.…" 12 minutes ago Up 12 minutes 0.0.0.0:6165->80/tcp mesos-6b61698d-b50b-4dda-8251-e275515d285d

You can access this service through your browser.

Deploy the Service on Public Node

Deploying the service on private node is the default behavior. But if you want to expose the service on a public node, you need to add the following into the JSON configuration.

<span class="token property">"acceptedResourceRoles"</span><span class="token operator">:</span><span class="token punctuation">[</span>
      <span class="token string">"slave_public"</span>
 <span class="token punctuation">]</span><span class="token punctuation">,</span>

"acceptedResourceRoles":[

"slave_public"

],

After updating the configuration and rerun the service, you will find the nginx service is running on the public node (172.17.0.4).

> docker exec dcos-e2e-default-8c975-public-agent-0 docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                  NAMES
d6f429c8e7cd        nginx:stable        "/docker-entrypoint.…"   2 minutes ago       Up 2 minutes        0.0.0.0:9134->80/tcp   mesos-b031120a-3411-47fb-8440-07b9b48793bd

> docker exec dcos-e2e-default-8c975-public-agent-0 docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

d6f429c8e7cd nginx:stable "/docker-entrypoint.…" 2 minutes ago Up 2 minutes 0.0.0.0:9134->80/tcp mesos-b031120a-3411-47fb-8440-07b9b48793bd

But usually, the private node will have much more CPU core and memory than public node. We can use load balancer, such as Marathon LB, Edge LB (Enterprise Edition) to expose the services on public nodes, meanwhile the services are running on private nodes. I will talk about it in the next chapter.

Great! Now you have the first service running on DC/OS!

References

Learn DC/OS with minidcos (1) – Installation and Initial DC/OS cluster Launch

As a distributed system, DC/OS is itself a distributed system, a cluster manager, a container platform, and an operating system. It includes a group of agent nodes with different responsibilities. To configure the whole system is not easy. Luckily, there is a tool named minidcos, which allow you to deploy “fake” agent nodes on the same machine, even a laptop.

This serial of articles aim to document all the details of install minidcos, configure the “fake” cluster and deploy services there.

Install minidcos

You can follow the instructions to install minidcos.

I did it on Ubuntu 18.04.5. First, install the missing software.

> sudo apt update
> sudo apt install -y git python3-dev python3.6-dev libffi-dev libssl-dev git python3-pip

> sudo pip3 install setuptools

> sudo apt update

> sudo apt install -y git python3-dev python3.6-dev libffi-dev libssl-dev git python3-pip

> sudo pip3 install setuptools

Install the minidcos through pip.

> pip3 install --user --upgrade git+https://github.com/dcos/dcos-e2e.git

1	> pip3 install --user --upgrade git+https://github.com/dcos/dcos-e2e.git

Note: here the official instruction has some mistakes. Use mine.

Next step is to install docker.

> sudo apt install docker.io

# add yourself into docker group
> sudo groupadd docker
> sudo usermod -aG docker $USER

> sudo apt install docker.io

# add yourself into docker group

> sudo groupadd docker

> sudo usermod -aG docker $USER

Need to restart/re-login to make sure your account is in docker group.

Use following command to test if minidcos and docker are installed correctly.

> minidcos docker doctor

1	> minidcos docker doctor

If there is any errors/warnings, you probably need to fix them.

minidcos supports different environments to manage the DC/OS cluster, such as AWS, Docker and Vagrant. Docker is the most familiar tool for me so I will use it in my rest explanation.

> minidcos
Usage: minidcos [OPTIONS] COMMAND [ARGS]...

  Manage DC/OS clusters.

Options:
  --version   Show the version and exit.
  -h, --help  Show this message and exit.

Commands:
  aws      Manage DC/OS clusters on AWS.
  docker   Manage DC/OS clusters on Docker.
  vagrant  Manage DC/OS clusters on Vagrant.

> minidcos

Usage: minidcos [OPTIONS] COMMAND [ARGS]...

Manage DC/OS clusters.

Options:

--version Show the version and exit.

-h, --help Show this message and exit.

Commands:

aws Manage DC/OS clusters on AWS.

docker Manage DC/OS clusters on Docker.

vagrant Manage DC/OS clusters on Vagrant.

Download DC/OS installation package

Some blogs suggest you to use “dcos download-installer” to download the latest DC/OS installation package. But as I tried, minidcos doesn’t work well with the latest DC/OS version. So I suggest you to download DC/OS 1.12.5 by this link. (All versions can be found on this page.) Because of it, you can’t try the latest features in newer DC/OS version at this moment.

Launch DC/OS Cluster

I downloaded DC/OS 1.12.5 (dcos_generate_config.sh) onto ~/dcos. It’s time to launch it. By the following command, the “fake” DC/OS cluster (named “default“) will be brought up.

> cd ~/dcos
> minidcos docker create ./dcos_generate_config.sh --agents 1
✔ Determining DC/OS variant
✔ Creating cluster
✔ Installing DC/OS
Cluster "default" has started. Run "minidcos docker wait --cluster-id default" to wait for DC/OS to become ready.
default

> cd ~/dcos

> minidcos docker create ./dcos_generate_config.sh --agents 1

✔ Determining DC/OS variant

✔ Creating cluster

✔ Installing DC/OS

Cluster "default" has started. Run "minidcos docker wait --cluster-id default" to wait for DC/OS to become ready.

default

Depends on how powerful your environment is, it may take some time to finish the deployment. Use following command to check.

> minidcos docker wait --cluster-id default
A cluster may take some time to be ready.
The amount of time it takes to start a cluster depends on a variety of factors.
If you are concerned that this is hanging, try "minidcos docker doctor" to diagnose common issues.
✔ Waiting for DC/OS variant
If you cancel this command while it is running, you may not be able to log in. To resolve that, run this command again.
✔ Waiting for DC/OS to start

> minidcos docker wait --cluster-id default

A cluster may take some time to be ready.

The amount of time it takes to start a cluster depends on a variety of factors.

If you are concerned that this is hanging, try "minidcos docker doctor" to diagnose common issues.

✔ Waiting for DC/OS variant

If you cancel this command while it is running, you may not be able to log in. To resolve that, run this command again.

✔ Waiting for DC/OS to start

If you want to find more logs, add “-vvvvv“. (P.S., That’s also how I found minidcos doesn’t support newer DC/OS version well.)

Open DC/OS Management Dashboard

When the cluster is launched, you can log into the management dashboard.

> minidcos docker web

1	> minidcos docker web

After this command, the dashboard page (http://172.17.0.2) should be opened in browser.

I chose “Log in with Google” and the dashboard will finally pop up.

The dashboard panel shows the usage of memory, CPU, disk, GPU, nodes and etc.

Something Under the Hood…

minidcos is using docker container to run the master/agent nodes.

> docker ps -s
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES SIZE
e6434c3253ca mesosphere/dcos-docker "/sbin/init" 26 minutes ago Up 26 minutes dcos-e2e-default-4baab-public-agent-0 701kB (virtual 558MB)
6674717280a3 mesosphere/dcos-docker "/sbin/init" 26 minutes ago Up 26 minutes dcos-e2e-default-4baab-agent-0 697kB (virtual 558MB)
5ee702583ad7 mesosphere/dcos-docker "/sbin/init" 26 minutes ago Up 26 minutes dcos-e2e-default-4baab-master-0 68MB (virtual 625MB)

> docker ps -s

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES SIZE

e6434c3253ca mesosphere/dcos-docker "/sbin/init" 26 minutes ago Up 26 minutes dcos-e2e-default-4baab-public-agent-0 701kB (virtual 558MB)

6674717280a3 mesosphere/dcos-docker "/sbin/init" 26 minutes ago Up 26 minutes dcos-e2e-default-4baab-agent-0 697kB (virtual 558MB)

5ee702583ad7 mesosphere/dcos-docker "/sbin/init" 26 minutes ago Up 26 minutes dcos-e2e-default-4baab-master-0 68MB (virtual 625MB)

There are three nodes running – master, public node and (private) node. Each of it runs Centos 7 image.

> docker exec -it dcos-e2e-default-4baab-master-0 bash
[root@dcos-e2e-default-4baab-master-0 /]# cat /etc/os-release 
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

> docker exec -it dcos-e2e-default-4baab-master-0 bash

[root@dcos-e2e-default-4baab-master-0 /]# cat /etc/os-release

NAME="CentOS Linux"

VERSION="7 (Core)"

ID="centos"

ID_LIKE="rhel fedora"

VERSION_ID="7"

PRETTY_NAME="CentOS Linux 7 (Core)"

ANSI_COLOR="0;31"

CPE_NAME="cpe:/o:centos:centos:7"

HOME_URL="https://www.centos.org/"

BUG_REPORT_URL="https://bugs.centos.org/"

DC/OS software runs on these nodes.And in each node, there runs another docker environment. I believe it’s needed by later service deployment.

[root@dcos-e2e-default-4baab-master-0 /]# docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE

1 2	[root@dcos-e2e-default-4baab-master-0 /]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE

And another useful command is “minidcos docker inspect“. With it, you can find more details of the cluster, such as the cluster id, IP addresses of each node, containers and etc.

yli@yli-Latitude-E6500:~/dcos$ minidcos docker inspect
{
    "Cluster ID": "default",
    "DC/OS Variant": "DCOSVariant.OSS",
    "Nodes": {
        "agents": [
            {
                "docker_container_id": "6674717280a309f24549993d14d4ba5214a7fcbd124c392651689c056040b6a3",
                "docker_container_name": "dcos-e2e-default-4baab-agent-0",
                "e2e_reference": "agent_0",
                "ip_address": "172.17.0.3",
                "ssh_key": "/tmp/e099e46b28b44a6a90646c4812460c20/ssh/id_rsa",
                "ssh_user": "root"
            }
        ],
        "masters": [
            {
                "docker_container_id": "5ee702583ad7a4760b94bdd070996be5fbe7e9ba3495694da53acfdf9a433b75",
                "docker_container_name": "dcos-e2e-default-4baab-master-0",
                "e2e_reference": "master_0",
                "ip_address": "172.17.0.2",
                "ssh_key": "/tmp/e099e46b28b44a6a90646c4812460c20/ssh/id_rsa",
                "ssh_user": "root"
            }
        ],
        "public_agents": [
            {
                "docker_container_id": "e6434c3253ca88bce4788e2af776e664206b231492be3ad13c1aefc86d9a075a",
                "docker_container_name": "dcos-e2e-default-4baab-public-agent-0",
                "e2e_reference": "public_agent_0",
                "ip_address": "172.17.0.4",
                "ssh_key": "/tmp/e099e46b28b44a6a90646c4812460c20/ssh/id_rsa",
                "ssh_user": "root"
            }
        ]
    },
    "Web UI": "http://172.17.0.2"
}

yli@yli-Latitude-E6500:~/dcos$ minidcos docker inspect

{

"Cluster ID": "default",

"DC/OS Variant": "DCOSVariant.OSS",

"Nodes": {

"agents": [

{

"docker_container_id": "6674717280a309f24549993d14d4ba5214a7fcbd124c392651689c056040b6a3",

"docker_container_name": "dcos-e2e-default-4baab-agent-0",

"e2e_reference": "agent_0",

"ip_address": "172.17.0.3",

"ssh_key": "/tmp/e099e46b28b44a6a90646c4812460c20/ssh/id_rsa",

"ssh_user": "root"

}

"masters": [

{

"docker_container_id": "5ee702583ad7a4760b94bdd070996be5fbe7e9ba3495694da53acfdf9a433b75",

"docker_container_name": "dcos-e2e-default-4baab-master-0",

"e2e_reference": "master_0",

"ip_address": "172.17.0.2",

"ssh_key": "/tmp/e099e46b28b44a6a90646c4812460c20/ssh/id_rsa",

"ssh_user": "root"

}

"public_agents": [

{

"docker_container_id": "e6434c3253ca88bce4788e2af776e664206b231492be3ad13c1aefc86d9a075a",

"docker_container_name": "dcos-e2e-default-4baab-public-agent-0",

"e2e_reference": "public_agent_0",

"ip_address": "172.17.0.4",

"ssh_key": "/tmp/e099e46b28b44a6a90646c4812460c20/ssh/id_rsa",

"ssh_user": "root"

}

]

"Web UI": "http://172.17.0.2"

}

Conclusion

By far, the installation and initial launch is finished. Hope it’s helpful to you. And in next chapter, let’s run some services on this platform.