Setting Up Docker Overlay Network

When I set forth with my journey of containerization with docker, I have gone through a misconception that Overlay networking in docker can’t be set up without any orchestrator like Docker swarm, Kubernetes. But after spending some time with containers I realized that I was wrong, Orchestrators leverage the functionality of overlay networking but it is not true that we cannot use overlay networks without any swarm or Kubernetes.

So in this blog, we will we talking about a bit on overlay networking and how we can set up it by using Docker and any service discovery tool like Consul.
But if you are new to Docker networking, you can read our blog on types and uses of Docker networking.

Overlay Networking

Overlay networking is nothing different from asking an address to a person. For example, if you ask an address or a location and the guy tells you that the address exists in House №14, B-block. Now the question is that there are multiple B-blocks, how will you determine which House №14, B-block you have to go, so you ask for area pin-code, location and on the basis of that you can easily identify the location.

A similar analogy is in overlay networking, it allows you to establish connections between different hosts that are hidden from each other.
If there are two hosts and each one runs a docker, the Overlay network will help to create a subnet at the top of these two hosts and each container connected to this Overlay network will be able to communicate with other containers.

Now to know these things like which container is running on what host, we need a central database that keeps this kind of information and this is where tools like consul, etcd, and zookeeper come into the picture.

Setting up Network using Docker and Consul

So for the central database and service discovery part, we will use consul as a database and for containers management, of course, we will use docker. So our architecture will look like this:-

So let’s get started.

First, we need 3 machines. 2 for docker host nodes and one for consul database. And all machines docker needs to be installed.

On node provisioned for consul, we can setup consul by using this command.

sudo docker run -d -p 8500:8500 -h consul \ 
--name consul progrium/consul -server -bootstrap 

Once the consul nodes are provisioned, we need to define that docker will store its meta and network information in consul, and the interface name which it will use to create an overlay network bridge. So we need to edit the existing systemd service of Docker.

/lib/systemd/system/docker.service

And then we have to add this entry in the ExecStart section.

ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --cluster-store=consul://<consul_server_ip>:8500 --cluster-advertise=<your_primary_ethernet_like_eth0>:2376
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --cluster-store=consul://192.168.1.10:8500 --cluster-advertise=eth0:2376

Once these changes are done on both machines, we can reload the system daemon and restart the service.

systemctl reload-daemon 
systemctl restart docker

After this our communication bridge and database are setup, now the only thing that is remaining is to create an overlay network.

docker network create --driver overlay \ 
--subnet 192.168.0.0/24 overlay_private

Once the network is created, we are all set to test the changes, we can simply test these settings by creating two dummy container on these two hosts.

docker run -it --rm --net overlay_private debian bash

So while creating the container we are providing the overlay network “overlay_private” which we have created earlier.

And through these containers, we can try to ping their Overlay Subnet’s IP which is provided by the docker network.

Conclusion

Since this is a POC to understand the docker networking a bit more easily so this can’t be used in production directly, I strongly recommend using orchestrators like Kubernetes, Openshift, and ECS to run the production workloads because they do a lot more than networking part.

Some of the popular overlay network drivers are:- Flannel, Project Calico, Weave Net. These tools can be used with the orchestrator to set up and manage your networking.

If you guys want to simulate this, here is a problem statement for you guys.

“Create two docker hosts one for webserver and application and another one for database and host this application using an overlay network. So web servers like Nginx and application can run on one host and DB can run on the other host and all the communication should happen on overlay private network.”

Thanks for reading, I’d really appreciate any and all feedback, please leave your comment below if you guys have any feedback.

Cheers till next time!!

Originally published at http://blog.opstree.com on July 7, 2020.