Managing multiple Micro-services in Development using Docker Compose

5 min readJul 4, 2020

Part 1: The Problem

Any growing tech project, is usually accompanied by an ever growing number of micro services, and with it a growing number of problems.

They each have their own database layers that they connect to (Redis, Mongo, Mysql etc.). With different micro-services connecting to different db versions.

When a new developer joins the team, he/she wastes a lot of time installing all the dependencies, and hates it.

And lastly, we want to keep our dev stack as close to production stack as possible.

Consider the Following Project Architecture

Service A is dependent on

Redis
MySql
Service B
Service C

And So on..(You get my point)

Now Imagine, starting all of the above, with proper language versions and database versions. Starting all the services above, is definitely not joy.

There should be an easier way to start our ecosystem of services, with one command/click, should be easy to setup, should be fast and finally should be fun.

Here comes Docker and Docker Compose to help you with it. To understand more of Docker and Docker Containers go here https://www.docker.com/.

In an Ideal Scenario if I have to test functioning of Service A, I’ll use unit and integrations tests and I’ll mock all the API calls to Service B and Service C.

Part 2 Using Docker Compose

Docker compose exactly lets you do that. It gives you a way to write the above configuration in a yaml file. And using one command everything boots up like magic.

One file to run them all, one file to bind them under one network. My Precious!!!

As soon I hit docker-compose up.

docker-compose up

Everything starts running

Mysql DB for service A
Mysql for Service C
Redis for Service A
Mongo Db for Service B.
Service B
Service C
Service A

High Definition Youtube Link: https://www.youtube.com/watch?v=HJ0D7Ns5tBU

TL;DR go to this repo, and experience it for yourself. (Only Pre Req — Docker should be installed)

Repo: https://github.com/tushartuteja/multiple-microservices-docker-compose

Let’s go to localhost:8080, localhost:8081, and localhost:8082 to see what happens.

Part 2.2 Understanding Service A and it Dependencies.

Service A is dependent on four things.

If you notice lines, 11–13 and 15–17, Service A is calling Service B and Service C.
Notice lines, 19–29, Service A is calling Redis.
Notice Lines 31–48, Service A is calling mysql.

The responses for the above are logged to console. (See Image Below)

We can see that first service A called Redis, it worked and printed “Reply Ok”, Then it called Mysql and printed “The solution is: 2”.

Then finally Service C and Service B were called, and their response was printed.

Part 3: Understanding docker-compose.yaml

One file to run them all, one file to bind them under one network. My Precious!!!

A. Version

This is the version of Docker-compose file format and this version also tells which docker engine is compatible for the file. More on this here.

B. Services

This tells docker compose that which all different services are to be run. We have listed out 7 such services. 3 Web Services and 4 Databases.

C. Build

This tells which dockerfile to use to form our image. We are using this in our three web services. We are using pre defined images for all DBs from Docker Hub.

D. Dockerfile

Let’s look at Dockerfile of Service A.

This tells the docker, to start with an alpine image of nodejs. After that create a directory myapp, copy package.json in that directory, run npm install, expose 3000 port and finally run our dev-start.sh file.

E. ports

Let’s notice for Service A

ports:
  - "8080:3000"

Ports let you expose multiple ports from your container to a parent machine. The above line says, expose port 3000 at port 8080 of the parent machine.

Notice how we navigated to localhost:8080 in the above image.

You may expose multiple ports, it takes an array as input.

F. Volumes

volumes:
  - ./service_a:/myapp

We are attaching service_a folder of our code as a volume to our container, which would be mounted at /myapp.

This way if we change any code in our service_a directory, that is instantly detected in our container and we don’t need to restart our container. They are always in sync. Neat! (I Know)

G. depends_upon

depends_on:
  - service_a_db
  - service_a_redis
  - service_b
  - service_c

This tells docker-compose that before you can start Service A, you need to start service_a_db, service_a_redis, service_b and service_c. As Service A depends on it.

It’s let you do one more thing. Suppose you have to just start Service B.

You can just run the following command.

docker-compose up service_b

This would just run Service B and its dependencies and nothing else.

H. Environment Variables

Let’s notice environment variables part of mysql

environment:
  MYSQL_ROOT_PASSWORD: service_a
  MYSQL_DATABASE: "db"
  MYSQL_USER: "user"
  MYSQL_PASSWORD: "pass"

The above would form environment variables for our mysql container. Notice how we used the above credentials to login to mysql in service A.

I. Network

When all of the containers run, they become a part of their own virtual network. This what makes it possible for Service A to connect to mysql with a host name of service_a_db.

The following are all hosts for Service A.

service_a_db
service_b
service_c

But None can be accessed from you machine. Try visiting these hosts from your machine.

Part 4: Conclusion.

As a tech lead, this was my de-facto way of managing projects. Whenever a developer was onboarded, he ran only one command and boom! everything was set up on day one. From day one, a new developer was able to code right away.

I have found this way much more efficient, as it also removes any dependencies in “production environment”, since we used docker to run our app in production. This brought a seamless experience between us and our dev-ops team.

If you are using rails, read this https://medium.com/better-programming/setting-up-rails-with-postgres-using-docker-426c853e8590

More on Docker Here https://www.docker.com/

More on Docker Compose Here https://docs.docker.com/compose/

Note: It takes time the first time you run it, as it builds all those docker images, downloads any dependencies. But after that it runs like a breeze.(Look at the video above)