learning-docker/notes/2_using_docker.md

# 2. Using Docker

Chapter 1 introduced what docker is, here introduce **what docker does**

## The Docker flow: Images to containers

Learning Outcome: Create container from image

In docker, everything begin with a image.

* **Docker Image**: an immutable (read only) file that contains source code, libraries, dependencies, tools and all other files just needed for an application to run as a linux app.
  * Also called **snapshot**
  * Represent as an app and its virtual env at a specific point in time
  * Act as a template for starting a container
* A **Container** is created from a image, via adding a writable layer (i.e. **container layer**) on top of the immutable images.
* A image can be used to create unlimited number of container, each are isolated from each other
  * Modification in one container does not affect another

Command Lines: 
* `docker run` cmd start a container from a image
  * `-it` run as a interactive terminal
  * e.g. `docker run -it ubuntu:latest bash`
    * `bash` is the COMMAND that command given to the container to do after creation
* `docker ps` cmd list all running containers
* `exit` in container will terminate container

## The Docker flow: Containers to images

Learning Outcome: Create image based on container (after modifying container)

After exit a container, the container is stopped but still existing

Command Lines:
* `docker ps -a` list all containers, including stopped ones
* `docker ps -l` list the last container
These cmd is good for inspecting container conditions

Stopped container can be used to create images

Command Lines:
* `docker commit CONTAINER_ID` create a image, return a sha256 string as name of image
* `docker tag SHA256_IMAGE NEW_IMAGE_NAME` tag the image

Previous 2 cmd can be merged as one:
* `docker commit CONTAINER_NAME/ID NEW_IMAGE_NAME ` create an image and tag it

## Run processors in containers

Learning Outcome: how to run things in docker

`docker run`
* start a container
* give container a process, which will be the main process
* The container stops when the process stops
* Container has names. If not given, they will initiate one

Command Line:
* `docker run --rm -ti ubuntu:latest sleep 5`
  * `--rm` tell docker to remove the container after exit
* `docker run -it ubuntu bash -c "sleep 3; echo all done"`
  * `bash -c "sleep 3; echo all done"` will sleep for 3 seconds and print all done on terminal after that
* `docker run -d ...` will detach the container in the background

`docker attach`
* `docker attach [CONTAINTER_NAME]` attach the container
* `ctl+p` detach from container
* `ctl+q` attach into most recent container

`docker exec`
* Starts another process in an existing container
* Great for debugging and DB administration
* Can't add ports, volumes, and so on (only through `docker run`)

Command Line:
* `docker exec -ti CONTAINER_NAME bash` run another bash shell in the container

## Manage containers

### Looking at Container Output
`docker logs`
* Keep the output of containers, until the container is destroyed
* View with `docker logs CONTAINER_NAME`
* Don't let the output get too large

e.g.
* `docker run --name CONTAINER_NAME -d ubuntu bash c "lose /etc/password"` failed as `lose` is typo of `less`, docker logs can be used to inspect the log

### Stopping and Removing Container
`docker kill CONTAINER_NAME`
* Killing containers (without entering it interactively and enter `exit` cmd)

`docker rm CONTAINER_NAME/CONTAINER_ID`
* Stopped container still existing after killing it
* Remove containers

### Resource Constraints

Feature of docker: fix limit of resource

* Memory limits
  * `docker run --memory maximum-allowed-memroy image-name command`
* CPU (time) limits
  * `docker run --cpu-shares` relative to other containers. Enforce relative % of cpu time a container can have
  * `docker run --cpu-quota` to set hard limit of cpu time
* **Orchestration** system describe later
  * Generally requires resource limiting

Tips in practice:
* Don't let containers fetch dependencies when they start
  * e.g. If upstream made change (rm library etc), it affect here
  * Hence, make image include dependencies
* Don't leave important things in **unnamed stopped containers**. Maybe you will prune it accidentally

## Exposing ports

### Container Networking

* Programs in containers are isolated from Internet by default
* Containers can be connected using "private" networks, which still separated from Internet
* Host machine can expose ports to let Internet connections into container

So, private networks + expose ports makes connection

### Exposing a Specific Port
* How to expose specific port: Explicitly specifies the port inside container and outside
* We can expose as many ports as you want
* Requires coordination btw containers
* Makes it easy to find exposed ports

e.g.
1. Open a terminal, create a simple server in docker:
   1. `docker run --rm -ti -p 45678:45678 -p 45679:45679 --name echo-server ubuntu:14.04 bash`
      1. this docker expose docker's port 45678 to localhost's port 45678, and docker's port 45679 to local host's port 45679 
   2. In the docker, create a simple echo server by `nc -lp 45678 | nc -lp 45679`. It listen on port 45678 and send data to port 45679 via linux pipeline
2. Open 2nd terminal, enter `nc localhost 45678` to start a sender
3. Open 3rd terminal, enter `nc localhost 45679` to start a listener
4. Enter anything in sender terminal, data will be echoed in listener terminal

### Exposing Ports Dynamically

If having multiple dockers running.
* The port inside the container is fixed, as given
* Port on the host can be assigned by docker using unused port
* It allows many containers running programs with fixed internal ports
* Often used with a service discovery program (e.g. k8t)

`docker port CONTAINTER_NAME`:
* show port mapping

### Exposing UDP Ports
Docker works with other protocol (not only tcp) like UDP
* `docker run -p outside-port:inside-port/protocol` 
* e.g. `docker run -p 1234:1234/udp`

`nc -ulp 45678` can start nc program using udp protocol

## Container networking

### Connecting directly between Containers

Container connection method 1: create a network that from outside of machine (i.e. expose to internet) to container
![Connection between container 1](imgs/docker2.png)
Connection established by:
1. Container reach out to the host level (connect to localhost)
2. Then turning back to another container

Container connection method 2; create a private network that only link between containers
![Connection between container 2](imgs/docker3.png)

`docker network ls`
* show established networks
* `bridge` specify in which, containers without network preference
* `host` are network in which, containers does not have network isolation (connect directly with host's network interface)
* `none` for containers with no networking

`docker network create NETWORK_NAME`
* Create a new network and return a shasum

`docker run -rm -ti --net NETWORK_NAME --name CONTAINER_NAME IMAGE:TAG cmd`
* Connect a container to a network

#### Exercise 1: 2 containers on a same network connecting each other

1. Create a network named learning `docker network create learning`
2. Create two containers connecting learning network that can ping each other
   1. Create a container that connected to learning network `docker run --rm -ti --network learning --name catserver ubuntu:14.04 bash`
   2. In catserver, we can ping the container itself `ping catserver`
   3. Create 2nd container "dogserver" that also connected to learning network
   4. In catserver, ping dogserver container `ping dogserver`
   5. In dogserver, we can ping catserver container `ping catserver` as well
3. Use `nc` to transfer information
   1. In catserver, `nc dogserver 1234` to connect to port 1234 of dogserver
   2. In dogserver, `nc -lp 1234` to listen incoming connection on port 1234

#### Exercise 2: Create multiple network to manage containers separately

1. Create a catonly network `docker network create catsonly`
2. Attach catserver to catsonly network `docker network connect catsonly catserver`
3. Create a container connect to catnet `docker run --rm -ti --net catsonly --name bobcatserver ubuntu:14.04 bash`
4. Then, catserver and bobcatserver can communicate
5. Because catserver is connected to learning network as well, so catserver can still communicate with dogserver

90% usage of docker network is up to this point, there are more options available for further research

## Legacy linking

Old way of connecting containers, before network. Should be avoided, but still remain in some products.

* Links all ports, just in only one way
* Secret environment variables are shared only one way 
  * e.g. env var like db password, are visible to any machine that later link to it
  * Reverse is not true: server cannot see env var on the machines that link
* Hence, require depends on startup order
* Restart only sometimes break the links

#### Exercise 1:

* `docker run --rm -it -e SECRET=theinternetlovescat --name catserver ubuntu:14.04 bash`
* `docker run --rm -it --link catserver --name dogserver ubuntu:14.04 bash`
* in catserver `nc -lp 1234`, in dog server `nc catserver 1234`. So dog server is linked to catserver
* reverse way is not true, catserver cannot ping dogserver
* env var `SECRET=tehinternetlovescat` is also copied to dogserver as `CATSERVER_ENV_SECRET=theinternetlovescat`. But catserver cannot see env var from dogserver

## Images

Learning Outcome: learn how to manage images

### Listing Images

`docker images`
* lists downloaded images
* SIZE of images means size of that image, but image layers maybe shared from one to another, hence real footprint is smaller

### Tagging Images

* Tagging gives images names
* `docker commit CONATINER_ID NAME:TAG` tags images for you, if not specify tag, the image is tagged as latest
* Naming structure: `registry.example.com:port/organization/image-name:version-tag`
  * You can leave out parts that don't need
  * `Organization/image-name` is often enough

### Getting images

* `docker pull`
  * Run automatically by `docker run`
  * Useful for offline work
* `docker push` opposite

### Cleaning Up

* Images can accumulate quickly
* Remove image:
  * `docker rmi IMAGE_NAME:TAG`
  * `docker rmi IMAGE_ID`
* Write a shell script to do recursive job

## Volumes

Learning Outcome: Sharing data between container and host

**Volumes** of a container:
* is a virtual "discs" to store and share data
* 2 main varieties
  * **Persistent**: data are permanently stored
  * **Ephemeral**: When no container using them, they are deleted
* Volumes are not part of images

### Sharing Data with Host (Persistent)

* `docker run -v VOLUME_HOST_PATH:VOLUME_CONTAINER_PATH ...` share the a directory in host with container (Like "Shared folder" with the host). Actions in container will affect host files
* `docker run -v VOLUME_HOST_FILE_PATH:VOLUME_CONTAINER_FILE_PATH ...` can sharing a "single file" into a container
  * Note a file must exit before start the container, otherwise it's assumed a directory

### Sharing Data between Containers (Can be Ephemeral)

`docker run -v volume-name`
* Create a ephemeral volume

`docker run --volumes-from`
* Shared "dics" that exist only as long as they are being used
* Can be shared between containers

#### Exercise: sharing data btw containers

1. Create a container with shared volume: `docker run -it -v /shared-data ubuntu:14.04 bash`
2. Create file within the shared volume: `echo hello > /shared-data/data-file`
3. Create 2nd container that also connect the shared volume: `docker run -it --volumes-from CONTAINER_NAME ubuntu bash`
4. We can find the just created file: `echo /shared-data/data-file`
5. The created directly will be inherited by 2nd container, even if 1st container exit, and inherit further
6. But when all containers that shared that volume exit, this volume is gone (i.e. **Ephemeral**)

## Docker registries

* Docker images are retrieved from & published to registries
* Registry is a program
* Registries manage and distribute images
* Docker (the company) has DockerHub for free usage
* We can run private registry

### Finding Images

`docker search`
* search images
* Same as go to DockerHub website.

`docker login` 
* log into DockerHub, so we can push (publish) images

Tips:
* Don't push images with password within it
* Clean up images regularly
* Be aware images, make sure they are official