312 lines
12 KiB
Markdown
312 lines
12 KiB
Markdown
# 2. Using Docker
|
|
|
|
Chapter 1 introduced what docker is, here introduce **what docker does**
|
|
|
|
## The Docker flow: Images to containers
|
|
|
|
Learning Outcome: Create container from image
|
|
|
|
In docker, everything begin with a image.
|
|
|
|
* **Docker Image**: an immutable (read only) file that contains source code, libraries, dependencies, tools and all other files just needed for an application to run as a linux app.
|
|
* Also called **snapshot**
|
|
* Represent as an app and its virtual env at a specific point in time
|
|
* Act as a template for starting a container
|
|
* A **Container** is created from a image, via adding a writable layer (i.e. **container layer**) on top of the immutable images.
|
|
* A image can be used to create unlimited number of container, each are isolated from each other
|
|
* Modification in one container does not affect another
|
|
|
|
Command Lines:
|
|
* `docker run` cmd start a container from a image
|
|
* `-it` run as a interactive terminal
|
|
* e.g. `docker run -it ubuntu:latest bash`
|
|
* `bash` is the COMMAND that command given to the container to do after creation
|
|
* `docker ps` cmd list all running containers
|
|
* `exit` in container will terminate container
|
|
|
|
## The Docker flow: Containers to images
|
|
|
|
Learning Outcome: Create image based on container (after modifying container)
|
|
|
|
After exit a container, the container is stopped but still existing
|
|
|
|
Command Lines:
|
|
* `docker ps -a` list all containers, including stopped ones
|
|
* `docker ps -l` list the last container
|
|
These cmd is good for inspecting container conditions
|
|
|
|
Stopped container can be used to create images
|
|
|
|
Command Lines:
|
|
* `docker commit CONTAINER_ID` create a image, return a sha256 string as name of image
|
|
* `docker tag SHA256_IMAGE NEW_IMAGE_NAME` tag the image
|
|
|
|
Previous 2 cmd can be merged as one:
|
|
* `docker commit CONTAINER_NAME/ID NEW_IMAGE_NAME ` create an image and tag it
|
|
|
|
## Run processors in containers
|
|
|
|
Learning Outcome: how to run things in docker
|
|
|
|
`docker run`
|
|
* start a container
|
|
* give container a process, which will be the main process
|
|
* The container stops when the process stops
|
|
* Container has names. If not given, they will initiate one
|
|
|
|
Command Line:
|
|
* `docker run --rm -ti ubuntu:latest sleep 5`
|
|
* `--rm` tell docker to remove the container after exit
|
|
* `docker run -it ubuntu bash -c "sleep 3; echo all done"`
|
|
* `bash -c "sleep 3; echo all done"` will sleep for 3 seconds and print all done on terminal after that
|
|
* `docker run -d ...` will detach the container in the background
|
|
|
|
`docker attach`
|
|
* `docker attach [CONTAINTER_NAME]` attach the container
|
|
* `ctl+p` detach from container
|
|
* `ctl+q` attach into most recent container
|
|
|
|
`docker exec`
|
|
* Starts another process in an existing container
|
|
* Great for debugging and DB administration
|
|
* Can't add ports, volumes, and so on (only through `docker run`)
|
|
|
|
Command Line:
|
|
* `docker exec -ti CONTAINER_NAME bash` run another bash shell in the container
|
|
|
|
## Manage containers
|
|
|
|
### Looking at Container Output
|
|
`docker logs`
|
|
* Keep the output of containers, until the container is destroyed
|
|
* View with `docker logs CONTAINER_NAME`
|
|
* Don't let the output get too large
|
|
|
|
e.g.
|
|
* `docker run --name CONTAINER_NAME -d ubuntu bash c "lose /etc/password"` failed as `lose` is typo of `less`, docker logs can be used to inspect the log
|
|
|
|
### Stopping and Removing Container
|
|
`docker kill CONTAINER_NAME`
|
|
* Killing containers (without entering it interactively and enter `exit` cmd)
|
|
|
|
`docker rm CONTAINER_NAME/CONTAINER_ID`
|
|
* Stopped container still existing after killing it
|
|
* Remove containers
|
|
|
|
### Resource Constraints
|
|
|
|
Feature of docker: fix limit of resource
|
|
|
|
* Memory limits
|
|
* `docker run --memory maximum-allowed-memroy image-name command`
|
|
* CPU (time) limits
|
|
* `docker run --cpu-shares` relative to other containers. Enforce relative % of cpu time a container can have
|
|
* `docker run --cpu-quota` to set hard limit of cpu time
|
|
* **Orchestration** system describe later
|
|
* Generally requires resource limiting
|
|
|
|
Tips in practice:
|
|
* Don't let containers fetch dependencies when they start
|
|
* e.g. If upstream made change (rm library etc), it affect here
|
|
* Hence, make image include dependencies
|
|
* Don't leave important things in **unnamed stopped containers**. Maybe you will prune it accidentally
|
|
|
|
## Exposing ports
|
|
|
|
### Container Networking
|
|
|
|
* Programs in containers are isolated from Internet by default
|
|
* Containers can be connected using "private" networks, which still separated from Internet
|
|
* Host machine can expose ports to let Internet connections into container
|
|
|
|
So, private networks + expose ports makes connection
|
|
|
|
### Exposing a Specific Port
|
|
* How to expose specific port: Explicitly specifies the port inside container and outside
|
|
* We can expose as many ports as you want
|
|
* Requires coordination btw containers
|
|
* Makes it easy to find exposed ports
|
|
|
|
e.g.
|
|
1. Open a terminal, create a simple server in docker:
|
|
1. `docker run --rm -ti -p 45678:45678 -p 45679:45679 --name echo-server ubuntu:14.04 bash`
|
|
1. this docker expose docker's port 45678 to localhost's port 45678, and docker's port 45679 to local host's port 45679
|
|
2. In the docker, create a simple echo server by `nc -lp 45678 | nc -lp 45679`. It listen on port 45678 and send data to port 45679 via linux pipeline
|
|
2. Open 2nd terminal, enter `nc localhost 45678` to start a sender
|
|
3. Open 3rd terminal, enter `nc localhost 45679` to start a listener
|
|
4. Enter anything in sender terminal, data will be echoed in listener terminal
|
|
|
|
### Exposing Ports Dynamically
|
|
|
|
If having multiple dockers running.
|
|
* The port inside the container is fixed, as given
|
|
* Port on the host can be assigned by docker using unused port
|
|
* It allows many containers running programs with fixed internal ports
|
|
* Often used with a service discovery program (e.g. k8t)
|
|
|
|
`docker port CONTAINTER_NAME`:
|
|
* show port mapping
|
|
|
|
### Exposing UDP Ports
|
|
Docker works with other protocol (not only tcp) like UDP
|
|
* `docker run -p outside-port:inside-port/protocol`
|
|
* e.g. `docker run -p 1234:1234/udp`
|
|
|
|
`nc -ulp 45678` can start nc program using udp protocol
|
|
|
|
## Container networking
|
|
|
|
### Connecting directly between Containers
|
|
|
|
Container connection method 1: create a network that from outside of machine (i.e. expose to internet) to container
|
|
|
|
Connection established by:
|
|
1. Container reach out to the host level (connect to localhost)
|
|
2. Then turning back to another container
|
|
|
|
Container connection method 2; create a private network that only link between containers
|
|
|
|
|
|
`docker network ls`
|
|
* show established networks
|
|
* `bridge` specify in which, containers without network preference
|
|
* `host` are network in which, containers does not have network isolation (connect directly with host's network interface)
|
|
* `none` for containers with no networking
|
|
|
|
`docker network create NETWORK_NAME`
|
|
* Create a new network and return a shasum
|
|
|
|
`docker run -rm -ti --net NETWORK_NAME --name CONTAINER_NAME IMAGE:TAG cmd`
|
|
* Connect a container to a network
|
|
|
|
#### Exercise 1: 2 containers on a same network connecting each other
|
|
|
|
1. Create a network named learning `docker network create learning`
|
|
2. Create two containers connecting learning network that can ping each other
|
|
1. Create a container that connected to learning network `docker run --rm -ti --network learning --name catserver ubuntu:14.04 bash`
|
|
2. In catserver, we can ping the container itself `ping catserver`
|
|
3. Create 2nd container "dogserver" that also connected to learning network
|
|
4. In catserver, ping dogserver container `ping dogserver`
|
|
5. In dogserver, we can ping catserver container `ping catserver` as well
|
|
3. Use `nc` to transfer information
|
|
1. In catserver, `nc dogserver 1234` to connect to port 1234 of dogserver
|
|
2. In dogserver, `nc -lp 1234` to listen incoming connection on port 1234
|
|
|
|
#### Exercise 2: Create multiple network to manage containers separately
|
|
|
|
1. Create a catonly network `docker network create catsonly`
|
|
2. Attach catserver to catsonly network `docker network connect catsonly catserver`
|
|
3. Create a container connect to catnet `docker run --rm -ti --net catsonly --name bobcatserver ubuntu:14.04 bash`
|
|
4. Then, catserver and bobcatserver can communicate
|
|
5. Because catserver is connected to learning network as well, so catserver can still communicate with dogserver
|
|
|
|
90% usage of docker network is up to this point, there are more options available for further research
|
|
|
|
## Legacy linking
|
|
|
|
Old way of connecting containers, before network. Should be avoided, but still remain in some products.
|
|
|
|
* Links all ports, just in only one way
|
|
* Secret environment variables are shared only one way
|
|
* e.g. env var like db password, are visible to any machine that later link to it
|
|
* Reverse is not true: server cannot see env var on the machines that link
|
|
* Hence, require depends on startup order
|
|
* Restart only sometimes break the links
|
|
|
|
#### Exercise 1:
|
|
|
|
* `docker run --rm -it -e SECRET=theinternetlovescat --name catserver ubuntu:14.04 bash`
|
|
* `docker run --rm -it --link catserver --name dogserver ubuntu:14.04 bash`
|
|
* in catserver `nc -lp 1234`, in dog server `nc catserver 1234`. So dog server is linked to catserver
|
|
* reverse way is not true, catserver cannot ping dogserver
|
|
* env var `SECRET=tehinternetlovescat` is also copied to dogserver as `CATSERVER_ENV_SECRET=theinternetlovescat`. But catserver cannot see env var from dogserver
|
|
|
|
## Images
|
|
|
|
Learning Outcome: learn how to manage images
|
|
|
|
### Listing Images
|
|
|
|
`docker images`
|
|
* lists downloaded images
|
|
* SIZE of images means size of that image, but image layers maybe shared from one to another, hence real footprint is smaller
|
|
|
|
### Tagging Images
|
|
|
|
* Tagging gives images names
|
|
* `docker commit CONATINER_ID NAME:TAG` tags images for you, if not specify tag, the image is tagged as latest
|
|
* Naming structure: `registry.example.com:port/organization/image-name:version-tag`
|
|
* You can leave out parts that don't need
|
|
* `Organization/image-name` is often enough
|
|
|
|
### Getting images
|
|
|
|
* `docker pull`
|
|
* Run automatically by `docker run`
|
|
* Useful for offline work
|
|
* `docker push` opposite
|
|
|
|
### Cleaning Up
|
|
|
|
* Images can accumulate quickly
|
|
* Remove image:
|
|
* `docker rmi IMAGE_NAME:TAG`
|
|
* `docker rmi IMAGE_ID`
|
|
* Write a shell script to do recursive job
|
|
|
|
## Volumes
|
|
|
|
Learning Outcome: Sharing data between container and host
|
|
|
|
**Volumes** of a container:
|
|
* is a virtual "discs" to store and share data
|
|
* 2 main varieties
|
|
* **Persistent**: data are permanently stored
|
|
* **Ephemeral**: When no container using them, they are deleted
|
|
* Volumes are not part of images
|
|
|
|
### Sharing Data with Host (Persistent)
|
|
|
|
* `docker run -v VOLUME_HOST_PATH:VOLUME_CONTAINER_PATH ...` share the a directory in host with container (Like "Shared folder" with the host). Actions in container will affect host files
|
|
* `docker run -v VOLUME_HOST_FILE_PATH:VOLUME_CONTAINER_FILE_PATH ...` can sharing a "single file" into a container
|
|
* Note a file must exit before start the container, otherwise it's assumed a directory
|
|
|
|
### Sharing Data between Containers (Can be Ephemeral)
|
|
|
|
`docker run -v volume-name`
|
|
* Create a ephemeral volume
|
|
|
|
`docker run --volumes-from`
|
|
* Shared "dics" that exist only as long as they are being used
|
|
* Can be shared between containers
|
|
|
|
#### Exercise: sharing data btw containers
|
|
|
|
1. Create a container with shared volume: `docker run -it -v /shared-data ubuntu:14.04 bash`
|
|
2. Create file within the shared volume: `echo hello > /shared-data/data-file`
|
|
3. Create 2nd container that also connect the shared volume: `docker run -it --volumes-from CONTAINER_NAME ubuntu bash`
|
|
4. We can find the just created file: `echo /shared-data/data-file`
|
|
5. The created directly will be inherited by 2nd container, even if 1st container exit, and inherit further
|
|
6. But when all containers that shared that volume exit, this volume is gone (i.e. **Ephemeral**)
|
|
|
|
## Docker registries
|
|
|
|
* Docker images are retrieved from & published to registries
|
|
* Registry is a program
|
|
* Registries manage and distribute images
|
|
* Docker (the company) has DockerHub for free usage
|
|
* We can run private registry
|
|
|
|
### Finding Images
|
|
|
|
`docker search`
|
|
* search images
|
|
* Same as go to DockerHub website.
|
|
|
|
`docker login`
|
|
* log into DockerHub, so we can push (publish) images
|
|
|
|
Tips:
|
|
* Don't push images with password within it
|
|
* Clean up images regularly
|
|
* Be aware images, make sure they are official |