learning-docker/notes/4_under_the_hood.md

# 4. Under the Hood

## Docker the program

### What Kernels Do

* software core of computer
* Run directly on hardware, had many jobs
  * Respond to messages from the hardware
  * Start and schedule programs
  * Control and organize storage
  * Pass messages between programs
  * Allocate resources, memory, CPU, network, etc
  * **Create containers by Docker configuring the kernel**

### What Docker Does

* Program written in Go
* Manages kernel features
  * Docker uses "cgroups" to contain processes, so each container (with its process) is isolated
  * Docker uses "namespaces" (feature of Linux Kernel) to contain networks
  * USes "copy-on-write" filesystems to build images
* Above features has been used by industry for years before Docker.

### What Docker Really Does

* Makes scripting distributed systems "easy"

### The Docker Control Socket

* Docker is two programs: a **client** & a **server**
  * They communicate via a socket
* The server receives commands over a socket (either over a network or through a "file")
  * When client & server are on same computer, a special "file" `socket` is created for communication
* The client can even run inside a docker container itself

location of docker socket file on linux: `/var/run/docker.sock`

We can control the docker server using docker client inside a container: `docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock docker sh`

## Networking in Brief

TODO: Need further research this topic

* **Ethernet** layer: moves "frames" on a wire (or wireless)
* **IP** layer: moves packets on a local network
* **Routing**: forwards packets between networks
* **Ports**: address particular programs on a computer

### Bridging

* Docker uses bridges to create virtual networks in computer
* They are like software network switches
* They control Ethernet layer
* You can turn off this protection with `docker run --net=host options image-name command` 
  * `--net=host` gives container direct access to network
  * Useful for learning/debugging

### Routing

* Creates "firewall" rules to move packets between networks
* NAT (Network Address Translation)
* Change the source address on the way out
* Change the destination address on the way back

`sudo iptables -n -L -t nat`
* `-n` for 

`--previlige=true` will give docker all access

* Exposing a port = "port forwarding" in network layer

### Namespaces

* Allow process to be attached to private network segments
* Private networks are bridged into a shared network with the rest of the containers
* Containers get their own copy of the networking stack

??? Need learn this section (Container Networking in linkedin) specifically

## Processes and cgroups

### Primer on Linux Processes

* Processes come from other processes (parent-child relationship)
* When a child process exits, it returns an exit code to its parent
* **Process Zero**  (`init`) is the process that starts the rest
* In Docker, a container starts with an init process and vanishes when that process exit
  * By knowing the init process of the container, we can kill the container

`docker inspect` can be ...???? 

### Resource Limiting

Set limit for
* Scheduling CPU time
* Memory allocation limits
* Inherited limitations and quotas (summed total cannot exceed the limit)

## Storage

Discuss how container storage work

### Unix Storage in Brief
* Actual storage devices are binary info on hardware, managed by kernel
* Logical storage devices (logical groups, like partition drive into different parts)
* Filesystems
* FUSE filesystems and network filesystems

### The Secret of Dockers: COWs

Copy on Write layer: Container see the combined result of real file and a its own file system layer. When writing to file, instead of directly write on file, we write on the file system layer

???? Need more understanding

### Moving Cows

* The contents of layers are moved between containers in gzip files
* Containers are independent of the storage engine
* Any container can be loaded (almost) everywhere
* It's possible to run out of layers on some of the storage engines

### Volumes and Bind Mounting

* The Linux VFS (Virtual File System)
* Mounting devices on the VFS

????

TODO: relearn these content (networking, process, storage of docker, etc)