Build Docker from Scratch in Go — Part 3: Cgroups, Networking, and a CLI

In Part 1, you isolated processes with Linux namespaces. In Part 2, you added filesystem isolation with pivot_root and OverlayFS. Now it is time to finish the container runtime.

In this final part, you will:

Add cgroups to limit memory and CPU
Set up virtual ethernet pairs for container networking
Build a CLI with Cobra
Run the final container and compare it with Docker

Cgroups: Resource Limits

Namespaces control what a process can see. Cgroups control what a process can use. Without cgroups, a container could use all the memory on the host and crash everything.

Cgroups (control groups) are a Linux kernel feature that limits and tracks resource usage. Modern Linux systems use cgroups v2, which has a unified hierarchy. All resource controllers are in one tree under /sys/fs/cgroup/.

How Cgroups v2 Works

Cgroups v2 uses a filesystem interface. You create a directory, write limits to files, and add process IDs. That is it.

/sys/fs/cgroup/
├── cgroup.controllers       # available controllers
├── cgroup.subtree_control   # enabled controllers for children
└── minicontainer/           # our cgroup
    ├── memory.max           # memory limit
    ├── cpu.max              # CPU limit
    ├── cgroup.procs         # PIDs in this cgroup
    └── pids.max             # max number of processes

Implementing Cgroups in Go

Add helper functions to parse CLI flags and create a cgroup:

// parseMemory converts a human-readable memory string to bytes.
// Examples: "256m" -> 268435456, "1g" -> 1073741824
func parseMemory(s string) (int64, error) {
	s = strings.TrimSpace(strings.ToLower(s))
	if len(s) == 0 {
		return 0, fmt.Errorf("empty memory value")
	}

	// Check if the last character is a unit suffix
	suffix := s[len(s)-1]
	switch suffix {
	case 'k', 'm', 'g':
		numStr := s[:len(s)-1]
		num, err := strconv.ParseInt(numStr, 10, 64)
		if err != nil {
			return 0, fmt.Errorf("invalid memory value: %s", s)
		}
		switch suffix {
		case 'k':
			return num * 1024, nil
		case 'm':
			return num * 1024 * 1024, nil
		case 'g':
			return num * 1024 * 1024 * 1024, nil
		}
	}

	// No suffix — treat the whole string as bytes
	num, err := strconv.ParseInt(s, 10, 64)
	if err != nil {
		return 0, fmt.Errorf("invalid memory value: %s", s)
	}
	return num, nil
}

// parseCPU converts a CPU fraction string to a cgroups v2 cpu.max value.
// Examples: "0.5" -> "50000 100000", "0.8" -> "80000 100000"
func parseCPU(s string) (string, error) {
	fraction, err := strconv.ParseFloat(s, 64)
	if err != nil {
		return "", fmt.Errorf("invalid cpu value: %s", s)
	}
	if fraction <= 0 || fraction > 1 {
		return "", fmt.Errorf("cpu must be between 0 and 1, got %s", s)
	}
	period := 100000
	quota := int(fraction * float64(period))
	return fmt.Sprintf("%d %d", quota, period), nil
}

func setupCgroup(containerID string, pid int, memLimit string, cpuLimit string) error {
	cgroupPath := filepath.Join("/sys/fs/cgroup", "minicontainer", containerID)

	// Create the cgroup directory
	// Note: on some systems, you may need to enable controllers first:
	// echo "+memory +cpu +pids" > /sys/fs/cgroup/cgroup.subtree_control
	if err := os.MkdirAll(cgroupPath, 0755); err != nil {
		return fmt.Errorf("create cgroup: %w", err)
	}

	// Set memory limit from the --memory flag (e.g. "256m", "1g")
	memBytes, err := parseMemory(memLimit)
	if err != nil {
		return fmt.Errorf("parse memory flag: %w", err)
	}
	if err := os.WriteFile(
		filepath.Join(cgroupPath, "memory.max"),
		[]byte(fmt.Sprintf("%d", memBytes)),
		0644,
	); err != nil {
		return fmt.Errorf("set memory limit: %w", err)
	}

	// Set CPU limit from the --cpu flag (e.g. "0.5" for 50%)
	cpuMax, err := parseCPU(cpuLimit)
	if err != nil {
		return fmt.Errorf("parse cpu flag: %w", err)
	}
	if err := os.WriteFile(
		filepath.Join(cgroupPath, "cpu.max"),
		[]byte(cpuMax),
		0644,
	); err != nil {
		return fmt.Errorf("set cpu limit: %w", err)
	}

	// Limit to 64 processes
	if err := os.WriteFile(
		filepath.Join(cgroupPath, "pids.max"),
		[]byte("64"),
		0644,
	); err != nil {
		return fmt.Errorf("set pids limit: %w", err)
	}

	// Add the container process to this cgroup
	if err := os.WriteFile(
		filepath.Join(cgroupPath, "cgroup.procs"),
		[]byte(fmt.Sprintf("%d", pid)),
		0644,
	); err != nil {
		return fmt.Errorf("add process to cgroup: %w", err)
	}

	return nil
}

Add cleanup functions for both the cgroup and the veth interface:

func removeCgroup(containerID string) {
	cgroupPath := filepath.Join("/sys/fs/cgroup", "minicontainer", containerID)
	// The kernel removes cgroup control files automatically when the
	// directory is deleted. os.Remove calls rmdir, which works only
	// when the cgroup has no child cgroups and no live processes.
	if err := os.Remove(cgroupPath); err != nil {
		fmt.Println("Warning: could not remove cgroup:", err)
	}
}

func removeNetworking(containerID string) {
	vethHost := "veth-" + containerID[len("container-"):]
	// Deleting the host end automatically removes the peer inside the container
	execCmd("ip", "link", "delete", vethHost)
}

Update the run function to accept the CLI flags and set up cgroups. Use cmd.Start() instead of cmd.Run() so you can get the PID before the process finishes:

func run(userArgs []string, memLimit string, cpuLimit string) {
	fmt.Println("[parent] Starting container...")

	containerID := fmt.Sprintf("container-%d", os.Getpid())

	mergedDir, err := setupOverlayfs(containerID)
	if err != nil {
		fmt.Println("Error setting up overlayfs:", err)
		os.Exit(1)
	}

	// Use userArgs from Cobra instead of os.Args to avoid passing flags to the child
	cmd := exec.Command("/proc/self/exe", append([]string{"child", mergedDir}, userArgs...)...)
	cmd.Stdin = os.Stdin
	cmd.Stdout = os.Stdout
	cmd.Stderr = os.Stderr

	cmd.SysProcAttr = &syscall.SysProcAttr{
		Cloneflags: syscall.CLONE_NEWUTS |
			syscall.CLONE_NEWPID |
			syscall.CLONE_NEWNS |
			syscall.CLONE_NEWIPC |
			syscall.CLONE_NEWNET,
	}

	// Start the process (don't wait yet)
	if err := cmd.Start(); err != nil {
		fmt.Println("Error starting container:", err)
		os.Exit(1)
	}

	// Set up cgroups using the CLI flags
	if err := setupCgroup(containerID, cmd.Process.Pid, memLimit, cpuLimit); err != nil {
		fmt.Println("Warning: cgroup setup failed:", err)
	}

	// Wait for the container process to finish
	if err := cmd.Wait(); err != nil {
		fmt.Println("[parent] Container exited with error:", err)
	}

	// Clean up: remove networking, cgroup, and OverlayFS mount
	removeNetworking(containerID)
	removeCgroup(containerID)
	if err := syscall.Unmount(mergedDir, 0); err != nil {
		fmt.Println("Warning: could not unmount overlay:", err)
	}
	fmt.Println("[parent] Container stopped.")
}

Testing Cgroup Limits

Build and test the memory limit:

go build -o minicontainer .
sudo ./minicontainer run /bin/sh

Inside the container, try to use more than 256MB of memory:

# Allocate memory until the OOM killer stops us
# The dd command only does I/O, it does not allocate memory
# Use a simple shell trick or install stress-ng:
apk add stress-ng
stress-ng --vm 1 --vm-bytes 300M --timeout 10s
# The container process will be killed by the OOM killer

Check the cgroup from the host (in another terminal):

cat /sys/fs/cgroup/minicontainer/container-*/memory.max
# 268435456 (256MB)

cat /sys/fs/cgroup/minicontainer/container-*/memory.current
# Shows current memory usage

Container Networking

Right now, your container has no network access. It has a network namespace with only a loopback interface. To give it network access, you need to create a virtual ethernet pair (veth pair).

A veth pair is like a virtual cable with two ends. One end goes inside the container’s network namespace. The other end stays on the host and connects to a bridge.

┌──────────────┐         ┌──────────────┐
│   Container  │         │     Host     │
│              │         │              │
│  eth0 ──────┼─────────┼── veth-xxx   │
│  172.16.0.2  │  veth   │  bridge0     │
│              │  pair   │  172.16.0.1  │
└──────────────┘         └──────────────┘
                                │
                           NAT/iptables
                                │
                           Internet

Setting Up Networking with ip Commands

Container networking requires several steps. We will use the ip command through exec.Command for clarity:

func setupNetworking(pid int, containerID string) error {
	vethHost := "veth-" + containerID[len("container-"):]
	vethContainer := "eth0"

	// Create a veth pair
	if err := execCmd("ip", "link", "add", vethHost, "type", "veth", "peer", "name", vethContainer); err != nil {
		return fmt.Errorf("create veth pair: %w", err)
	}

	// Move one end into the container's network namespace
	if err := execCmd("ip", "link", "set", vethContainer, "netns", fmt.Sprintf("%d", pid)); err != nil {
		return fmt.Errorf("move veth to netns: %w", err)
	}

	// Set up the host end
	if err := execCmd("ip", "addr", "add", "172.16.0.1/24", "dev", vethHost); err != nil {
		// Address might already exist, ignore error
		fmt.Println("Warning: could not set host IP:", err)
	}
	if err := execCmd("ip", "link", "set", vethHost, "up"); err != nil {
		return fmt.Errorf("bring up host veth: %w", err)
	}

	// Set up the container end (run inside the namespace using nsenter)
	nsenterPrefix := []string{"nsenter", fmt.Sprintf("--net=/proc/%d/ns/net", pid), "--"}
	if err := execCmd(append(nsenterPrefix, "ip", "addr", "add", "172.16.0.2/24", "dev", "eth0")...); err != nil {
		return fmt.Errorf("set container IP: %w", err)
	}
	if err := execCmd(append(nsenterPrefix, "ip", "link", "set", "eth0", "up")...); err != nil {
		return fmt.Errorf("bring up container eth0: %w", err)
	}
	if err := execCmd(append(nsenterPrefix, "ip", "link", "set", "lo", "up")...); err != nil {
		return fmt.Errorf("bring up container lo: %w", err)
	}
	if err := execCmd(append(nsenterPrefix, "ip", "route", "add", "default", "via", "172.16.0.1")...); err != nil {
		return fmt.Errorf("set default route: %w", err)
	}

	return nil
}

func execCmd(args ...string) error {
	cmd := exec.Command(args[0], args[1:]...)
	cmd.Stdout = os.Stdout
	cmd.Stderr = os.Stderr
	return cmd.Run()
}

Enabling Internet Access

The container can now talk to the host (172.16.0.1), but it cannot reach the internet yet. You need NAT (Network Address Translation) with iptables:

func enableNAT() error {
	// Enable IP forwarding
	if err := os.WriteFile("/proc/sys/net/ipv4/ip_forward", []byte("1"), 0644); err != nil {
		return fmt.Errorf("enable ip forwarding: %w", err)
	}

	// Check if the NAT rule already exists before adding it.
	// Without this check, every container run appends a duplicate rule.
	natArgs := []string{"-t", "nat", "POSTROUTING", "-s", "172.16.0.0/24", "-j", "MASQUERADE"}
	if err := execCmd(append([]string{"iptables"}, append([]string{"-C"}, natArgs...)...)...); err != nil {
		// Rule does not exist yet — add it
		if err := execCmd(append([]string{"iptables"}, append([]string{"-A"}, natArgs...)...)...); err != nil {
			return fmt.Errorf("add NAT rule: %w", err)
		}
	}

	return nil
}

Add networking setup to the run function after cmd.Start():

// Set up cgroups using the CLI flags
if err := setupCgroup(containerID, cmd.Process.Pid, memLimit, cpuLimit); err != nil {
	fmt.Println("Warning: cgroup setup failed:", err)
}

// Set up networking
if err := setupNetworking(cmd.Process.Pid, containerID); err != nil {
	fmt.Println("Warning: networking setup failed:", err)
} else {
	if err := enableNAT(); err != nil {
		fmt.Println("Warning: NAT setup failed:", err)
	}
}

Note: Our implementation assigns a fixed IP (172.16.0.2) to every container. If you run two containers at the same time, the second one will fail. Docker solves this with a bridge interface (docker0). All container veth pairs connect to the bridge, and each container gets a unique IP from a DHCP-like allocator. For a production runtime, you would use a bridge too.

Testing Networking

go build -o minicontainer .
sudo ./minicontainer run /bin/sh

Inside the container:

# Check network interfaces
ip addr
# Shows eth0 with 172.16.0.2/24

# Ping the host
ping -c 1 172.16.0.1
# Should work

# Access the internet (if NAT is configured)
ping -c 1 8.8.8.8
# Should work

# DNS might not work yet — add a nameserver
echo "nameserver 8.8.8.8" > /etc/resolv.conf
ping -c 1 google.com
# Should work now

Building a CLI with Cobra

Let’s turn our container runtime into a proper CLI tool. Install Cobra:

go get github.com/spf13/cobra

Restructure the project:

minicontainer/
├── cmd/
│   ├── root.go
│   ├── run.go
│   ├── ps.go
│   └── exec.go
├── container/
│   ├── container.go
│   ├── cgroup.go
│   ├── network.go
│   └── filesystem.go
├── main.go
└── go.mod

Here is the root command:

// cmd/root.go
package cmd

import (
	"fmt"
	"os"

	"github.com/spf13/cobra"
)

var rootCmd = &cobra.Command{
	Use:   "minicontainer",
	Short: "A mini container runtime built from scratch",
}

func Execute() {
	if err := rootCmd.Execute(); err != nil {
		fmt.Println(err)
		os.Exit(1)
	}
}

The run command:

// cmd/run.go
package cmd

import (
	"fmt"

	"github.com/spf13/cobra"
)

var (
	memoryLimit string
	cpuLimit    string
)

var runCmd = &cobra.Command{
	Use:   "run [command] [args...]",
	Short: "Run a command in a new container",
	Args:  cobra.MinimumNArgs(1),
	Run: func(cmd *cobra.Command, args []string) {
		fmt.Printf("Starting container with command: %v\n", args)
		fmt.Printf("Memory limit: %s, CPU limit: %s\n", memoryLimit, cpuLimit)
		// Pass positional args and CLI flags to the run function
		run(args, memoryLimit, cpuLimit)
	},
}

func init() {
	runCmd.Flags().StringVar(&memoryLimit, "memory", "256m", "Memory limit (e.g., 256m, 1g)")
	runCmd.Flags().StringVar(&cpuLimit, "cpu", "0.5", "CPU limit (e.g., 0.5 for 50%)")
	rootCmd.AddCommand(runCmd)
}

The ps command lists running containers:

// cmd/ps.go
package cmd

import (
	"fmt"
	"os"
	"path/filepath"
	"strings"

	"github.com/spf13/cobra"
)

var psCmd = &cobra.Command{
	Use:   "ps",
	Short: "List running containers",
	Run: func(cmd *cobra.Command, args []string) {
		fmt.Printf("%-20s %-10s %-20s\n", "CONTAINER ID", "PID", "STATUS")
		fmt.Println(strings.Repeat("-", 50))

		cgroupBase := "/sys/fs/cgroup/minicontainer"
		entries, err := os.ReadDir(cgroupBase)
		if err != nil {
			return
		}

		for _, entry := range entries {
			if !entry.IsDir() {
				continue
			}

			procsFile := filepath.Join(cgroupBase, entry.Name(), "cgroup.procs")
			data, err := os.ReadFile(procsFile)
			if err != nil {
				continue
			}

			lines := strings.Split(strings.TrimSpace(string(data)), "\n")
			if len(lines) == 0 || lines[0] == "" {
				continue
			}

			// Show the first PID (the container's init process)
			fmt.Printf("%-20s %-10s %-20s\n", entry.Name(), lines[0], "running")
		}
	},
}

func init() {
	rootCmd.AddCommand(psCmd)
}

The exec command runs a command in an existing container:

// cmd/exec.go
package cmd

import (
	"fmt"
	"os"
	"os/exec"
	"path/filepath"
	"strings"

	"github.com/spf13/cobra"
)

var execCmd2 = &cobra.Command{
	Use:   "exec [container-id] [command] [args...]",
	Short: "Run a command in a running container",
	Args:  cobra.MinimumNArgs(2),
	Run: func(cmd *cobra.Command, args []string) {
		containerID := args[0]
		command := args[1:]

		// Find the PID from the cgroup
		procsFile := filepath.Join("/sys/fs/cgroup/minicontainer", containerID, "cgroup.procs")
		data, err := os.ReadFile(procsFile)
		if err != nil {
			fmt.Println("Container not found:", containerID)
			os.Exit(1)
		}

		pid := strings.TrimSpace(strings.Split(string(data), "\n")[0])
		if pid == "" {
			fmt.Println("Container has no running processes")
			os.Exit(1)
		}

		// Use nsenter to join the container's namespaces
		nsenterArgs := []string{
			fmt.Sprintf("--pid=/proc/%s/ns/pid", pid),
			fmt.Sprintf("--uts=/proc/%s/ns/uts", pid),
			fmt.Sprintf("--net=/proc/%s/ns/net", pid),
			fmt.Sprintf("--mount=/proc/%s/ns/mnt", pid),
			fmt.Sprintf("--ipc=/proc/%s/ns/ipc", pid),
			"--",
		}
		nsenterArgs = append(nsenterArgs, command...)

		nsenter := exec.Command("nsenter", nsenterArgs...)
		nsenter.Stdin = os.Stdin
		nsenter.Stdout = os.Stdout
		nsenter.Stderr = os.Stderr

		if err := nsenter.Run(); err != nil {
			fmt.Println("Error:", err)
			os.Exit(1)
		}
	},
}

func init() {
	rootCmd.AddCommand(execCmd2)
}

You also need a hidden child command. This is the command that /proc/self/exe calls inside the new namespaces. Cobra must know about it, but users should not see it:

// cmd/child.go
package cmd

import (
	"fmt"
	"os"

	"github.com/spf13/cobra"
)

var childCmd = &cobra.Command{
	Use:    "child",
	Hidden: true, // Internal command — not for users
	Run: func(cmd *cobra.Command, args []string) {
		// This runs inside the new namespaces
		// Call setupRootfs, setupHostname, then exec the user command
		fmt.Printf("[container] Running %v as PID %d\n", args, os.Getpid())
		// Move your child() logic from Part 1/2 here
	},
}

func init() {
	rootCmd.AddCommand(childCmd)
}

The run command’s Run function should call exec.Command("/proc/self/exe", "child", ...) just like in Part 1 and Part 2. Cobra separates positional arguments from flags, so args contains only the user command (e.g., ["/bin/sh"]). Pass those along with the merged directory to the child process.

The updated main.go:

// main.go
package main

import "github.com/kemalcodes/minicontainer/cmd"

func main() {
	cmd.Execute()
}

Using the CLI

# Run a container with custom limits
sudo minicontainer run --memory 512m --cpu 0.8 /bin/sh

# List running containers (in another terminal)
sudo minicontainer ps
# CONTAINER ID         PID        STATUS
# --------------------------------------------------
# container-12345      67890      running

# Execute a command in a running container
sudo minicontainer exec container-12345 /bin/sh

Running the Final Container

Let’s put it all together and run a complete container:

# Build the project
go build -o minicontainer .

# Run a container
sudo ./minicontainer run /bin/sh

Inside the container:

# Check isolation
hostname            # "container" — UTS namespace
ps aux              # Only our processes — PID namespace
ip addr             # Only eth0 + lo — Network namespace
cat /etc/os-release # Alpine Linux — filesystem isolation

# Check resource limits
cat /sys/fs/cgroup/memory.max  # 268435456 (256MB)

# Install software (isolated to this container)
apk add curl
curl https://httpbin.org/ip

# Exit
exit
# Container stopped. Changes are gone (OverlayFS upper layer).

What We Built vs What Docker Does

Here is a comparison of our mini container runtime and Docker:

Feature	minicontainer	Docker
Process isolation	Namespaces	Namespaces
Resource limits	Cgroups v2	Cgroups v2
Filesystem	OverlayFS + pivot_root	OverlayFS + pivot_root
Networking	veth pairs + NAT	veth pairs + bridge + NAT
Image format	Alpine tarball	OCI image spec
Image registry	None	Docker Hub, etc.
CLI	Cobra (run, ps, exec)	Docker CLI
Security	Basic namespaces	seccomp, AppArmor, capabilities
Orchestration	None	Docker Compose, Swarm

The core is the same. Docker adds image management, security profiles, logging, volume management, and much more. But the foundation — namespaces, cgroups, and OverlayFS — is exactly what we built.

What We Learned

In this three-part series, you built a container runtime from scratch. Here is what you learned:

Part 1 — Namespaces:

Containers are just Linux processes with isolation
Linux namespaces control visibility (PID, UTS, Mount, Network, IPC)
Go’s syscall.SysProcAttr makes namespace creation simple
The two-phase pattern (parent creates namespaces, child runs inside them)

Part 2 — Filesystem:

pivot_root is more secure than chroot because you can unmount the old root
Alpine Linux provides a tiny but complete root filesystem
OverlayFS enables copy-on-write layered filesystems
Docker images are stacks of OverlayFS layers

Part 3 — Cgroups and Networking:

Cgroups v2 limits memory, CPU, and process count through filesystem writes
Virtual ethernet pairs connect container and host network namespaces
NAT with iptables gives containers internet access
Cobra makes building Go CLIs straightforward (see our Cobra tutorial)

Going Further

Want to keep building? Here are some ideas:

Add seccomp filters — restrict which system calls the container can make
Implement image pulling — download OCI images from Docker Hub
Add volume mounts — share directories between host and container
Implement port mapping — forward host ports to the container
Add logging — capture container stdout and stderr to files

You can also study real container runtimes:

runc — the OCI reference runtime (written in Go)
crun — a fast OCI runtime (written in C)
youki — an OCI runtime (written in Rust)

The source code for this series is available in our examples. Clone it and experiment.

Build Docker from Scratch in Go — Part 1: Linux Namespaces — process isolation
Build Docker from Scratch in Go — Part 2: Filesystem Isolation — chroot and OverlayFS
Go Tutorial: Getting Started with Go — learn Go from the beginning
Go Tutorial: CLI Tools with Cobra — building CLIs in Go
Docker Cheat Sheet 2026 — quick reference for Docker commands
Go Tutorial: Docker for Go — packaging Go apps with Docker

Cgroups: Resource Limits#

How Cgroups v2 Works#

Implementing Cgroups in Go#

Testing Cgroup Limits#

Container Networking#

Setting Up Networking with ip Commands#

Enabling Internet Access#

Testing Networking#

Building a CLI with Cobra#

Using the CLI#

Running the Final Container#

What We Built vs What Docker Does#

What We Learned#

Going Further#

Related Articles#