Linux Cgroups Overview

Linux Cgroups (Control Groups) provide the ability to limit, control, and monitor the resources used by a group of processes and their future child processes. These resources include CPU, memory, storage, and network. With Cgroups, it’s easy to limit a process’s resource usage and monitor its metrics in real time.

Three Components of Cgroups

• cgroup

  A mechanism for managing groups of processes. A cgroup contains a group of processes, and various Linux subsystem parameters can be configured on this cgroup, associating a group of processes with a group of system parameters from subsystems.

• subsystem

  A module that controls a set of resources.

  

  Each subsystem is linked to a cgroup that defines the respective limits, and the subsystem imposes these limits on the processes in the cgroup.

• hierarchy

  A hierarchy is a tree structure that links multiple cgroups. With this tree structure, cgroups can inherit attributes from their parent cgroups.

  Example Scenario: Suppose there is a group of periodic tasks limited by cgroup1 in terms of CPU usage. If one of these tasks is a logging process that also needs to be limited by disk I/O, a new cgroup2 can be created that inherits from cgroup1. cgroup2 will inherit the CPU limit from cgroup1 and add its own disk I/O limitation, without affecting other processes in cgroup1.

Relationships Between the Three

• When a new hierarchy is created, all processes in the system join the root cgroup of that hierarchy by default. This root cgroup is created automatically with the hierarchy.
• A subsystem can only be attached to one hierarchy.
• A hierarchy can have multiple subsystems attached.
• A process can belong to multiple cgroups in different hierarchies.
• A child process is in the same cgroup as its parent process but can be moved to a different cgroup later.

Kernel Interface

Hierarchies in cgroups are organized in a tree structure. The kernel provides a virtual tree-like file system to configure cgroups, making it intuitive to work with them through a hierarchical directory structure.

• Create a hierarchy and add sub-cgroups:

mkdir cgroup # Create mount point
sudo mount -t cgroup -o none,name=cgroup-test cgroup-test ./cgroup-test # Mount hierarchy
sudo mkdir cgroup-1
sudo mkdir cgroup-2
tree

.
├── cgroup-1
│   ├── cgroup.clone_children
│   ├── cgroup.procs
│   ├── notify_on_release
│   └── tasks
├── cgroup-2
│   ├── cgroup.clone_children
│   ├── cgroup.procs
│   ├── notify_on_release
│   └── tasks
├── cgroup.clone_children
├── cgroup.procs
├── cgroup.sane_behavior
├── notify_on_release
├── release_agent
└── tasks

Meaning of Different Files

• Add and move processes to a cgroup (move process PID into the corresponding tasks file):

sudo sh -c "echo $$ >> ./cgroup-1/tasks" # Move terminal process to cgroup-1
cat /proc/$$/cgroup

>> 
13:name=cgroup-test:/cgroup-1
12:memory:/user.slice/user-1002.slice/session-12331.scope
11:perf_event:/
10:cpuset:/
9:freezer:/
8:blkio:/user.slice
7:rdma:/
6:hugetlb:/
5:pids:/user.slice/user-1002.slice/session-12331.scope
4:cpu,cpuacct:/user.slice
3:net_cls,net_prio:/
2:devices:/user.slice
1:name=systemd:/user.slice/user-1002.slice/session-12331.scope
0::/user.slice/user-1002.slice/session-12331.scope

• Limit cgroup resource usage via subsystems:

  First, link the hierarchy to a subsystem. By default, the system links the hierarchy to a memory subsystem.

  # Start a memory-intensive stress process without any limitations
  stress --vm-bytes 200m --vm-keep -m 1
  sudo mkdir test-limit-memory && cd test-limit-memory # Create a cgroup
  sudo sh -c "echo "100m" > memory.limit" # Set max memory usage to 100m
  sudo sh -c "echo $$ > tasks" # Move current process to cgroup
  stress --vm-bytes 200m --vm-keep -m 1
  

Observation

The memory usage of the process is limited by the specified setting.