Initial commit with cgexec and user_cgroup
This commit is contained in:
parent
a978cb0c20
commit
8815334401
112
README.md
112
README.md
@ -1,5 +1,115 @@
|
|||||||
|
# Linux Control Groups
|
||||||
|
|
||||||
|
Linux Control Groups, commonly referred to as cgroups, is a feature in the Linux kernel that provides a way to organise and manage system resources for processes. It allows you to allocate and limit resources like CPU, memory, and I/O bandwidth among different processes or groups of processes.
|
||||||
|
|
||||||
|
There are two versions of cgroups, the original version 1 and the `unified`, version 2.
|
||||||
|
|
||||||
# cgexec
|
# cgexec
|
||||||
|
|
||||||
cgexec is a Bash script that allows users to execute a command within a cgroup, providing control over resource limits such as CPU, I/O, and memory.
|
cgexec is a Bash script that allows users to execute a command within a cgroup, providing control over resource limits such as CPU, I/O, and memory.
|
||||||
|
|
||||||
https://wiki.tnonline.net/w/Linux/cgexec
|
The script requires v2/unified cgroups to function.
|
||||||
|
|
||||||
|
Documentation is available at <https://wiki.tnonline.net/w/Linux/cgexec>.
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
```
|
||||||
|
Attaches a program <cmd> to a unified cgroup with defined limits.
|
||||||
|
|
||||||
|
Usage: cgexec [options] <cmd> [cmd args]
|
||||||
|
Options:
|
||||||
|
-c cpu.weight (0-10000) Set CPU priority
|
||||||
|
-C cpu.max (1-100) Set max CPU time in percent
|
||||||
|
-i io.weight (1-10000) Set I/O weight
|
||||||
|
-m memory.high (0-max) Set soft memory limit
|
||||||
|
-M memory.max (0-max) Set hard memory limit
|
||||||
|
-g group Create or attach to existing cgroup. Default is to use an ephemeral group
|
||||||
|
-b path Use <path> as cgroup root
|
||||||
|
|
||||||
|
Requires Linux Control Groups v2 mounted at <path>.
|
||||||
|
```
|
||||||
|
|
||||||
|
## Examples
|
||||||
|
|
||||||
|
Execute a command with default settings
|
||||||
|
```
|
||||||
|
cgexec echo "Hello, cgroups!"
|
||||||
|
```
|
||||||
|
|
||||||
|
Limit CPU and memory for a command
|
||||||
|
```
|
||||||
|
cgexec -c 50 -m 1G my_command
|
||||||
|
```
|
||||||
|
|
||||||
|
Attach to an existing cgroup
|
||||||
|
```
|
||||||
|
cgexec -g mygroup my_command
|
||||||
|
```
|
||||||
|
|
||||||
|
Use a custom cgroup root
|
||||||
|
```
|
||||||
|
cgexec -b /sys/fs/cgroup/mysubsystem my_command
|
||||||
|
```
|
||||||
|
|
||||||
|
# Enable user cgroups
|
||||||
|
It is possible to allow normal users to create their own cgroups by creating an initial cgroup container that is owned by the user.
|
||||||
|
|
||||||
|
First set up a container for the user
|
||||||
|
```
|
||||||
|
mkdir -p /sys/fs/cgroup/user/forza/main
|
||||||
|
echo "+cpu +memory +io" > /sys/fs/cgroup/user/cgroup.subtree_control
|
||||||
|
echo "+cpu +memory +io" > /sys/fs/cgroup/user/forza/cgroup.subtree_control
|
||||||
|
chown forza:forza /sys/fs/cgroup/user/forza
|
||||||
|
chown forza:forza /sys/fs/cgroup/user/forza/cgroup.procs
|
||||||
|
chown forza:forza /sys/fs/cgroup/user/forza/cgroup.threads
|
||||||
|
chown -R forza:forza /sys/fs/cgroup/user/forza/main
|
||||||
|
chmod 775 /sys/fs/cgroup/user/forza
|
||||||
|
chmod 775 /sys/fs/cgroup/user/forza/main
|
||||||
|
```
|
||||||
|
User `<forza>` can now create new nested cgroups in `/sys/fs/cgroup/user/forza` and attach processes to it.
|
||||||
|
The catch-22 of cgroups is that all programs by default reside in in the root `/sys/fs/cgroup/cgroup.procs`, and even though a user can write to its own cgroup, they can not remove themselves from the root cgroup.
|
||||||
|
|
||||||
|
One solution is to let the user start a `screen` or `tmux` session and then use root/sudo to move it to the user's cgroup.
|
||||||
|
|
||||||
|
Then as root, check what `pids` belong to the uaer session and add them to the user cgroup:
|
||||||
|
```
|
||||||
|
# ps af -u forza
|
||||||
|
5309 pts/0 S+ 0:00 | \_ screen
|
||||||
|
5310 ? Ss 0:00 | \_ SCREEN
|
||||||
|
5311 pts/1 Ss 0:00 | \_ -/bin/bash
|
||||||
|
5483 pts/1 R+ 0:00 | \_ ps af -u forza
|
||||||
|
|
||||||
|
# echo 5309 > /sys/fs/cgroup/user/forza/main/cgroup.procs
|
||||||
|
# echo 5310 > /sys/fs/cgroup/user/forza/main/cgroup.procs
|
||||||
|
# echo 5311 > /sys/fs/cgroup/user/forza/main/cgroup.procs
|
||||||
|
```
|
||||||
|
Now, the user's screen session is in the `main` cgroup and the user can create additional cgroups and move processes started from within this session to that cgroup.
|
||||||
|
```
|
||||||
|
## Helper utility
|
||||||
|
The `user_cgroup` helper utility can automate the process of setting of user cgroups.
|
||||||
|
|
||||||
|
- `user_cgroup.initd` is an OpenRC init script that creates a cgroup hierarchy for specified users.
|
||||||
|
- `user_cgroup.confd` is the configuration file for OpenRC.
|
||||||
|
- `user_cgroup.sudoersd` is a sudo config file for allowing users to use the `user_cgroup` utility.
|
||||||
|
- `user_cgroup` small tool to move a uaer's shell to the user cgroup.
|
||||||
|
|
||||||
|
The sudo file is optional, but allows a user to put the script in `~/.bash_profile` for automatic use.
|
||||||
|
|
||||||
|
Once installed, a user can simply run `user_cgroup` to move itselslf to the user cgroup. Now it is possible for the user to create its own sub cgroups with `cgexec`.
|
||||||
|
|
||||||
|
```
|
||||||
|
# ./cgexec ls -l
|
||||||
|
* Creating cgroup: /sys/fs/cgroup/user/forza/cmd-GEP0
|
||||||
|
* Executing: ls -l
|
||||||
|
|
||||||
|
total 60
|
||||||
|
-rwxr-xr-x 1 forza forza 4402 Dec 17 14:20 cgexec
|
||||||
|
-rw-r--r-- 1 forza forza 34580 Dec 16 19:02 LICENSE
|
||||||
|
drwxr-xr-x 1 forza forza 72 Dec 16 21:16 openrc
|
||||||
|
-rw-r--r-- 1 forza forza 4106 Dec 17 13:57 README.md
|
||||||
|
-rwxr-xr-x 1 forza forza 315 Dec 17 13:37 user_cgroup
|
||||||
|
-rw-r--r-- 1 forza forza 79 Dec 17 13:40 user_cgroup.sudoersd
|
||||||
|
|
||||||
|
* Cleaning up cgroup /sys/fs/cgroup/user/forza/cmd-GEP0
|
||||||
|
```
|
||||||
199
cgexec
Executable file
199
cgexec
Executable file
@ -0,0 +1,199 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
# shellcheck shell=bash
|
||||||
|
|
||||||
|
# cgexec - execute a command in a cgroup
|
||||||
|
# version 2023-12-16.0
|
||||||
|
#
|
||||||
|
# Requires Linux Control Groups version v2 (unified)
|
||||||
|
# https://www.kernel.org/doc/Documentation/cgroup-v2.txt
|
||||||
|
#
|
||||||
|
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
# Copyright 2023 Forza <forza@tnonline.net>
|
||||||
|
|
||||||
|
# Declare variables
|
||||||
|
declare -i -r self=$$
|
||||||
|
declare -i cpu=0
|
||||||
|
declare -i cpu_max=100
|
||||||
|
declare -i io=1
|
||||||
|
declare -i errorlevel=0
|
||||||
|
declare -i cg_keep=0
|
||||||
|
declare -i wait=10
|
||||||
|
declare mem_high="max"
|
||||||
|
declare mem_max="max"
|
||||||
|
declare cg=""
|
||||||
|
declare cgcmd=""
|
||||||
|
declare group=""
|
||||||
|
declare -r cgmnt="$(findmnt --noheadings --output TARGET -t cgroup2)"
|
||||||
|
|
||||||
|
# Set base to the cgroup v2 mount point
|
||||||
|
if [ "$USER" != "root" ] && [ -d "${cgmnt}/user/${USER}" ]; then
|
||||||
|
declare base="${cgmnt}/user/${USER}"
|
||||||
|
else
|
||||||
|
declare base="${cgmnt}"
|
||||||
|
fi
|
||||||
|
|
||||||
|
help_text(){
|
||||||
|
cat << END
|
||||||
|
|
||||||
|
Attaches a program <cmd> to a cgroup with defined limits.
|
||||||
|
Requires Linux Control Groups v2.
|
||||||
|
|
||||||
|
Usage: cgexec [options] <cmd> [cmd args]
|
||||||
|
Options:
|
||||||
|
-c cpu.weight (0-10000) Set CPU priority
|
||||||
|
-C cpu.max (1-100) Set max CPU time in percent
|
||||||
|
-i io.weight (1-10000) Set I/O weight
|
||||||
|
-m memory.high (0-max) Set soft memory limit
|
||||||
|
-M memory.max (0-max) Set hard memory limit
|
||||||
|
-g group Create or attach to existing cgroup. Default is to use an ephemeral group
|
||||||
|
-b path Use <path> as cgroup root
|
||||||
|
END
|
||||||
|
}
|
||||||
|
|
||||||
|
cleanup(){
|
||||||
|
# Clean up and remove the cgroup
|
||||||
|
if [ $cg_keep -eq 1 ]; then
|
||||||
|
echo "* Not removing existing cgroup"
|
||||||
|
exit $errorlevel
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Move the script to the base cgroup
|
||||||
|
echo -e "\n* Cleaning up cgroup ${cg}"
|
||||||
|
|
||||||
|
if [ "$USER" != "root" ]; then
|
||||||
|
if [ -w "${base}/main/cgroup.procs" ]; then
|
||||||
|
echo $self > "${base}/main/cgroup.procs"
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
if [ -w "${base}/cgroup.procs" ]; then
|
||||||
|
echo -e "\n* Cleaning up cgroup ${cg}"
|
||||||
|
echo $self > "${base}/cgroup.procs"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Attempt to remove the cgroup
|
||||||
|
while [ $wait -gt 0 ]; do
|
||||||
|
rmdir "${cg}" >/dev/null 2>&1
|
||||||
|
if [ ! -d "${cg}" ]; then
|
||||||
|
break
|
||||||
|
else
|
||||||
|
echo -n "."
|
||||||
|
sleep 1
|
||||||
|
((wait--))
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
if [ $wait -eq 0 ]; then
|
||||||
|
echo "* Command exited but ${cg}/cgroup.procs is not empty"
|
||||||
|
echo "* Not removing ${cg}"
|
||||||
|
fi
|
||||||
|
exit $errorlevel
|
||||||
|
}
|
||||||
|
|
||||||
|
# Check arguments
|
||||||
|
if [ $# -eq 0 ]; then
|
||||||
|
help_text
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
while getopts 'c:C:i:m:M:g:b:h' OPT; do
|
||||||
|
case "$OPT" in
|
||||||
|
i)
|
||||||
|
io="$OPTARG"
|
||||||
|
echo "* I/O weight = $OPTARG"
|
||||||
|
;;
|
||||||
|
c)
|
||||||
|
cpu="$OPTARG"
|
||||||
|
echo "* CPU weight = $OPTARG"
|
||||||
|
;;
|
||||||
|
C)
|
||||||
|
cpu_max="$OPTARG"
|
||||||
|
echo "* CPU max = $OPTARG %"
|
||||||
|
;;
|
||||||
|
m)
|
||||||
|
mem_high="$OPTARG"
|
||||||
|
echo "* Memory High = $OPTARG"
|
||||||
|
;;
|
||||||
|
M)
|
||||||
|
mem_max="$OPTARG"
|
||||||
|
echo "* Memory Max = $OPTARG"
|
||||||
|
;;
|
||||||
|
g)
|
||||||
|
group="$OPTARG"
|
||||||
|
;;
|
||||||
|
b)
|
||||||
|
base="$OPTARG"
|
||||||
|
echo "* Base = $OPTARG"
|
||||||
|
;;
|
||||||
|
h)
|
||||||
|
help_text
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
help_text
|
||||||
|
exit 1
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
shift $(( OPTIND - 1 ))
|
||||||
|
cgcmd="${1}"
|
||||||
|
shift
|
||||||
|
|
||||||
|
# Check if cgcmd is a valid executable
|
||||||
|
if ! command -v "${cgcmd}" &> /dev/null; then
|
||||||
|
echo "Error: '${cgcmd}' is not a valid executable."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Check for existing cgroup
|
||||||
|
if [ -n "${group}" ]; then
|
||||||
|
cg="${base}/${group}"
|
||||||
|
if [ ! -d "${cg}" ]; then
|
||||||
|
if [ ! -w "$(dirname "${cg}")" ]; then
|
||||||
|
echo "Error: You don't have write access to $(dirname "${cg}")."
|
||||||
|
echo "Cannot create ${cg}"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
mkdir "${cg}"
|
||||||
|
else
|
||||||
|
echo "* Using existing cgroup: ${cg}"
|
||||||
|
cg_keep=1
|
||||||
|
if [ ! -w "${cg}/cgroup.procs" ]; then
|
||||||
|
echo "Error: You don't have write access to ${cg}."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
echo $self > "${cg}/cgroup.procs"
|
||||||
|
else
|
||||||
|
# Create a temporary cgroup
|
||||||
|
cg="$(mktemp -d -p "${base}" cmd-XXXX)"
|
||||||
|
echo "* Creating cgroup: ${cg}"
|
||||||
|
echo $self > "${cg}/cgroup.procs"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Set cgroup limits
|
||||||
|
test -w "${cg}/io.weight" && echo $io > "${cg}/io.weight"
|
||||||
|
test -w "${cg}/io.bfq.weight" && echo $io > "${cg}/io.bfq.weight"
|
||||||
|
test -w "${cg}/memory.high" && echo "${mem_high}" > "${cg}/memory.high"
|
||||||
|
test -w "${cg}/memory.max" && echo "${mem_max}" > "${cg}/memory.max"
|
||||||
|
|
||||||
|
if [ $cpu -eq 0 ] && [ -w "${cg}/cpu.idle" ]; then
|
||||||
|
echo 1 > "${cg}/cpu.idle"
|
||||||
|
else
|
||||||
|
test -w "${cg}/cpu.weight" && echo $cpu > "${cg}/cpu.weight"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ $cpu_max -gt 0 ] && [ $cpu_max -lt 100 ] && [ -w "${cg}/cpu.max" ]; then
|
||||||
|
echo "$((cpu_max * 1000)) 100000" > "${cg}/cpu.max"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Execute command
|
||||||
|
echo -e "* Executing: ${cgcmd} ${*} \n"
|
||||||
|
"${cgcmd}" "${@}"
|
||||||
|
errorlevel=$?
|
||||||
|
if [ $errorlevel -ne 0 ]; then
|
||||||
|
echo -e "\nWARNING: ${cgcmd} exited with return code $errorlevel"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Cleanup before exiting
|
||||||
|
cleanup
|
||||||
13
openrc/user_cgroups.confd
Normal file
13
openrc/user_cgroups.confd
Normal file
@ -0,0 +1,13 @@
|
|||||||
|
# user_cgroups configuration file
|
||||||
|
|
||||||
|
# Enable a cgroup for these users.
|
||||||
|
users="user1 user2"
|
||||||
|
|
||||||
|
# Enable controllers tgat users can use
|
||||||
|
#controllers="+cpu +memory +io"
|
||||||
|
|
||||||
|
# Limit number of child cgroups that a user can create
|
||||||
|
#max_decendants=10
|
||||||
|
|
||||||
|
# Mount path for cgroup2
|
||||||
|
#cgroup_path="/sys/fs/cgroup"
|
||||||
61
openrc/user_cgroups.initd
Executable file
61
openrc/user_cgroups.initd
Executable file
@ -0,0 +1,61 @@
|
|||||||
|
#!/sbin/openrc-run
|
||||||
|
|
||||||
|
# OpenRC init script for setting up cgroups for specified users
|
||||||
|
# Requires Linux Control Groups version v2 (unified)
|
||||||
|
# https://www.kernel.org/doc/Documentation/cgroup-v2.txt
|
||||||
|
#
|
||||||
|
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
# Copyright 2023 Forza <forza@tnonline.net>
|
||||||
|
|
||||||
|
# shellcheck shell=sh
|
||||||
|
# shellcheck disable=SC2034
|
||||||
|
|
||||||
|
name="${RC_SVCNAME}"
|
||||||
|
description="User cgroups"
|
||||||
|
: "${controllers:="+cpu +memory +io"}"
|
||||||
|
: "${max_decendants:=10}"
|
||||||
|
: "${users:=""}"
|
||||||
|
: "${cgroup_path:="$(findmnt --noheadings --output TARGET -t cgroup2)"}"
|
||||||
|
base_path="${cgroup_path}/user"
|
||||||
|
|
||||||
|
depend() {
|
||||||
|
need cgroups
|
||||||
|
}
|
||||||
|
|
||||||
|
start() {
|
||||||
|
ebegin "Starting user cgroups"
|
||||||
|
|
||||||
|
if [ -z "${cgroup_path}" ]; then
|
||||||
|
eerror "cgroup2 is not mounted."
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Create base user cgrouo container
|
||||||
|
checkpath -d -m 0775 -o root:root "${base_path}"
|
||||||
|
|
||||||
|
# Enable cgroup controllers
|
||||||
|
echo "${controllers}" > "${base_path}/cgroup.subtree_control"
|
||||||
|
|
||||||
|
# Set up user cgroups
|
||||||
|
for user in ${users}; do
|
||||||
|
if ! id "${user}" > /dev/null 2>&1; then
|
||||||
|
eerror "User '${user}' does not exist. Skipping cgroup setup for this user."
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
if [ ! -d "${base_path}/${user}" ]; then
|
||||||
|
checkpath -d -m 0775 -o "${user}":"${user}" "${base_path}/${user}"
|
||||||
|
chown -R "${user}":"${user}" "${base_path}/${user}"
|
||||||
|
#find "${base_path}/${user}" -type f -exec chmod 664 {} +
|
||||||
|
fi
|
||||||
|
echo "$max_decendants" > "${base_path}/cgroup.max.descendants"
|
||||||
|
echo "${controllers}" > "${base_path}/${user}/cgroup.subtree_control"
|
||||||
|
done
|
||||||
|
einfo "User cgroups are mounted at ${base_path}/<username>"
|
||||||
|
eend
|
||||||
|
}
|
||||||
|
|
||||||
|
status() {
|
||||||
|
einfo "User cgroups are mounted at ${base_path}/<username>"
|
||||||
|
einfo "The following users' cgroups are enabled:"
|
||||||
|
find "${base_path}" -mindepth 1 -maxdepth 1 -type d -exec basename {} \;
|
||||||
|
}
|
||||||
13
user_cgroup
Executable file
13
user_cgroup
Executable file
@ -0,0 +1,13 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
[ "root" != "$USER" ] && exec env -i sudo $0 $PPID "$(id -un)"
|
||||||
|
declare -r user_id=$1
|
||||||
|
declare -r user_name=$2
|
||||||
|
cgroup="/sys/fs/cgroup/user/${user_name}/main"
|
||||||
|
|
||||||
|
[ "$user_name" = "root" ] && exit 0
|
||||||
|
|
||||||
|
if [ -w "${cgroup}" ]; then
|
||||||
|
echo "cgroup: ${cgroup}"
|
||||||
|
echo "${user_id}" > "${cgroup}/cgroup.procs"
|
||||||
|
fi
|
||||||
|
exit 0
|
||||||
2
user_cgroup.sudoersd
Normal file
2
user_cgroup.sudoersd
Normal file
@ -0,0 +1,2 @@
|
|||||||
|
# Allow user to use cgroups
|
||||||
|
forza ALL=(root) NOPASSWD: /opt/cgexec/user_cgroup
|
||||||
Loading…
Reference in New Issue
Block a user