Kürzlich stand ich vor dem Problem, dass einer meiner VPS Server immer vom Hoster gestoppt wurde. Dieses Problem tritt häufig auf, wenn bei einem 1 vCPU Server der CPU-load länger als 10 Minuten über 1.00 liegt.
Zur Lösung des Problems habe ich das Tool CPULIMIT entdeckt. Es gibt es auf Ubuntu & Centos / bereits als pre-kompiliertes Paket un kann einfach via apt-get oder yum installiert werden.
yum install cpulimit gawk
Im Anschluss habe ich ein Skript zu start eines Daemon hier /usr/bin/cpulimit_daemon.sh angelegt, mit diesem Inhalt:
#!/bin/bash # ============================================================== # CPU limit daemon - set PID's max. percentage CPU consumptions # ============================================================== # Variables CPU_LIMIT=20 # Maximum percentage CPU consumption by each PID DAEMON_INTERVAL=3 # Daemon check interval in seconds BLACK_PROCESSES_LIST= # Limit only processes defined in this variable. If variable is empty (default) all violating processes are limited. WHITE_PROCESSES_LIST= # Limit all processes except processes defined in this variable. If variable is empty (default) all violating processes are limited. # Check if one of the variables BLACK_PROCESSES_LIST or WHITE_PROCESSES_LIST is defined. if [[ -n "$BLACK_PROCESSES_LIST" && -n "$WHITE_PROCESSES_LIST" ]] ; then # If both variables are defined then error is produced. echo "At least one or both of the variables BLACK_PROCESSES_LIST or WHITE_PROCESSES_LIST must be empty." exit 1 elif [[ -n "$BLACK_PROCESSES_LIST" ]] ; then # If this variable is non-empty then set NEW_PIDS_COMMAND variable to bellow command NEW_PIDS_COMMAND="top -b -n1 -c | grep -E '$BLACK_PROCESSES_LIST' | gawk '\$9>CPU_LIMIT {print \$1}' CPU_LIMIT=$CPU_LIMIT" elif [[ -n "$WHITE_PROCESSES_LIST" ]] ; then # If this variable is non-empty then set NEW_PIDS_COMMAND variable to bellow command NEW_PIDS_COMMAND="top -b -n1 -c | gawk 'NR>6' | grep -E -v '$WHITE_PROCESSES_LIST' | gawk '\$9>CPU_LIMIT {print \$1}' CPU_LIMIT=$CPU_LIMIT" else NEW_PIDS_COMMAND="top -b -n1 -c | gawk 'NR>6 && \$9>CPU_LIMIT {print \$1}' CPU_LIMIT=$CPU_LIMIT" fi # Search and limit violating PIDs while sleep $DAEMON_INTERVAL do NEW_PIDS=$(eval "$NEW_PIDS_COMMAND") # Violating PIDs LIMITED_PIDS=$(ps -eo args | gawk '$1=="cpulimit" {print $3}') # Already limited PIDs QUEUE_PIDS=$(comm -23 <(echo "$NEW_PIDS" | sort -u) <(echo "$LIMITED_PIDS" | sort -u) | grep -v '^$') # PIDs in queue for i in $QUEUE_PIDS do cpulimit -p $i -l $CPU_LIMIT -z & # Limit new violating processes done done
Um das Skript ausführen zu können müssen noch die Berechtigungen geändert werden:
chmod 755 /usr/bin/cpulimit_daemon.sh
Um nun den Job zu starten einfach ein Skript wie folgt unter /etc/init.d/cpulimit anlegen:
#!/bin/sh # # Script to start CPU limit daemon # set -e case "$1" in start) if [ $(ps -eo pid,args | gawk '$3=="/usr/bin/cpulimit_daemon.sh" {print $1}' | wc -l) -eq 0 ]; then nohup /usr/bin/cpulimit_daemon.sh >/dev/null 2>&1 & ps -eo pid,args | gawk '$3=="/usr/bin/cpulimit_daemon.sh" {print}' | wc -l | gawk '{ if ($1 == 1) print " * cpulimit daemon started successfully"; else print " * cpulimit daemon can not be started" }' else echo " * cpulimit daemon can't be started, because it is already running" fi ;; stop) CPULIMIT_DAEMON=$(ps -eo pid,args | gawk '$3=="/usr/bin/cpulimit_daemon.sh" {print $1}' | wc -l) CPULIMIT_INSTANCE=$(ps -eo pid,args | gawk '$2=="cpulimit" {print $1}' | wc -l) CPULIMIT_ALL=$((CPULIMIT_DAEMON + CPULIMIT_INSTANCE)) if [ $CPULIMIT_ALL -gt 0 ]; then if [ $CPULIMIT_DAEMON -gt 0 ]; then ps -eo pid,args | gawk '$3=="/usr/bin/cpulimit_daemon.sh" {print $1}' | xargs kill -9 # kill cpulimit daemon fi if [ $CPULIMIT_INSTANCE -gt 0 ]; then ps -eo pid,args | gawk '$2=="cpulimit" {print $1}' | xargs kill -9 # release cpulimited process to normal priority fi ps -eo pid,args | gawk '$3=="/usr/bin/cpulimit_daemon.sh" {print}' | wc -l | gawk '{ if ($1 == 1) print " * cpulimit daemon can not be stopped"; else print " * cpulimit daemon stopped successfully" }' else echo " * cpulimit daemon can't be stopped, because it is not running" fi ;; restart) $0 stop sleep 3 $0 start ;; status) ps -eo pid,args | gawk '$3=="/usr/bin/cpulimit_daemon.sh" {print}' | wc -l | gawk '{ if ($1 == 1) print " * cpulimit daemon is running"; else print " * cpulimit daemon is not running" }' ;; esac exit 0
Nun noch die Berechtigungen ändern:
chown root:root /etc/init.d/cpulimit
Um das Skript auf CentOs zu starten habe ich es einfach am ende von /etc/rc.config eingefügt. somit wird es nach dem Booten automatisch gestartet und limitiert dann die CPU intensiven Jobs.