How to use parallel ssh (PSSH) for executing ssh in parallel on a number of Linux/Unix/BSD servers

Recently I come across a nice little nifty tool called pssh to run a single command on multiple Linux / UNIX / BSD servers. You can easily increase your productivy with this SSH tool.

More about pssh

pssh is a command line tool for executing ssh in parallel on some hosts. It specialties includes:

1. Sending input to all of the processes

2. Inputting a password to ssh

3. Saving output to files

4. IT/sysadmin taks automation such as patching servers

5. Timing out and more

Let us see how to install and use pssh on Linux and Unix-like system.

Installation

You can install pssh as per your Linux and Unix variant. Once package installed, you can get parallel versions of the openssh tools. Included in the installation:

1. Parallel ssh (pssh command)

2. Parallel scp (pscp command )

3. Parallel rsync (prsync command)

4. Parallel nuke (pnuke command)

5. Parallel slurp (pslurp command)

Install pssh on Debian/Ubuntu Linux

Type the following apt-get command/apt command to install pssh:
$ sudo apt install pssh
OR
$ sudo apt-get install pssh
Sample outputs:

Fig.01: Installing pssh on Debian/Ubuntu Linux

Install pssh on Apple MacOS X

Type the following brew command:
$ brew install pssh
Sample outputs:

Fig.02: Installing pssh on MacOS Unix

Install pssh on FreeBSD unix

Type any one of the command:
# cd /usr/ports/security/pssh/ && make install clean
OR
# pkg install pssh
Sample outputs:

Fig.03: Installing pssh on FreeBSD

Install pssh on RHEL/CentOS/Fedora Linux

First turn on EPEL repo and type the following command yum command:
$ sudo yum install pssh
Sample outputs:

Fig.04: Installing pssh on RHEL/CentOS/Red Hat Enterprise Linux

Install pssh on Fedora Linux

Type the following dnf command:
$ sudo dnf install pssh
Sample outputs:

Fig.05: Installing pssh on Fedora

Install pssh on Arch Linux

Type the following command:
$ sudo pacman -S python-pip
$ pip install pssh

How to use pssh command

First you need to create a text file called hosts file from which pssh read hosts names. The syntax is pretty simple. Each line in the host file are of the form [user@]host[:port] and can include blank lines and comments lines beginning with “#”. Here is my sample file named ~/.pssh_hosts_files:
$ cat ~/.pssh_hosts_files
vivek@dellm6700
root@192.168.2.30
root@192.168.2.45
root@192.168.2.46
Run the date command all hosts:
$ pssh -i -h ~/.pssh_hosts_files date
Sample outputs:

[1] 18:10:10 [SUCCESS] root@192.168.2.46 Sun Feb 26 18:10:10 IST 2017 [2] 18:10:10 [SUCCESS] vivek@dellm6700 Sun Feb 26 18:10:10 IST 2017 [3] 18:10:10 [SUCCESS] root@192.168.2.45 Sun Feb 26 18:10:10 IST 2017 [4] 18:10:10 [SUCCESS] root@192.168.2.30 Sun Feb 26 18:10:10 IST 2017

Run the uptime command on each host:
$ pssh -i -h ~/.pssh_hosts_files uptime
Sample outputs:

[1] 18:11:15 [SUCCESS] root@192.168.2.45 18:11:15 up 2:29, 0 users, load average: 0.00, 0.00, 0.00 [2] 18:11:15 [SUCCESS] vivek@dellm6700 18:11:15 up 19:06, 0 users, load average: 0.13, 0.25, 0.27 [3] 18:11:15 [SUCCESS] root@192.168.2.46 18:11:15 up 1:55, 0 users, load average: 0.00, 0.00, 0.00 [4] 18:11:15 [SUCCESS] root@192.168.2.30 6:11PM up 1 day, 21:38, 0 users, load averages: 0.12, 0.14, 0.09

You can now automate common sysadmin tasks such as patching all servers:
$ pssh -h ~/.pssh_hosts_files -- sudo yum -y update
OR
$ pssh -h ~/.pssh_hosts_files -- sudo apt-get -y update
$ pssh -h ~/.pssh_hosts_files -- sudo apt-get -y upgrade

How do I use pssh to copy file to all servers?

The syntax is:
pscp -h ~/.pssh_hosts_files src dest
To copy $HOME/demo.txt to /tmp/ on all servers, enter:
$ pscp -h ~/.pssh_hosts_files $HOME/demo.txt /tmp/
Sample outputs:

[1] 18:17:35 [SUCCESS] vivek@dellm6700 [2] 18:17:35 [SUCCESS] root@192.168.2.45 [3] 18:17:35 [SUCCESS] root@192.168.2.46 [4] 18:17:35 [SUCCESS] root@192.168.2.30

Or use the prsync command for efficient copying of files:
$ prsync -h ~/.pssh_hosts_files /etc/passwd /tmp/
$ prsync -h ~/.pssh_hosts_files *.html /var/www/html/

How do I kill processes in parallel on a number of hosts?

Use the pnuke command for killing processes in parallel on a number of hosts. The syntax is:
$ pnuke -h .pssh_hosts_files process_name
### kill nginx and firefox on hosts:
$ pnuke -h ~/.pssh_hosts_files firefox
$ pnuke -h ~/.pssh_hosts_files nginx
See pssh/pscp command man pages for more information.

Conclusion

pssh is a pretty good tool for parallel SSH command execution on many servers. It quite is useful if you have 5 or 10 servers. Nevertheless, if you need to do something complicated you should look into Ansible and co.

Advertisement

30 Shades of “Alias” Command – UNIX

You can define various types aliases as follows to save time and increase productivity.

#1: Control ls command output

The ls command lists directory contents and you can colorize the output:

## Colorize the ls output ## alias ls='ls --color=auto'   ## Use a long listing format ## alias ll='ls -la'   ## Show hidden files ## alias l.='ls -d .* --color=auto'

#2: Control cd command behavior

## get rid of command not found ## alias cd..='cd ..'   ## a quick way to get out of current directory ## alias ..='cd ..' alias ...='cd ../../../' alias ....='cd ../../../../' alias .....='cd ../../../../' alias .4='cd ../../../../' alias .5='cd ../../../../..'

#3: Control grep command output

grep command is a command-line utility for searching plain-text files for lines matching a regular expression:

## Colorize the grep command output for ease of use (good for log files)## alias grep='grep --color=auto' alias egrep='egrep --color=auto' alias fgrep='fgrep --color=auto'

#4: Start calculator with math support

alias bc='bc -l'

#4: Generate sha1 digest

alias sha1='openssl sha1'

#5: Create parent directories on demand

mkdir command is used to create a directory:

alias mkdir='mkdir -pv'

#6: Colorize diff output

You can compare files line by line using diff and use a tool called colordiff to colorize diff output:

# install colordiff package 🙂 alias diff='colordiff'

#7: Make mount command output pretty and human readable format

alias mount='mount |column -t'

#8: Command short cuts to save time

# handy short cuts # alias h='history' alias j='jobs -l'

#9: Create a new set of commands

alias path='echo -e ${PATH//:/\\n}' alias now='date +"%T"' alias nowtime=now alias nowdate='date +"%d-%m-%Y"'

#10: Set vim as default

alias vi=vim alias svi='sudo vi' alias vis='vim "+set si"' alias edit='vim'

#11: Control output of networking tool called ping

# Stop after sending count ECHO_REQUEST packets # alias ping='ping -c 5' # Do not wait interval 1 second, go fast # alias fastping='ping -c 100 -s.2'

#12: Show open ports

Use netstat command to quickly list all TCP/UDP port on the server:

alias ports='netstat -tulanp'

#13: Wakeup sleeping servers

Wake-on-LAN (WOL) is an Ethernet networking standard that allows a server to be turned on by a network message. You can quickly wakeup nas devices and server using the following aliases:

## replace mac with your actual server mac address # alias wakeupnas01='/usr/bin/wakeonlan 00:11:32:11:15:FC' alias wakeupnas02='/usr/bin/wakeonlan 00:11:32:11:15:FD' alias wakeupnas03='/usr/bin/wakeonlan 00:11:32:11:15:FE'

#14: Control firewall (iptables) output

Netfilter is a host-based firewall for Linux operating systems. It is included as part of the Linux distribution and it is activated by default. This post list most common iptables solutions required by a new Linux user to secure his or her Linux operating system from intruders.

## shortcut for iptables and pass it via sudo# alias ipt='sudo /sbin/iptables'   # display all rules # alias iptlist='sudo /sbin/iptables -L -n -v --line-numbers' alias iptlistin='sudo /sbin/iptables -L INPUT -n -v --line-numbers' alias iptlistout='sudo /sbin/iptables -L OUTPUT -n -v --line-numbers' alias iptlistfw='sudo /sbin/iptables -L FORWARD -n -v --line-numbers' alias firewall=iptlist

#15: Debug web server / cdn problems with curl

# get web server headers # alias header='curl -I'   # find out if remote server supports gzip / mod_deflate or not # alias headerc='curl -I --compress'

#16: Add safety nets

# do not delete / or prompt if deleting more than 3 files at a time # alias rm='rm -I --preserve-root'   # confirmation # alias mv='mv -i' alias cp='cp -i' alias ln='ln -i'   # Parenting changing perms on / # alias chown='chown --preserve-root' alias chmod='chmod --preserve-root' alias chgrp='chgrp --preserve-root'

#17: Update Debian Linux server

apt-get command is used for installing packages over the internet (ftp or http). You can also upgrade all packages in a single operations:

# distro specific - Debian / Ubuntu and friends # # install with apt-get alias apt-get="sudo apt-get" alias updatey="sudo apt-get --yes"   # update on one command alias update='sudo apt-get update && sudo apt-get upgrade'

#18: Update RHEL / CentOS / Fedora Linux server

yum command is a package management tool for RHEL / CentOS / Fedora Linux and friends:

## distrp specifc RHEL/CentOS ## alias update='yum update' alias updatey='yum -y update'

#19: Tune sudo and su

# become root # alias root='sudo -i' alias su='sudo -i'

#20: Pass halt/reboot via sudo

shutdown command bring the Linux / Unix system down:

# reboot / halt / poweroff alias reboot='sudo /sbin/reboot' alias poweroff='sudo /sbin/poweroff' alias halt='sudo /sbin/halt' alias shutdown='sudo /sbin/shutdown'

#21: Control web servers

# also pass it via sudo so whoever is admin can reload it without calling you # alias nginxreload='sudo /usr/local/nginx/sbin/nginx -s reload' alias nginxtest='sudo /usr/local/nginx/sbin/nginx -t' alias lightyload='sudo /etc/init.d/lighttpd reload' alias lightytest='sudo /usr/sbin/lighttpd -f /etc/lighttpd/lighttpd.conf -t' alias httpdreload='sudo /usr/sbin/apachectl -k graceful' alias httpdtest='sudo /usr/sbin/apachectl -t && /usr/sbin/apachectl -t -D DUMP_VHOSTS'

#22: Alias into our backup stuff

# if cron fails or if you want backup on demand just run these commands # # again pass it via sudo so whoever is in admin group can start the job # # Backup scripts # alias backup='sudo /home/scripts/admin/scripts/backup/wrapper.backup.sh --type local --taget /raid1/backups' alias nasbackup='sudo /home/scripts/admin/scripts/backup/wrapper.backup.sh --type nas --target nas01' alias s3backup='sudo /home/scripts/admin/scripts/backup/wrapper.backup.sh --type nas --target nas01 --auth /home/scripts/admin/.authdata/amazon.keys' alias rsnapshothourly='sudo /home/scripts/admin/scripts/backup/wrapper.rsnapshot.sh --type remote --target nas03 --auth /home/scripts/admin/.authdata/ssh.keys --config /home/scripts/admin/scripts/backup/config/adsl.conf' alias rsnapshotdaily='sudo /home/scripts/admin/scripts/backup/wrapper.rsnapshot.sh --type remote --target nas03 --auth /home/scripts/admin/.authdata/ssh.keys --config /home/scripts/admin/scripts/backup/config/adsl.conf' alias rsnapshotweekly='sudo /home/scripts/admin/scripts/backup/wrapper.rsnapshot.sh --type remote --target nas03 --auth /home/scripts/admin/.authdata/ssh.keys --config /home/scripts/admin/scripts/backup/config/adsl.conf' alias rsnapshotmonthly='sudo /home/scripts/admin/scripts/backup/wrapper.rsnapshot.sh --type remote --target nas03 --auth /home/scripts/admin/.authdata/ssh.keys --config /home/scripts/admin/scripts/backup/config/adsl.conf' alias amazonbackup=s3backup

#23: Desktop specific – play avi/mp3 files on demand

## play video files in a current directory ## # cd ~/Download/movie-name # playavi or vlc alias playavi='mplayer *.avi' alias vlc='vlc *.avi'   # play all music files from the current directory # alias playwave='for i in *.wav; do mplayer "$i"; done' alias playogg='for i in *.ogg; do mplayer "$i"; done' alias playmp3='for i in *.mp3; do mplayer "$i"; done'   # play files from nas devices # alias nplaywave='for i in /nas/multimedia/wave/*.wav; do mplayer "$i"; done' alias nplayogg='for i in /nas/multimedia/ogg/*.ogg; do mplayer "$i"; done' alias nplaymp3='for i in /nas/multimedia/mp3/*.mp3; do mplayer "$i"; done'   # shuffle mp3/ogg etc by default # alias music='mplayer --shuffle *'

#24: Set default interfaces for sys admin related commands

vnstat is console-based network traffic monitor. dnstop is console tool to analyze DNS traffic. tcptrack and iftop commands displays information about TCP/UDP connections it sees on a network interface and display bandwidth usage on an interface by host respectively.

## All of our servers eth1 is connected to the Internets via vlan / router etc ## alias dnstop='dnstop -l 5 eth1' alias vnstat='vnstat -i eth1' alias iftop='iftop -i eth1' alias tcpdump='tcpdump -i eth1' alias ethtool='ethtool eth1'   # work on wlan0 by default # # Only useful for laptop as all servers are without wireless interface alias iwconfig='iwconfig wlan0'

#25: Get system memory, cpu usage, and gpu memory info quickly

## pass options to free ## alias meminfo='free -m -l -t'   ## get top process eating memory alias psmem='ps auxf | sort -nr -k 4' alias psmem10='ps auxf | sort -nr -k 4 | head -10'   ## get top process eating cpu ## alias pscpu='ps auxf | sort -nr -k 3' alias pscpu10='ps auxf | sort -nr -k 3 | head -10'   ## Get server cpu info ## alias cpuinfo='lscpu'   ## older system use /proc/cpuinfo ## ##alias cpuinfo='less /proc/cpuinfo' ##   ## get GPU ram on desktop / laptop## alias gpumeminfo='grep -i --color memory /var/log/Xorg.0.log'

#26: Control Home Router

The curl command can be used to reboot Linksys routers.

# Reboot my home Linksys WAG160N / WAG54 / WAG320 / WAG120N Router / Gateway from *nix. alias rebootlinksys="curl -u 'admin:my-super-password' 'http://192.168.1.2/setup.cgi?todo=reboot'"   # Reboot tomato based Asus NT16 wireless bridge alias reboottomato="ssh admin@192.168.1.1 /sbin/reboot"

#27 Resume wget by default

The GNU Wget is a free utility for non-interactive download of files from the Web. It supports HTTP, HTTPS, and FTP protocols, and it can resume downloads too:

## this one saved by butt so many times ## alias wget='wget -c'

#28 Use different browser for testing website

## this one saved by butt so many times ## alias ff4='/opt/firefox4/firefox' alias ff13='/opt/firefox13/firefox' alias chrome='/opt/google/chrome/chrome' alias opera='/opt/opera/opera'   #default ff alias ff=ff13   #my default browser alias browser=chrome

#29: A note about ssh alias

Do not create ssh alias, instead use ~/.ssh/config OpenSSH SSH client configuration files. It offers more option. An example:

Host server10 Hostname 1.2.3.4 IdentityFile ~/backups/.ssh/id_dsa user foobar Port 30000 ForwardX11Trusted yes TCPKeepAlive yes

You can now connect to peer1 using the following syntax:
$ ssh server10

#30: It’s your turn to share…

## set some other defaults ## alias df='df -H' alias du='du -ch'   # top is atop, just like vi is vim alias top='atop'   ## nfsrestart - must be root ## ## refresh nfs mount / cache etc for Apache ## alias nfsrestart='sync && sleep 2 && /etc/init.d/httpd stop && umount netapp2:/exports/http && sleep 2 && mount -o rw,sync,rsize=32768,wsize=32768,intr,hard,proto=tcp,fsc natapp2:/exports /http/var/www/html && /etc/init.d/httpd start'   ## Memcached server status ## alias mcdstats='/usr/bin/memcached-tool 10.10.27.11:11211 stats' alias mcdshow='/usr/bin/memcached-tool 10.10.27.11:11211 display'   ## quickly flush out memcached server ## alias flushmcd='echo "flush_all" | nc 10.10.27.11 11211'   ## Remove assets quickly from Akamai / Amazon cdn ## alias cdndel='/home/scripts/admin/cdn/purge_cdn_cache --profile akamai' alias amzcdndel='/home/scripts/admin/cdn/purge_cdn_cache --profile amazon'   ## supply list of urls via file or stdin alias cdnmdel='/home/scripts/admin/cdn/purge_cdn_cache --profile akamai --stdin' alias amzcdnmdel='/home/scripts/admin/cdn/purge_cdn_cache --profile amazon --stdin'

Error :- sudo: effective uid is not 0, is sudo installed setuid root?

We all as a Linux administrator must have come across this error sometime in our lives.

[user@host dir]$ sudo bash
sudo: effective uid is not 0, is sudo installed setuid root?

This happens when sudo does not get the right access permissions.

The Solution for this error is giving the following permissions as root user

chmod u+s /usr/bin/sudo

That Must Sort the issue for CentOS Kind of distros.

 

/proc/sys for you to manipulate a running kernel

The /proc/sys directory in the /proc virtual filesytem contains a lot of useful and interesting files and directories. Many kernel settings can be manipulated by writing to files in the proc filesystem. A lot of important information can be retrieved from these files. This is especially useful when you are troubleshooting or fine tuning your linux system.

Following is a description of the most important files.

Especially the files in /proc/sys/vm are very interesting and useful.

You can also use the sysctl command to make this changes persistent, or to see all the possible kernel options you can change at run-time.

/proc/sys/dev

Contains device specific information. For instance the directory cdrom /proc/sys/dev/cdrom/info shows you cdrom capabilities. The other files in /proc/sys/dev/cdrom are writable and allow you to actually set options for your cdrom drive.

For instance echo 1 > /proc/sys/dev/cdrom/autoeject makes your tray open automagically when you unmount your cdrom.

/proc/sys/dev/parport holds information about parallel ports. Browse these directories to learn more about their contents.

/proc/sys/fs

Virtual filesystem information/tuning

/proc/sys/fs/binfmt_misc

binfmt_misc allows you to configure the system to execute miscellaneous binary formats. For instance it enables you to make the system execute .exe files using wine and java files using the java interpreter, just by typing a file name.

/proc/sys/fs/dentry-state

Linux caches directory access to speed up sub-sequential access to the same directory, this file contains information about the status of the directory cache.

/proc/sys/fs/dir-notify-enable

enable/disable dnotify interface, dnotify is a signal used to notify a process about file/directory changes. This is mainly interesting to programmers.

/proc/sys/fs/dquot-nr

number of allocated disk quota entries and the number of free disk quota entries

/proc/sys/fs/dquot-max

maximum number of cached disk quota entries.

/proc/sys/fs/file-max

system-wide limit on the number of open files for all processes.

/proc/sys/fs/file-nr

number of files the system has presently opened.

/proc/sys/fs/inode-max

maximum number of in-memory inodes

/proc/sys/fs/inode-nr

number of inodes and number of free inodes

/proc/sys/fs/inode-state

This file contains seven numbers: number of inodes, number of free inodes, preshrink, and four dummy values. nr_inodes is the number of inodes the system has allocated. Preshrink is non-zero when the nr_inodes is bigger than inode-max.

/proc/sys/fs/inotify
(since kernel 2.6.13)

This directory contains files that can be used to limit the amount of kernel memory consumed by the inotify interface.

/proc/sys/fs/lease-break-time

This file specifies the grace period that the kernel grants to a process holding a file lease after it has sent a signal to that process notifying it that another process is waiting to open the file.

/proc/sys/fs/leases-enable

This file can be used to enable or disable file leases on a system-wide basis.

/proc/sys/fs/mqueue
(since kernel 2.6.6)

This directory contains files controlling the resources used by POSIX message queues.

/proc/sys/fs/overflowgid

Allows you to change the value of the fixed GID, if a filesystem is mounted which only supports 16 bit GID’s the Linux GID’s which are 32 bit sometimes need to be converted to lower values this is fixed at this value.

/proc/sys/fs/overflowuid

Allows you to change the value of the fixed UID, if a filesystem is mounted which only supports 16 bit UID’s the Linux UID’s which are 32 bit sometimes need to be converted to lower values this is fixed at this value.

/proc/sys/fs/suid_dumpable
(since kernel 2.6.13)

Determines whether core dump files are produced for set-user-ID or otherwise protected/tainted binaries.

Possible values are 0,1,2:

0 (default) A core dump will not be produced for a process which has changed credentials or whose binary does not have read permission enabled.

1 (debug) All processes dump core when possible.

2 (suidsafe) Any binary which normally would not be dumped is dumped readable by root only. This allows the user to remove the core dump file but not to read it. For security reasons core dumps in this mode will not overwrite one another or other files. This mode is appropriate when administrators are attempting to debug problems in a normal environment.

/proc/sys/fs/super-max

Controls the maximum number of superblocks, and thus the maximum number of mounted file systems the kernel can have.

/proc/sys/fs/super-nr

The number of file systems currently mounted.

/proc/sys/kernel/acct

highwater, lowwater, and frequency. Used with BSD-style process accounting.

/proc/sys/kernel/cap-bound
(from Linux 2.2 to 2.6.24)

Holds the value of the kernel capability bounding set.

/proc/sys/kernel/core_pattern

Can be used to define a template for naming core dump files

/proc/sys/kernel/core_uses_pid

See core(5).

/proc/sys/kernel/ctrl-alt-del

Controls the handling of Ctrl-Alt-Del from the keyboard.

If it’s 0, Linux will do a graceful restart. When the value is > 0, Linux’s will do an immediate reboot, without even syncing its dirty buffers.

/proc/sys/kernel/hotplug

Contains the path for the hotplug policy agent. Can be used to set the NIS/YP domainname

/proc/sys/kernel/domainname

can be used to set the NIS/YP domainname

/proc/sys/kernel/hostname

can be used to set the hostname

/proc/sys/kernel/modprobe

Contains the path for the kernel module loader.

/proc/sys/kernel/msgmax

This file defines a system-wide limit specifying the maximum number of bytes in a single message written on a System V message queue.

/proc/sys/kernel/msgmnb

Defines a system-wide parameter used to initialize the msg_qbytes setting for subsequently created message queues.

/proc/sys/kernel/ostype and /proc/sys/kernel/osrelease

These files give substrings of /proc/version.

/proc/sys/kernel/overflowgid and /proc/sys/kernel/overflowuid

These files duplicate the files /proc/sys/fs/overflowgid and /proc/sys/fs/overflowuid.

/proc/sys/kernel/panic

Gives read/write access to the kernel variable panic_timeout. If this is zero, the kernel will loop on a panic; if non-zero it indicates that the kernel should autoreboot after this number of seconds.

/proc/sys/kernel/panic_on_oops
(since kernel 2.5.68)

This file controls the kernel’s behavior when an oops or BUG is encountered. If this file contains 0, then the system tries to continue operation. If it contains 1, then the system delays a few seconds and then panics. If the /proc/sys/kernel/panic file is also non-zero then the machine will be rebooted.

/proc/sys/kernel/pid_max
(since kernel 2.5.34)

This file specifies the value at which PIDs wrap around (i.e., the value in this file is one greater than the maximum PID).

/proc/sys/kernel/printk

The four values in this file are console_loglevel, default_message_loglevel, minimum_console_level, and default_console_loglevel. This allows configuration of which messages will be logged to the console. (ever worked on a console printing messages all the time to your screen? Here’s how to fix that) Messages with a higher priority than console_loglevel will be printed to the console.

/proc/sys/kernel/pty
(since kernel 2.6.4)

This directory contains two files relating to the number of Unix 98 pseudo-terminals on the system.

/proc/sys/kernel/pty/max

Defines the maximum number of pseudo-terminals.

/proc/sys/kernel/pty/nr

This read-only file indicates how many pseudo-terminals are currently in use.

/proc/sys/kernel/random

This directory contains various parameters controlling the operation of the file /dev/random.

/proc/sys/kernel/real-root-dev

Used by the deprecated change_root initrd system

/proc/sys/kernel/rtsig-max

( until kernel 2.6.7)

Can be used to tune the maximum number of POSIX real-time (queued) signals that can be outstanding in the system.

/proc/sys/kernel/rtsig-nr

(until kernel 2.6.7)

This file shows the number POSIX real-time signals currently queued.

/proc/sys/kernel/sem
(since kernel 2.4)

Contains 4 numbers defining limits for System V IPC semaphores.

/proc/sys/kernel/sg-big-buff

Shows the size of the generic SCSI device (sg) buffer.

/proc/sys/kernel/shmall

Contains the system-wide limit on the total number of pages of System V shared memory.

/proc/sys/kernel/shmmax

This file can be used to query and set the run-time limit on the maximum (System V IPC) shared memory segment size that can be created.

/proc/sys/kernel/shmmni
(from kernel 2.4)

Specifies the system-wide maximum number of System V shared memory segments that can be created.

/proc/sys/kernel/version

Kernel version number and build date

/proc/sys/net

Networking information.

/proc/sys/net/core/somaxconn

Defines a ceiling value for the backlog argument of listen

/proc/sys/net/core/rmem_max

Maximum TCP Receive Window.

/proc/sys/net/core/wmem_maxx

Maximum TCP Send Window.

/proc/sys/net/ipv4/ip_forward

Enable or disable routing.

/proc/sys/sunrpc

This directory supports Sun remote procedure call for network file system (NFS).

/proc/sys/vm

This directory contains files for memory management tuning, buffer and cache management. One of the more interresting directories in proc sys as it allows manipulating memory handling in real time.

/proc/sys/vm/swappiness
(since kernel 2.6.16)

vm.swappiness takes a value between 0 and 100 to change the balance between swapping applications and freeing cache. At 100, the kernel will always prefer to find inactive pages and swap them out; in other cases, whether a swapout occurs depends on how much application memory is in use and how poorly the cache is doing at finding and releasing inactive items.

/proc/sys/vm/drop_caches
(since kernel 2.6.16)

Writing to this file causes the kernel to drop clean caches, dentries and inodes from memory, causing that memory to become free.

To free pagecache, write 1 to this file.

To free dentries and inodes, write 2 to this file.

To free pagecache, dentries and inodes, write 3 to this file.

Just try echo 1 > /proc/sys/vm/drop_caches, and watch your memory usage drop by all kernel cache memory.

/proc/sys/vm/legacy_va_layout
(since kernel 2.6.9)

If non-zero, this disables the new 32-bit memory-mapping layout; the kernel will use the legacy (2.4) layout for all processes.

/proc/sys/vm/oom_dump_tasks
(since kernel 2.6.25)

Enables a system-wide task dump (excluding kernel threads) to be produced when the kernel performs an OOM-killing. The dump includes the following information for each task (thread, process): thread ID, real user ID, thread group ID (process ID), virtual memory size, resident set size, the CPU that the task is scheduled on, oom_adj score (see the description of /proc[number]/oom_adj), and command name. This is helpful to determine why the OOM-killer was invoked and to identify the rogue task that caused it.

If this contains the value zero, this information is suppressed.

It defaults to 0, so if you have a problem requiring it, enable it :

echo 1 > /proc/sys/vm/oom_dump_tasks

/proc/sys/vm/oom_kill_allocating_task
(since kernel 2.6.24)

This enables or disables killing the OOM-triggering task in out-of-memory situations. If this is set to zero, the OOM-killer will scan through the entire tasklist and select a task based on heuristics to kill. This normally selects a rogue memory-hogging task that frees up a large amount of memory when killed.

If this is set to non-zero, the OOM-killer simply kills the task that triggered the out-of-memory condition. This avoids a possibly expensive tasklist scan.

If /proc/sys/vm/panic_on_oom is non-zero, it takes precedence over whatever value is used in /proc/sys/vm/oom_kill_allocating_task.

The default value is 0.

/proc/sys/vm/overcommit_memory

This file contains the kernel virtual memory accounting mode. Values are:

0: heuristic overcommit (default)

1: always overcommit, never check

2: always check, never overcommit

/proc/sys/vm/overcommit_ratio

Value used in calculating virtual address space

/proc/sys/vm/panic_on_oom
(since kernel 2.6.18)

This enables or disables a kernel panic in an out-of-memory situation.

0 (default) : no panic

1 : panic but not if a process limits allocations to certain nodes using memory

policies mbind or cpusets and those nodes reach memory exhaustion status.

2 : always panic

Linux Concepts :- Quotas

Rules : 1. Quotas can only be created for partitions

Eg., If a 1MB quota is set for partition [/home]

then every subdir under that /home can use a max of 1 MB

user | group

block [diskspace] inode [no of files] | block [diskspace] inode [files]

SOFT HARD GRACE SOFT HARD GRACE | SOFT HARD GRACE SOFT HARD GRACE

<—- Limits —->

==========================================

Part I. Configuring / Setting Up Quotas

==========================================

1. Configure /etc/fstab – usrquota or grpquota which ever you want

In the 4th column of /etc/fstab add ‘usrquota’

2. Refresh /etc/fstab – actually /etc/mtab :

a. If you do not want hassles just REBOOT and skip to Part II.

or

b. mount -a which will do nothing if FS’s are already mounted, which

they always are

or

c. umount /home and mount /home

But this will not work if any users are online which they always are

or

d. mount -o remount /home <<<=========================

[mount -o remount,rw /]

Refresh and will work even if users are online

or

e. reboot

which is the coward’s way

and then Check /etc/mtab or mount*

3. To create the aquota.user file

–> quotacheck -vc /home [force] – Note : ‘u’ is the default if not given

and ‘v’, of course, is always

optional but friendly

This will create the /home/aquota.user which monitors all

quota activity for the /home paritition

4. – To turn on quotas

–> quotaon -v /home – To turn on quotas

or load /home/aquota.user file

in to RAM

5. Check everybody’s current usage and quotas :

# repquota -a

Configuring quotas is now over !

Now we will implement quotas.

In short :

1. Configure /etc/fstab

2. mount -o remount /home Cross check with # cat /etc/mtab

3. quotacheck -vc /home Creates aquota.user

4. quotaon -v /home Loads aquota.user in RAM

5. repquota -a Enjoy your work !

eg

1. repquota -a

2. repquota -u /

3. repquota -u sachin

The first line shows quota info for all users and groups for all file

systems.

The second line shows user quota info for the / file system.

The third line shows quota information for user sachin on all file systems.

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

=============================

Part II. Implementing Quotas

=============================

6. edquota -u user

7. edquota -p foo bar <——— use foo as quota prototype for bar

8. edquota -t <———— To change the grace period

or use :

# edquota -p foo `awk -F: ‘$3 > 499 {print $1}’ /etc/passwd`

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

========================================

Part III. Repairing the aquota.user file

========================================

If your system hangs and then restarts, the aquota.user file gets corrupted

and all quotas for all users are now in an unknown state :

To repair, boot into single user mode imme-asap, and do this _FIRST_ :

quotaoff -v /home

quotacheck -avug Minimum reqd is : quotacheck /home

since ‘u’ is the default

and we do not have ‘g’

and ‘v’ is optional

a is check /etc/mtab

Re-Creates file /home/aquota.user

quotaon -v /home

Misc : /etc/warnquota.conf

warnquota*

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

GNU/Linux LDUP : 27-Jul-2k3

PART II – SysAdministration

08. QUOTAS

1. What are the two aspects of disk storage that quotas allow you specify?

A: Disk space [Block] and Files [inode] quotas

2. Which init script checks for the presence or absence of quotas ?

A: /etc/rc.d/rc.sysinit

3. I wish to implement quotas on my /home dir ? Should /home be a partition?

A: Yes.

4. Which file is configured when setting up quotas ?

A: /etc/fstab

5. What is the min I have to do, to implement both user and group quotas for

my /home partition?

A: Configure /etc/fstab with :

LABEL=/home /home ext3 defaults,usrquota,grpquota 1 2

and then just reboot the machine !!

6. But I wish to to implement only user quotas, is this /etc/fstab OK ?

LABEL=/home /home ext3 defaults, usrquota 1 2

A: No. No space in 4th field after defaults

7. Is this /etc/fstab OK ?

LABEL=/home /home ext3 defaults,userquota 1 2

A: No. Its usrquota.

8. Where is this quota info for user and group quotas stored for /home?

A: In the /home partition, in 2 data files : aquota.user, aquota.group

9. Which file keeps track of all the mounted filesystems ?

A: /etc/mtab

10 How would you implement user quotas without rebooting your machine ?

A: Configure /etc/fstab with :

1. LABEL=/home /home ext3 defaults,usrquota 1 2

2. mount -o remount /home

3. quotacheck -vc /home

4. quotaon -v /home

11 Why would you want to do a quotaoff before you do a quotacheck manually

from the CLI ?

A: Corrupts the aquota.user file

12 Can a user – foo – modify/create his quota ?

A: Obviously not! Only root can do that! Or else every user will

abuse the quota system for his benefit !

13 Can foo at least see his quota status ?

A: Yes.

14 How ?

A: Login as foo and run ‘quota’.

15. What are these quotas that he will see ?

A: His soft limit, hard limit and grace period

16. How then will root set [create/modify] quotas for ‘foo’ ?

A: edquota -u foo

17. Examine the following o/p generated by the quota command given by foo:

Disk quotas for user foo (uid 500):

Filesystem blocks soft hard inodes soft hard

/dev/hda 20 100 0 14 0 0

18. How much disk space has foo already used ?

A: 20 blocks i.e. 20 KB

19. His soft limit appears to be 100 KB. Can he use 130 KB ?

A: Yes. But but he will get warning messages

20. Can he use unlimited disk space and fill the partition ?

A: Yes. But up and until the grace period expires. After that the soft limit

is enforced as the hard limit. Additionally, he will get warning messages

21. Then what is the use of this soft limit ? How can I remedy it ?

A: By giving a hard limit too. foo can never cross the hard limit.

22. Now I do this :

Disk quotas for user foo (uid 500):

Filesystem blocks soft hard inodes soft hard

/dev/hda 20 100 200 14 0 0

foo creates files worth 160 KB. What will happen then ?

A: He will be allowed to create up to 200 KB max. Also after 7 days

he will be shut down regardless of whether he has reached his hard

limit or not. He will have to clean up under the soft limit to work.

23. What command is used to change a user’s grace period?

A: edquota -t

24. What command is used to see the entire quota details of all users?

A: repquota -a

25. What command sets a quota template?

A: edquota -p

26. What does ‘p’ mean and how would you use it?

A: ‘prototype’.

Suppose ‘foo’ has his quota set. Then you could clone his details,

# edquota -p foo bar

bar now has the same quota limits as foo.

27. If you had 2000 users, the above would clearly be inconvenient. Solve!

a: edquota -p foo `awk -F: ‘$3 > 499 { print $1 }’ /etc/passwd`

28. An over-limit quota generates a mail message to the user on login.

Which file would you modify to customize the mail delivered ?

A: /etc/warnquota.conf

29. If quotas were run as a daily cron job, where would you find the script

file concerned?

a: /etc/crond.daily/

30 A user owns 150 inodes; the soft limit is 100 and the hard is 200.

Which of the following is correct if the grace period has not expired?

a. The user can create no more files

b. The user cannot append data to an existing file

c. The user cannot log off without deleting some files

d. The user will receive an email notice of violation

A: d.

31 What is the purpose of ‘convertquota’ ?

A: convertquota converts old quota files quota.user and quota.group to

files aquota.user and aquota.group in new format currently used by

2.4.0-ac and newer or by Red Hat Linux 2.4 kernels on filesystem.

New file format allows using quotas for 32-bit uids / gids, setting

quotas for root, accounting used space in bytes (and so allowing use of

quotas in ReiserFS) and it is also architecture independent. This format

introduces Radix Tree (a simple form of tree structure) to quota file.