Working with Linux Shell Limits
-
One of the most commonly used, but poorly understood, concepts in Linux is the concept around shell limits. Shell limits are system limits imposed, not by the kernel or the operating system itself, but entirely by the user’s shell. Shell limits are very handy as they can be used to restrict any single instance of a shell from consuming an inordinate number of resources. Think of it as a safety check. If you do something in your shell that spawns far too many processes, the shell will cap the damage that you can do to your system. It is designed to protect you from yourself.
It is very important to remember that the limits are imposed by the running instance of a shell (BASH, ZSH, KSH, CSH, etc.) and not by the OS. The limits are not system wide but apply to each running shell individually. So if my shell allows for 1,024 file handles to be open and I run out, I can always open a new shell and open 1,024 more. And just because I run out does not mean that someone else using the same machine will have any issues as they have their own, personal shell limit. Shell limits don’t stop malicious behaviour, they stop accidents.
Since we are working with Linux we will look exclusively at BASH and how BASH handles limits. Most shells work nearly identically but there are some differences.
Working with shell limits is done using one standard configuration file, /etc/security/limits.conf, and through the ulimit command. The limits.conf configuration file allows you to set persistent shell limits for individual users as well as for system defaults to be applied to all non-listed users. The ulimit command shows current settings for a shell instance and allows that shell to manipulate limit values within a permitted range.
Limits have a soft and a hard limit. The soft limit can be thought of as the “current” limit. When ulimit reports a soft limit, that is the current limit for that particular value. The shell can manipulate its soft limit up to the hard limit value but no further. The hard limit is the “maximum” that a shell can set on its own. Soft and hard limits are often set to be the same for ease of use as very few users or processes would want to manipulate limit values manually. When setting a soft limit in the limits.conf file, we can think of this as the “default” value that a shell will see without further change and the hard limit again being the max.
Let’s begin by looking into the current soft limits of the system. Soft limits are what affect our shell so this is what we care about most of the time. When requesting status from ulimit we can ask for soft limits “S” or hard limits “H”. If we do not specify which we want, ulimit will default to soft limits (a.k.a. current or working limits.) So in this case we could use the -a or the -aS flags to denote the same thing. The -a flag requests “all” settings.
# ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited file size (blocks, -f) unlimited pending signals (-i) 1024 max locked memory (kbytes, -l) 32 max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 stack size (kbytes, -s) 10240 cpu time (seconds, -t) unlimited max user processes (-u) 262143 virtual memory (kbytes, -v) unlimited file locks (-x) unlimitedThis provides us with a view into the limits that will affect our current, running shell. Now, we can use the -aH flags to see what the system will allow us to set these values to at a maximum.
# ulimit -aH core file size (blocks, -c) unlimited data seg size (kbytes, -d) unlimited file size (blocks, -f) unlimited pending signals (-i) 1024 max locked memory (kbytes, -l) 32 max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 stack size (kbytes, -s) unlimited cpu time (seconds, -t) unlimited max user processes (-u) 262143 virtual memory (kbytes, -v) unlimited file locks (-x) unlimitedFor the most part, we can see that the soft limits are the same as the hard limits. But in some cases, such as the “core file size” and the “stack size” the hard limit is unlimited while a definite limit is placed upon us by default. Handily, the output of “ulimit -a” not only provides us with the values for these settings but also displays the necessary flags to use in order to manipulate individual variables. Let us look at “core file size”, a common limit to be modified. From our existing output, we can see that our current value is zero (or disabled) and that our maximum allowed value is unlimited and that the flag for manipulating this value is “-c”. If we run “ulimit -c” without specifying a value, we will get the the current setting for that one value. This can be very handy in scripts. And if we run “ulimit -Hc” we will find the maximum allowed value. Notice that the “H” must precede the “c” in this case or we get an error.
# ulimit -c 0 #ulimit -Hc unlimitedNow we can set our own value here up to the limit specified by “-Hc” which, in this case, is unlimited.
# ulimit -c 1024 # ulimit -c 1024 # ulimit -c unlimited # ulimit -c unlimitedNow that we know how to work with the ulimit command to change settings within our own shell, we will look at the limits.conf file to see how we can modify the allowable limits across all shells.
The limits.conf file has a lot of information in it as to how the file can be configured. But the basics are pretty simple. Each limit setting line consists of four fields: domain, type, item and value.
The first field contains the name of the user or group that will be affected by the setting. For example, my user is samiller and by group is user, a username can just be entered or a group can be specified by leading with the @ sign. You can also use a * as a wildcard to denote “everyone” which effectively sets system-wide settings.
The second field, type, is easy. Type is either hard or soft. Hard, as we said above, is the maximum allowable limit. The soft, when used here, denotes the “default” value that the shell will receive when it starts. This is what we will see with “ulimit -a” if we have not changed anything manually within the shell.
The third field is item. This field contains the name of the variable which we want to set. The complete list of items is available in the limits.conf file itself. Common items include nofile, core and nice.
The fourth and final field is value. This is, in case it wasn’t obvious, where you set the actual limit for the given settings.
In the following example, we will set the core values for the default user, my user group and for myself. I will set both soft and hard limits for each – although this is not necessary. For the default and user group I will set soft and hard different and for myself I will make them the same so that I can get unlimited cores by default when running a process as myself. I also have, in the final two lines, configuration to modify the number of open files values for my own user as well.
* soft core 0 * hard core 0 @user soft core 0 @user hard core 16384 samiller soft core unlimited samiller hard core unlimited samiller soft nofile 1024 samiller hard nofile 2048Changes to the limits.conf file will take effect as soon as a new shell is initiated. It will not affect running shells. And that is about all that you need to know about Linux BASH Shell Limits.
Originally found on my Linux blog: https://web.archive.org/web/20140823021548/http://www.scottalanmiller.com/linux/2011/01/26/working-with-linux-shell-limits/
