Working with NIC Ring Buffers in Linux

scottalanmiller

Making configuration changes to the settings of a network interface card is not one that we typically do on an average day but it can be an important component of network performance tuning. The most common change that we might need to make is to increase the receive (RX) ring buffer size. Most often we will find that the ring buffer is set rather small and might need to be increased on systems that are receiving a high volume of network traffic.

We can examine our current and available NIC settings using the ethtool command combined with the -g flag.

# ethtool -g eth0
Ring parameters for eth0:
Pre-set maximums:
RX:             2040
RX Mini:        0
RX Jumbo:       8160
TX:             255
Current hardware settings:
RX:             255
RX Mini:        0
RX Jumbo:       0
TX:             255

Here we have two sections in our output. The first section is “Pre-set maximums” which tells us the maximum values that could be set for each available parameter. The second section shows us to what each parameter is currently set. We are most interested in the top most parameter labeled simply “RX” which is our receive ring buffer.

Buffers are generally tuned small for latency reasons. The smaller the buffer the lower the latency. But low latency comes at a price and that price is maximum throughput. For greater throughput we need a larger buffer. Factory defaults are good, generally, for most systems but don’t be afraid to tune this for your own scenario.

To modify the current parameter settings we use the -G flag as opposed to the -g flag used to read them. A common setting is to set the RX parameter to 1020. Here is the command to do that:

# ethtool -G eth0 rx 1020

The change will take affect immediately and requires no restart to your system or even your network stack. Read your current settings again with -g to verify the change.

Remember that these changes made with ethtool are being made to the network card itself and not to the operating system. We are not changing the kernel network stack parameters but the NIC parameters in the firmware.

Originally found on my Linux blog: https://web.archive.org/web/20140813032509/http://www.scottalanmiller.com/linux/2011/06/20/working-with-nic-ring-buffers/