What are the recommendations for 10 Gigabit network adapter tuning parameters?
https://access.redhat.com/solutions/127143
What are the recommendations for 10 Gigabit network adapter tuning parameters?
SOLUTION 已验证 - 已更新 2017年三月20日01:07 -
环境
- Red Hat Enterprise Linux 7
- Red Hat Enterprise Linux 6
- Red Hat Enterprise Linux 5
- 10 Gigabit Ethernet network interface adapters (10GbE)
问题
- What are the recommendations for 10 Gigabit network adapter tuning parameters?
- 10Gbit NIC with MTU of 9000. Testing using
iperfbetween two systems only results in 6-7 gigabits/s transfer speed. - What are the expected and recommended tuning parameters to configure to achieve 10Gbps connection wirespeed for streaming bulk transfers?
决议
The following parameters are very general recommendations to achieve faster streaming transfers over 10 Gigabit Ethernet.
Depending on the actual production use of the network, these settings may not be optimal for your usage, however they will certainly make artificial benchmarks like iperf run faster.
Interrupt Balancing
Ensure the irqbalance service is installed and running.
Ring Buffer
It is recommended to make the NIC receive and transmit ring buffers as large as possible to prevent packet drops.
The procedure to increase the buffer size is described at:
The change can be persisted across reboots by following:
Socket Buffers
Increasing the maximum socket buffer size to 16MiB allows buffers to grow larger if required.
The following changes can be applied in /etc/sysctl.conf and applied immediately with sysctl -p:
# set min/default/max TCP read buffer
net.ipv4.tcp_rmem = 4096 87380 16777216
# set min/default/max TCP write buffer
net.ipv4.tcp_wmem = 4096 16384 16777216
# set maximum non-TCP receive socket buffer size
net.core.rmem_max = 16777216
# set maximum non-TCP write socket buffer size
net.core.wmem_max = 16777216
Increasing the default socket buffer sizes is not required as this basic level of tuning as it may harm efficiency of non-streaming transfers.
Network Budget
If the third column of /proc/net/softnet_stat is increasing, eg:
# cat /proc/net/softnet_stat
0073d76b 00000000 000049ae 00000000 00000000 00000000 00000000 00000000 00000000 00000000
Then there could be benefit to allowing the kernel packet receive handler to run for a slightly longer time.
The default value is 300 meaning 300 packets are received before the handler ends. This can be increased, though there is unlikely to be much improvement increasing this past several thousand.
The following change can be applied in /etc/sysctl.conf and applied immediately with sysctl -p:
# set number of packets softirq receives off NIC before exiting
net.core.netdev_budget = 1000
Further Reading
The following documents can assist in more precise tuning:
上一篇： 解决MySQL [Err] 1055