Nginx 性能优化之net.core.somaxconn

web应用中listen函数的backlog默认会给我们内核参数的net.core.somaxconn限制到128,而nginx定义的NGX_LISTEN_BACKLOG默认为511,所以有必要调整这个值。如:

net.core.netdev_max_backlog = 262144

我们线上服务器net.core.somaxconn都是默认的128,这个参数会影响到所有AF_INET类型socket的listen队列
Man 2 listen可以知道:

int listen(int s, int backlog);

The  backlog  parameter  defines the maximum length the queue of pending connections may grow to.  If a connection request

       arrives with the queue full the client may receive an error with an indication of ECONNREFUSED or, if the underlying  pro-

       tocol supports retransmission, the request may be ignored so that retries succeed.

BUGS

       If the socket is of type AF_INET, and the backlog argument is greater than the constant SOMAXCONN  (128  in  Linux  2.0  &

       2.2),  it  is silently truncated to SOMAXCONN.

也就是说,web应用中listen函数的backlog会给我们内核参数的net.core.somaxconn 限制到128,在高突发的请求中可能会导致链接超时或者触发重传

比如nginx 定义NGX_LISTEN_BACKLOG默认到511, 却由于我们参数未曾优化会限制到128,只有128个connections can be queued in kernel listen queue(by Igor Sysoev).
define NGX_LISTEN_BACKLOG 511/ls.backlog = NGX_LISTEN_BACKLOG;/ if (listen(s, ls.backlog) == -1) {
相信其他应用比如squid也会有类似问题,突发的大并发connect请求会由于内核listen队列的限制导致链接超时或者重传,从而影响用户体验
以下是实验测试情况,使用2台机器分别以1000个并发,benchmark方式,请求服务器 ( 相当于2000个并发请求同时请求服务器 )
情景1,默认配置, net.core.somaxconn=128,服务器为nginx
测试客户端A:

Transactions:                2072870 hits
 
Availability:                  99.99 %
 
Elapsed time:                 179.59 secs
 
Data transferred:            6096.59 MB
 
Response time:                  0.08 secs
 
Transaction rate:           11542.24 trans/sec
 
Throughput:                    33.95 MB/sec
 
Concurrency:                  927.34
 
Successful transactions:     2072871
 
Failed transactions:             300
 
Longest transaction:           45.30
 
Shortest transaction:           0.00

错误率大概是1.5%

测试客户端B:

Transactions:                1859454 hits
 
Availability:                  99.99 %
 
Elapsed time:                 179.11 secs
 
Data transferred:            5468.90 MB
 
Response time:                  0.09 secs
 
Transaction rate:           10381.63 trans/sec
 
Throughput:                    30.53 MB/sec
 
Concurrency:                  904.45
 
Successful transactions:     1859454
 
Failed transactions:             276
 
Longest transaction:           49.60
 
Shortest transaction:           0.00

错误率大概也是1.5%
错误提示大都为:
socket: connection timed out
warning: socket: -1803417280 select timed out: Connection timed out

情景2,调整配置, net.core.somaxconn=8192, nginx显式配置 listen 80 default backlog=8192;
测试客户端A:

** SIEGE 2.69
 
** Preparing 1000 concurrent users for battle.
 
The server is now under siege...
 
Lifting the server siege...      done.
 
Transactions:                1789818 hits
 
Availability:                 100.00 %
 
Elapsed time:                 180.00 secs
 
Data transferred:            5264.09 MB
 
Response time:                  0.10 secs
 
Transaction rate:            9943.43 trans/sec
 
Throughput:                    29.24 MB/sec
 
Concurrency:                  997.06
 
Successful transactions:     1789818
 
Failed transactions:               0
 
Longest transaction:            0.87
 
Shortest transaction:           0.00

错误率是0

测试客户端B:

** SIEGE 2.69
 
** Preparing 1000 concurrent users for battle.
 
The server is now under siege...
 
Lifting the server siege...      done.
 
Transactions:                1768585 hits
 
Availability:                 100.00 %
 
Elapsed time:                 179.31 secs
 
Data transferred:            5201.65 MB
 
Response time:                  0.10 secs
 
Transaction rate:            9863.28 trans/sec
 
Throughput:                    29.01 MB/sec
 
Concurrency:                  998.30
 
Successful transactions:     1768588
 
Failed transactions:               0
 
Longest transaction:            3.10
 
Shortest transaction:           0.03

错误率是0 nginx的配置中增加监听队列的数量,当然前提是不能超过net.core.somaxconn的值。如:

server {
        listen 80 backlog=8192;
        server_name http://www.linuxgit.org;
    }

还没有评论,快来抢沙发!

发表评论

  • 😉
  • 😐
  • 😡
  • 😈
  • 🙂
  • 😯
  • 🙁
  • 🙄
  • 😛
  • 😳
  • 😮
  • emoji-mrgree
  • 😆
  • 💡
  • 😀
  • 👿
  • 😥
  • 😎
  • ➡
  • 😕
  • ❓
  • ❗
  • 70 queries in 0.408 seconds