最近在使用packetbeat来捕获流量中的http包。packetbeat部署在物理机上,捕获的是镜像流量,启用af_packet模式,镜像流量中只有tcp流量,没有udp。
packetbeat版本为:6.2.4
操作系统为:ubuntu16.04
我的packetbeat配置如下:
#============================== Network device ================================
packetbeat.interfaces.device: eth0
packetbeat.interfaces.type: af_packet
packetbeat.interfaces.buffer_size_mb: 3072
#========================== Transaction protocols =============================
packetbeat.protocols:
- type: http
ports: [80, 7001, 8080, 8000, 8001, 8081, 5000, 5001, 8002, 11080, 82, 7002, 8088, 18880]
send_all_headers: true
include_body_for: ["text/xml", "text/html", "text/plain", "application/xml", "application/json", "multipart/form-data", "application/csv", "text/csv", "application/x-ndjson", "application/x-amf"]
split_cookie: true
#transaction_timeout: 20s
#================================ General =====================================
processors:
- drop_event:
when:
not:
or:
- regexp:
ip: "10.xxx.*.*"
- regexp:
ip: "10.xxx.*.*"
#------------------------------- Kafka output ----------------------------------
output.kafka:
enabled: true
hosts: ["10.xx.xx.xx:9092"]
topic: xxx
worker: 6#---------------------------------------------------------------------------------
发现当网络流量为400+m/s的时候,packetbeat有时会产生较严重的丢包,大约会丢10+%,有时甚至可以丢包80%。请问这是正常情况吗?如何可以调优呢,物理机资源足够。
注意:
1、通过排查packetbeat的日志,未发现其他问题
2、已经关闭了gro功能
packetbeat版本为:6.2.4
操作系统为:ubuntu16.04
我的packetbeat配置如下:
#============================== Network device ================================
packetbeat.interfaces.device: eth0
packetbeat.interfaces.type: af_packet
packetbeat.interfaces.buffer_size_mb: 3072
#========================== Transaction protocols =============================
packetbeat.protocols:
- type: http
ports: [80, 7001, 8080, 8000, 8001, 8081, 5000, 5001, 8002, 11080, 82, 7002, 8088, 18880]
send_all_headers: true
include_body_for: ["text/xml", "text/html", "text/plain", "application/xml", "application/json", "multipart/form-data", "application/csv", "text/csv", "application/x-ndjson", "application/x-amf"]
split_cookie: true
#transaction_timeout: 20s
#================================ General =====================================
processors:
- drop_event:
when:
not:
or:
- regexp:
ip: "10.xxx.*.*"
- regexp:
ip: "10.xxx.*.*"
#------------------------------- Kafka output ----------------------------------
output.kafka:
enabled: true
hosts: ["10.xx.xx.xx:9092"]
topic: xxx
worker: 6#---------------------------------------------------------------------------------
发现当网络流量为400+m/s的时候,packetbeat有时会产生较严重的丢包,大约会丢10+%,有时甚至可以丢包80%。请问这是正常情况吗?如何可以调优呢,物理机资源足够。
注意:
1、通过排查packetbeat的日志,未发现其他问题
2、已经关闭了gro功能
1 个回复
rochy - rochy_he
赞同来自:
你可以对你 kafka 的 topic 进行优化配置,例如多设置几个分区等提高写入的并发。