日志文件还在写,为何filebeat 读取不到日志?
匿名 | 发布于2019年10月18日 | 阅读数:5908
日志:
2019-09-27T20:32:19.961+0800 INFO input/input.go:114 Starting input of type: log; ID: 17475743357618369855
2019-09-27T20:32:19.961+0800 INFO crawler/crawler.go:106 Loading and starting Inputs completed. Enabled inputs: 1
2019-09-27T20:32:19.962+0800 INFO cfgfile/reload.go:141 Config reloader started
2019-09-27T20:32:19.962+0800 INFO cfgfile/reload.go:196 Loading of config files completed.
2019-09-27T20:32:19.965+0800 INFO log/harvester.go:251 Harvester started for file: /data/logs/sss-service/sss-service.log
2019-09-27T20:32:19.967+0800 INFO log/harvester.go:251 Harvester started for file: /data/logs/sss-service/sss-service.log.1
2019-09-27T20:32:19.969+0800 INFO log/harvester.go:251 Harvester started for file: /data/logs/sss-service/sss-service.log.2
2019-09-27T20:32:19.970+0800 INFO log/harvester.go:251 Harvester started for file: /data/logs/sss-service/sss-service.log.7
2019-09-27T20:32:19.970+0800 INFO log/harvester.go:251 Harvester started for file: /data/logs/sss-service/sss-service.log.3
2019-09-27T20:32:19.970+0800 INFO log/harvester.go:251 Harvester started for file: /data/logs/sss-service/sss-service.log.4
2019-09-27T20:32:19.970+0800 INFO log/harvester.go:251 Harvester started for file: /data/logs/sss-service/sss-service.log.5
2019-09-27T20:32:19.970+0800 INFO log/harvester.go:251 Harvester started for file: /data/logs/sss-service/sss-service.log.6
2019-09-27T20:32:19.990+0800 INFO pipeline/output.go:95 Connecting to backoff(async(tcp://10.10.130.151:4666))
2019-09-27T20:32:19.990+0800 INFO pipeline/output.go:105 Connection to backoff(async(tcp://10.10.130.151:4666)) established
2019-09-27T20:32:29.955+0800 INFO pipeline/output.go:95 Connecting to backoff(failover(publish(elasticsearch(http://10.10.130.149:9910)),publish(elasticsearch(http://10.10.130.149:9911))))
2019-09-27T20:32:29.956+0800 INFO pipeline/output.go:105 Connection to backoff(failover(publish(elasticsearch(http://10.10.130.149:9910)),publish(elasticsearch(http://10.10.130.149:9911)))) established
2019-09-27T20:32:49.954+0800 INFO [monitoring] log/log.go:141 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":30,"time":{"ms":34}},"total":{"ticks":160,"time":{"ms":171},"value":160},"user":{"ticks":130,"time" :{"ms":137}}},"info":{"ephemeral_id":"42c4ceec-33e6-4b34-a0ea-cb318bcffe13","uptime":{"ms":30012}},"memstats":{"gc_next":17368016,"memory_alloc":13482480,"memory_total":25397560,"rss":33992704}},"filebeat":{"events":{"active":4129,"added":4140,"done":11},"harvester":{"ope n_files":8,"running":8,"st arted":8}},"libbeat":{"config":{"module":{"running":0},"reloads":1},"output":{"events":{"active":4096,"batches":2,"total":4096},"read":{"bytes":30},"type":"logstash","write":{"bytes":228980}},"pipeline":{"clients":1,"events":{"active":4117,"filte red":16,"published":4116,"retry":2048,"total":4133}}},"registrar":{"states":{"current":8,"update":11},"writes":{"success":11,"total":11}},"system":{"cpu":{"cores":2},"load":{"1":0,"15":0.31,"5":0,"norm":{"1":0,"15":0.155,"5":0}}}}}}
2019-09-27T20:33:19.954+0800 INFO [monitoring] log/log.go:141 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":30,"time":{"ms":5}},"total":{"ticks":170,"time":{"ms":16},"value":170},"user":{"ticks":140,"time":{ "ms":11}}},"info":{"ephemeral_id":"42c4ceec-33e6-4b34-a0ea-cb318bcffe13","uptime":{"ms":60012}},"memstats":{"gc_next":12191264,"memory_alloc":6643040,"memory_total":26802576,"rss":819200}},"filebeat":{"harvester":{"open_files":8,"running":8}},"libbeat":{"config":{"module" :{"running":0}},"output":{"read":{"bytes":36}},"pipeline":{"clients":1,"events":{"active":4117}}},"registrar":{"states":{"current":8}},"system":{"load":{"1":0,"15":0.29,"5":0,"norm":{"1":0,"15":0.145,"5":0}}}}}}
2019-09-27T20:33:49.954+0800 INFO [monitoring] log/log.go:141 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":40,"time":{"ms":7}},"total":{"ticks":190,"time":{"ms":11},"value":190},"user":{"ticks":150,"time":{ "ms":4}}},"info":{"ephemeral_id":"42c4ceec-33e6-4b34-a0ea-cb318bcffe13","uptime":{"ms":90013}},"memstats":{"gc_next":12191264,"memory_alloc":8285744,"memory_total":28445280}},"filebeat":{"harvester":{"open_files":8,"running":8}},"libbeat":{"config":{"module":{"running":0} },"output":{"read":{"bytes":36}},"pipeline":{"clients":1,"events":{"active":4117}}},"registrar":{"states":{"current":8}},"system":{"load":{"1":0,"15":0.28,"5":0,"norm":{"1":0,"15":0.14,"5":0}}}}}}
2019-09-27T20:34:19.954+0800 INFO [monitoring] log/log.go:141 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":50,"time":{"ms":6}},"total":{"ticks":200,"time":{"ms":10},"value":200},"user":{"ticks":150,"time":{ "ms":4}}},"info":{"ephemeral_id":"42c4ceec-33e6-4b34-a0ea-cb318bcffe13","uptime":{"ms":120012}},"memstats":{"gc_next":12191264,"memory_alloc":9571928,"memory_total":29731464}},"filebeat":{"harvester":{"open_files":8,"running":8}},"libbeat":{"config":{"module":{"running":0 }},"output":{"read":{"bytes":42}},"pipeline":{"clients":1,"events":{"active":4117}}},"registrar":{"states":{"current":8}},"system":{"load":{"1":0,"15":0.27,"5":0,
2019-09-27T20:32:19.961+0800 INFO input/input.go:114 Starting input of type: log; ID: 17475743357618369855
2019-09-27T20:32:19.961+0800 INFO crawler/crawler.go:106 Loading and starting Inputs completed. Enabled inputs: 1
2019-09-27T20:32:19.962+0800 INFO cfgfile/reload.go:141 Config reloader started
2019-09-27T20:32:19.962+0800 INFO cfgfile/reload.go:196 Loading of config files completed.
2019-09-27T20:32:19.965+0800 INFO log/harvester.go:251 Harvester started for file: /data/logs/sss-service/sss-service.log
2019-09-27T20:32:19.967+0800 INFO log/harvester.go:251 Harvester started for file: /data/logs/sss-service/sss-service.log.1
2019-09-27T20:32:19.969+0800 INFO log/harvester.go:251 Harvester started for file: /data/logs/sss-service/sss-service.log.2
2019-09-27T20:32:19.970+0800 INFO log/harvester.go:251 Harvester started for file: /data/logs/sss-service/sss-service.log.7
2019-09-27T20:32:19.970+0800 INFO log/harvester.go:251 Harvester started for file: /data/logs/sss-service/sss-service.log.3
2019-09-27T20:32:19.970+0800 INFO log/harvester.go:251 Harvester started for file: /data/logs/sss-service/sss-service.log.4
2019-09-27T20:32:19.970+0800 INFO log/harvester.go:251 Harvester started for file: /data/logs/sss-service/sss-service.log.5
2019-09-27T20:32:19.970+0800 INFO log/harvester.go:251 Harvester started for file: /data/logs/sss-service/sss-service.log.6
2019-09-27T20:32:19.990+0800 INFO pipeline/output.go:95 Connecting to backoff(async(tcp://10.10.130.151:4666))
2019-09-27T20:32:19.990+0800 INFO pipeline/output.go:105 Connection to backoff(async(tcp://10.10.130.151:4666)) established
2019-09-27T20:32:29.955+0800 INFO pipeline/output.go:95 Connecting to backoff(failover(publish(elasticsearch(http://10.10.130.149:9910)),publish(elasticsearch(http://10.10.130.149:9911))))
2019-09-27T20:32:29.956+0800 INFO pipeline/output.go:105 Connection to backoff(failover(publish(elasticsearch(http://10.10.130.149:9910)),publish(elasticsearch(http://10.10.130.149:9911)))) established
2019-09-27T20:32:49.954+0800 INFO [monitoring] log/log.go:141 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":30,"time":{"ms":34}},"total":{"ticks":160,"time":{"ms":171},"value":160},"user":{"ticks":130,"time" :{"ms":137}}},"info":{"ephemeral_id":"42c4ceec-33e6-4b34-a0ea-cb318bcffe13","uptime":{"ms":30012}},"memstats":{"gc_next":17368016,"memory_alloc":13482480,"memory_total":25397560,"rss":33992704}},"filebeat":{"events":{"active":4129,"added":4140,"done":11},"harvester":{"ope n_files":8,"running":8,"st arted":8}},"libbeat":{"config":{"module":{"running":0},"reloads":1},"output":{"events":{"active":4096,"batches":2,"total":4096},"read":{"bytes":30},"type":"logstash","write":{"bytes":228980}},"pipeline":{"clients":1,"events":{"active":4117,"filte red":16,"published":4116,"retry":2048,"total":4133}}},"registrar":{"states":{"current":8,"update":11},"writes":{"success":11,"total":11}},"system":{"cpu":{"cores":2},"load":{"1":0,"15":0.31,"5":0,"norm":{"1":0,"15":0.155,"5":0}}}}}}
2019-09-27T20:33:19.954+0800 INFO [monitoring] log/log.go:141 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":30,"time":{"ms":5}},"total":{"ticks":170,"time":{"ms":16},"value":170},"user":{"ticks":140,"time":{ "ms":11}}},"info":{"ephemeral_id":"42c4ceec-33e6-4b34-a0ea-cb318bcffe13","uptime":{"ms":60012}},"memstats":{"gc_next":12191264,"memory_alloc":6643040,"memory_total":26802576,"rss":819200}},"filebeat":{"harvester":{"open_files":8,"running":8}},"libbeat":{"config":{"module" :{"running":0}},"output":{"read":{"bytes":36}},"pipeline":{"clients":1,"events":{"active":4117}}},"registrar":{"states":{"current":8}},"system":{"load":{"1":0,"15":0.29,"5":0,"norm":{"1":0,"15":0.145,"5":0}}}}}}
2019-09-27T20:33:49.954+0800 INFO [monitoring] log/log.go:141 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":40,"time":{"ms":7}},"total":{"ticks":190,"time":{"ms":11},"value":190},"user":{"ticks":150,"time":{ "ms":4}}},"info":{"ephemeral_id":"42c4ceec-33e6-4b34-a0ea-cb318bcffe13","uptime":{"ms":90013}},"memstats":{"gc_next":12191264,"memory_alloc":8285744,"memory_total":28445280}},"filebeat":{"harvester":{"open_files":8,"running":8}},"libbeat":{"config":{"module":{"running":0} },"output":{"read":{"bytes":36}},"pipeline":{"clients":1,"events":{"active":4117}}},"registrar":{"states":{"current":8}},"system":{"load":{"1":0,"15":0.28,"5":0,"norm":{"1":0,"15":0.14,"5":0}}}}}}
2019-09-27T20:34:19.954+0800 INFO [monitoring] log/log.go:141 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":50,"time":{"ms":6}},"total":{"ticks":200,"time":{"ms":10},"value":200},"user":{"ticks":150,"time":{ "ms":4}}},"info":{"ephemeral_id":"42c4ceec-33e6-4b34-a0ea-cb318bcffe13","uptime":{"ms":120012}},"memstats":{"gc_next":12191264,"memory_alloc":9571928,"memory_total":29731464}},"filebeat":{"harvester":{"open_files":8,"running":8}},"libbeat":{"config":{"module":{"running":0 }},"output":{"read":{"bytes":42}},"pipeline":{"clients":1,"events":{"active":4117}}},"registrar":{"states":{"current":8}},"system":{"load":{"1":0,"15":0.27,"5":0,
1 个回复
ziyou - 一个学习ELK的Java程序员
赞同来自:
这个是filebeat的工作机制,你可以看看是不是你filebeat的注册文件除了问题,看看是不是你的注册文件导致的。