初次使用logstash,输入日志到logstash,我在grok debbuger中调试是通过的(包括自定义patterns),我的目的是想分割这种日志,让日志中每个字段一起存入,不知道为什么只要我在output加入要分割后存入的字段老是报错 (报错日志在最下面),我试了一下output: rspcode => "%{rspcode}" ,是不是我我的写法有问题?
日志原始数据:
INFO[12-06 14:53:18,995] -> TransactionInvokerpay377000001167636|TransactionInvokerpay377000001167636|p.rdosvr|TransactionInvoker|150|SCM00000||
patterns_dir中的文件内容:
WORDPOINT [a-zA-Z0-9._-]+
WORDRSPMSG .*
logstash 6.5.2的logstash.conf配置:
input {
beats {
port => 5044
}
}
filter {
if [tags] == "txn" {
grok {
patterns_dir => ["/home/rmqadm/elastic/logstash-6.4.2/config/patterns"]
match=>{"message"=>"%{LOGLEVEL:level}\[%{DATA:time}\] \-> %{WORD:logid}\|%{WORD:logid_child}\|%{WORDPOINT:serviceid}\|%{WORD:actionid}\|%{NUMBER:ms}\|%{WORD:rspcode}\|%{WORDRSPMSG:rspmsg}\|"}
}
}
}
output {
elasticsearch {
hosts => ["http://192.168.1.232:9200"]
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
rspcode => "%{rspcode}"
}
}
logstash中的报错日志:
[2018-12-11T10:24:57,507][ERROR][logstash.outputs.elasticsearch] Unknown setting 'rspcode' for elasticsearch
[2018-12-11T10:24:57,529][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Something is wrong with your configuration.", :backtrace=>["/home/rmqadm/elastic/logstash-6.4.2/logstash-core/lib/logstash/config/mixin.rb:86:in `config_init'", "/home/rmqadm/elastic/logstash-6.4.2/logstash-core/lib/logstash/outputs/base.rb:60:in `initialize'", "org/logstash/config/ir/compiler/OutputStrategyExt.java:224:in `initialize'", "org/logstash/config/ir/compiler/OutputDelegatorExt.java:48:in `initialize'", "org/logstash/config/ir/compiler/OutputDelegatorExt.java:30:in `initialize'", "org/logstash/plugins/PluginFactoryExt.java:217:in `plugin'", "org/logstash/plugins/PluginFactoryExt.java:166:in `plugin'", "/home/rmqadm/elastic/logstash-6.4.2/logstash-core/lib/logstash/pipeline.rb:71:in `plugin'", "(eval):35:in `<eval>'", "org/jruby/RubyKernel.java:994:in `eval'", "/home/rmqadm/elastic/logstash-6.4.2/logstash-core/lib/logstash/pipeline.rb:49:in `initialize'", "/home/rmqadm/elastic/logstash-6.4.2/logstash-core/lib/logstash/pipeline.rb:90:in `initialize'", "/home/rmqadm/elastic/logstash-6.4.2/logstash-core/lib/logstash/pipeline_action/create.rb:38:in `execute'", "/home/rmqadm/elastic/logstash-6.4.2/logstash-core/lib/logstash/agent.rb:309:in `block in converge_state'"]}
日志原始数据:
INFO[12-06 14:53:18,995] -> TransactionInvokerpay377000001167636|TransactionInvokerpay377000001167636|p.rdosvr|TransactionInvoker|150|SCM00000||
patterns_dir中的文件内容:
WORDPOINT [a-zA-Z0-9._-]+
WORDRSPMSG .*
logstash 6.5.2的logstash.conf配置:
input {
beats {
port => 5044
}
}
filter {
if [tags] == "txn" {
grok {
patterns_dir => ["/home/rmqadm/elastic/logstash-6.4.2/config/patterns"]
match=>{"message"=>"%{LOGLEVEL:level}\[%{DATA:time}\] \-> %{WORD:logid}\|%{WORD:logid_child}\|%{WORDPOINT:serviceid}\|%{WORD:actionid}\|%{NUMBER:ms}\|%{WORD:rspcode}\|%{WORDRSPMSG:rspmsg}\|"}
}
}
}
output {
elasticsearch {
hosts => ["http://192.168.1.232:9200"]
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
rspcode => "%{rspcode}"
}
}
logstash中的报错日志:
[2018-12-11T10:24:57,507][ERROR][logstash.outputs.elasticsearch] Unknown setting 'rspcode' for elasticsearch
[2018-12-11T10:24:57,529][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Something is wrong with your configuration.", :backtrace=>["/home/rmqadm/elastic/logstash-6.4.2/logstash-core/lib/logstash/config/mixin.rb:86:in `config_init'", "/home/rmqadm/elastic/logstash-6.4.2/logstash-core/lib/logstash/outputs/base.rb:60:in `initialize'", "org/logstash/config/ir/compiler/OutputStrategyExt.java:224:in `initialize'", "org/logstash/config/ir/compiler/OutputDelegatorExt.java:48:in `initialize'", "org/logstash/config/ir/compiler/OutputDelegatorExt.java:30:in `initialize'", "org/logstash/plugins/PluginFactoryExt.java:217:in `plugin'", "org/logstash/plugins/PluginFactoryExt.java:166:in `plugin'", "/home/rmqadm/elastic/logstash-6.4.2/logstash-core/lib/logstash/pipeline.rb:71:in `plugin'", "(eval):35:in `<eval>'", "org/jruby/RubyKernel.java:994:in `eval'", "/home/rmqadm/elastic/logstash-6.4.2/logstash-core/lib/logstash/pipeline.rb:49:in `initialize'", "/home/rmqadm/elastic/logstash-6.4.2/logstash-core/lib/logstash/pipeline.rb:90:in `initialize'", "/home/rmqadm/elastic/logstash-6.4.2/logstash-core/lib/logstash/pipeline_action/create.rb:38:in `execute'", "/home/rmqadm/elastic/logstash-6.4.2/logstash-core/lib/logstash/agent.rb:309:in `block in converge_state'"]}
1 个回复
rochy - rochy_he
赞同来自:
可添加到 filter 中,使用 addFields 即可
具体参考:https://www.elastic.co/guide/e ... field