刚接触logstash,想做一个demo,结果没有数据,甚至改变文件时,控制台都没有输出变化
用的7.12.0版本的
1、这个是我的conf文件, test.conf
input {
file {
path => "D:\logstash\test\demo.log"
codec => "json"
}
}
output {
elasticsearch {
ecs_compatibility => disabled
doc_as_upsert => true
action => "update"
# ES的IP地址及端口
hosts => ["....."]
# 索引名称 可自定义
index => "demo"
# 对应id
document_id => "%{id}"
}
}
2、这是json文件,demo.log
{"id":"1","count":"2"}
3、logstash-plain.log日志,其它日志没有内容
[2021-08-11T18:59:00,368][INFO ][logstash.runner ] Log4j configuration path used is: D:\logstash\logstash-7.12.0\config\log4j2.properties
[2021-08-11T18:59:00,379][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.12.0", "jruby.version"=>"jruby 9.2.13.0 (2.5.7) 2020-08-03 9a89c94bcc Java HotSpot(TM) 64-Bit Server VM 25.201-b09 on 1.8.0_201-b09 +indy +jit [mswin32-x86_64]"}
[2021-08-11T18:59:00,474][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2021-08-11T18:59:01,389][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2021-08-11T18:59:02,554][INFO ][org.reflections.Reflections] Reflections took 35 ms to scan 1 urls, producing 23 keys and 47 values
[2021-08-11T18:59:03,493][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:removed=>, :added=>[......]}}
[2021-08-11T18:59:03,714][WARN ][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"......"}
[2021-08-11T18:59:03,791][INFO ][logstash.outputs.elasticsearch][main] ES Output version determined {:es_version=>7}
[2021-08-11T18:59:03,794][WARN ][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7}
[2021-08-11T18:59:03,886][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//......"]}
[2021-08-11T18:59:03,902][WARN ][logstash.javapipeline ][main] 'pipeline.ordered' is enabled and is likely less efficient, consider disabling if preserving event order is not necessary
[2021-08-11T18:59:03,978][INFO ][logstash.outputs.elasticsearch][main] Using a default mapping template {:es_version=>7, :ecs_compatibility=>:disabled}
[2021-08-11T18:59:03,989][INFO ][logstash.javapipeline ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>125, "pipeline.sources"=>["D:/logstash/logstash-7.12.0/plugins/test.conf"], :thread=>"#<Thread:0x6d5cae26 run>"}
[2021-08-11T18:59:04,075][INFO ][logstash.outputs.elasticsearch][main] Attempting to install template {:manage_template=>{"index_patterns"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s", "number_of_shards"=>1}, "mappings"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}
[2021-08-11T18:59:04,608][INFO ][logstash.javapipeline ][main] Pipeline Java execution initialization time {"seconds"=>0.62}
[2021-08-11T18:59:04,983][INFO ][logstash.inputs.file ][main] No sincedb_path set, generating one based on the "path" setting {:sincedb_path=>"D:/logstash/logstash-7.12.0/data/plugins/inputs/file/.sincedb_90fca5704cee10e880a16fd13edc3a88", :path=>["D:\\logstash\\test\\demo.log"]}
[2021-08-11T18:59:05,006][INFO ][logstash.javapipeline ][main] Pipeline started {"pipeline.id"=>"main"}
[2021-08-11T18:59:05,076][INFO ][filewatch.observingtail ][main][2faf73846c0d4b5b75aaf8bf82521563e9772e09c90b1453b053aa5f4f1e9a7c] START, creating Discoverer, Watch with file and sincedb collections
[2021-08-11T18:59:05,076][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>}
用的7.12.0版本的
1、这个是我的conf文件, test.conf
input {
file {
path => "D:\logstash\test\demo.log"
codec => "json"
}
}
output {
elasticsearch {
ecs_compatibility => disabled
doc_as_upsert => true
action => "update"
# ES的IP地址及端口
hosts => ["....."]
# 索引名称 可自定义
index => "demo"
# 对应id
document_id => "%{id}"
}
}
2、这是json文件,demo.log
{"id":"1","count":"2"}
3、logstash-plain.log日志,其它日志没有内容
[2021-08-11T18:59:00,368][INFO ][logstash.runner ] Log4j configuration path used is: D:\logstash\logstash-7.12.0\config\log4j2.properties
[2021-08-11T18:59:00,379][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.12.0", "jruby.version"=>"jruby 9.2.13.0 (2.5.7) 2020-08-03 9a89c94bcc Java HotSpot(TM) 64-Bit Server VM 25.201-b09 on 1.8.0_201-b09 +indy +jit [mswin32-x86_64]"}
[2021-08-11T18:59:00,474][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2021-08-11T18:59:01,389][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2021-08-11T18:59:02,554][INFO ][org.reflections.Reflections] Reflections took 35 ms to scan 1 urls, producing 23 keys and 47 values
[2021-08-11T18:59:03,493][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:removed=>, :added=>[......]}}
[2021-08-11T18:59:03,714][WARN ][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"......"}
[2021-08-11T18:59:03,791][INFO ][logstash.outputs.elasticsearch][main] ES Output version determined {:es_version=>7}
[2021-08-11T18:59:03,794][WARN ][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7}
[2021-08-11T18:59:03,886][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//......"]}
[2021-08-11T18:59:03,902][WARN ][logstash.javapipeline ][main] 'pipeline.ordered' is enabled and is likely less efficient, consider disabling if preserving event order is not necessary
[2021-08-11T18:59:03,978][INFO ][logstash.outputs.elasticsearch][main] Using a default mapping template {:es_version=>7, :ecs_compatibility=>:disabled}
[2021-08-11T18:59:03,989][INFO ][logstash.javapipeline ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>125, "pipeline.sources"=>["D:/logstash/logstash-7.12.0/plugins/test.conf"], :thread=>"#<Thread:0x6d5cae26 run>"}
[2021-08-11T18:59:04,075][INFO ][logstash.outputs.elasticsearch][main] Attempting to install template {:manage_template=>{"index_patterns"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s", "number_of_shards"=>1}, "mappings"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}
[2021-08-11T18:59:04,608][INFO ][logstash.javapipeline ][main] Pipeline Java execution initialization time {"seconds"=>0.62}
[2021-08-11T18:59:04,983][INFO ][logstash.inputs.file ][main] No sincedb_path set, generating one based on the "path" setting {:sincedb_path=>"D:/logstash/logstash-7.12.0/data/plugins/inputs/file/.sincedb_90fca5704cee10e880a16fd13edc3a88", :path=>["D:\\logstash\\test\\demo.log"]}
[2021-08-11T18:59:05,006][INFO ][logstash.javapipeline ][main] Pipeline started {"pipeline.id"=>"main"}
[2021-08-11T18:59:05,076][INFO ][filewatch.observingtail ][main][2faf73846c0d4b5b75aaf8bf82521563e9772e09c90b1453b053aa5f4f1e9a7c] START, creating Discoverer, Watch with file and sincedb collections
[2021-08-11T18:59:05,076][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>}
2 个回复
tongchuan1992 - 学无止境、学以致用
赞同来自:
zhuyangping - 人不能一直幼稚下去
赞同来自: