logstash5.6.8 kafka 多个topic写入ES(topic写入不同index)
Logstash | 作者 zhjike0801 | 发布于2020年01月03日 | 阅读数:3535
logstash配置
input {
kafka {
bootstrap_servers => ["localhost:9092"]
group_id => "test"
auto_offset_reset => "latest"
consumer_threads => 5
decorate_events => true
topics_pattern => ".*"
codec => "json"
}
}
output {
elasticsearch {
hosts => ["192.168.101.82:9200","192.168.101.83:9200"]
index => "%{[@metadata][kafka][topic]}-%{+YYYY.MM.dd}"
}
}
但是最后在es中的 index 是 %{[@metadata][kafka][topic]}-2020.01.03
metadata解析失败, decorate_events => true在input已经加上
input {
kafka {
bootstrap_servers => ["localhost:9092"]
group_id => "test"
auto_offset_reset => "latest"
consumer_threads => 5
decorate_events => true
topics_pattern => ".*"
codec => "json"
}
}
output {
elasticsearch {
hosts => ["192.168.101.82:9200","192.168.101.83:9200"]
index => "%{[@metadata][kafka][topic]}-%{+YYYY.MM.dd}"
}
}
但是最后在es中的 index 是 %{[@metadata][kafka][topic]}-2020.01.03
metadata解析失败, decorate_events => true在input已经加上
1 个回复
shitangjiejie
赞同来自: