用了Elasticsearch,一口气上5T

logstash使用geoip库报错

Logstash | 作者 sun_changlong | 发布于2018年07月31日 | 阅读数:5732

我先使用kibana 进行添加静态映射,然后使用logstash接收数据,发送给es,打算在kibana上进行地理位置的显示。但是在更改logstash配置以后,logstash启动失败,直接报错。我得配置如下:
 

静态映射:(只是挑选了其中一部分贴出来)
PUT my_index
{
"mappings": {
"my_type": {
"properties": {
"@timestamp" : {
"type" : "date"
},
"client_ip" : {
"type" : "ip"
},
"dip" : {
"type" : "ip"
},
"hostname" : {
"type" : "keyword"
},
"location_dict" : {
"type" : "geo_point"
},
"location_list" : {
"type" : "geo_point"
},
"location_str" : {
"type" : "geo_point"
}
}
}
}
}

 
 
logstash.conf:
input {
rabbitmq {
host => "192.168.33.29"
port => 5672
user => "guest"
password => "guest"
ssl => false
queue => 'center_probe_to_ztms'
exchange => "center_probe_to_ztms"
durable => true
heartbeat => 5
subscription_retry_interval_seconds => 5
type => "topav"
}

syslog {
host => "192.168.33.85"
port => 515
type => "syslog"
}
}

filter {

if[type] == "topav" {
geoip {
source => "[b]client_ip[/b]"
# fields => ["client_ip","sip"]
target => "geoip"
database => "/home/logstash/GeoLigCity/[b]GeoLite2-City.mmdb[/b]"
# database => "/home/logstash/GeoLigCity/GeoLiteCity.dat"
add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}" ]
}
}

mutate {
convert => [ "[geoip][coordinates]", "float" ]
}

if[type] == "topav" {
kv {
source => "message"
field_split => ","
value_split => ":"
}
}

date {
match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
}
}

output {

if[type] == "topav" {
elasticsearch {
hosts => ["http://192.168.33.85:9200"]
index => "my_index"
document_type => "my_type"
}
}
stdout { codec => rubydebug }
}

 
 
错误信息:



Error registering plugin {:pipeline_id=>"main", :plugin=>"#<LogStash::FilterDelegator:0x5e314eb5 @metric_events_out=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: out value:0, @metric_events_in=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: in value:0, @metric_events_time=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: duration_in_millis value:0, @id=\"77dd015941db6ff89f4b57e8a834892c57067ddaeef3ea035c4b32b2bacd33fb\", @klass=LogStash::Filters::GeoIP, @metric_events=#<LogStash::Instrument::NamespacedMetric:0x73a86d75 @metric=#<LogStash::Instrument::Metric:0x3128d50 @collector=#<LogStash::Instrument::Collector:0x62663e6c @agent=nil, @metric_store=#<LogStash::Instrument::MetricStore:0x36768073 @store=#<Concurrent::Map:0x00000000000fb0 entries=3 default_proc=nil>, @structured_lookup_mutex=#<Mutex:0x422c3c9f>, @fast_lookup=#<Concurrent::Map:0x00000000000fb4 entries=91 default_proc=nil>>>>, @namespace_name=[:stats, :pipelines, :main, :plugins, :filters, :\"77dd015941db6ff89f4b57e8a834892c57067ddaeef3ea035c4b32b2bacd33fb\", :events]>, @filter=<LogStash::Filters::GeoIP source=>\"client_ip\", target=>\"geoip\", database=>\"/home/logstash/GeoLigCity/GeoLite2-City.mmdb\", add_field=>{\"[geoip][coordinates]\"=>[\"%{[geoip][longitude]}\", \"%{[geoip][latitude]}\"]}, id=>\"77dd015941db6ff89f4b57e8a834892c57067ddaeef3ea035c4b32b2bacd33fb\", enable_metric=>true, periodic_flush=>false, default_database_type=>\"City\", cache_size=>1000, tag_on_failure=>[\"_geoip_lookup_failure\"]>>", :error=>"The database provided is invalid or corrupted.", :thread=>"#<Thread:0x2c749264 run>"}
 



 
有哪位大佬遇到过或者知道原因的么?
 
 
已邀请:

sun_changlong

赞同来自:

库解析出现问题,重新下载替换即可

zqc0512 - andy zhou

赞同来自:

database => "/home/logstash/GeoLigCity/[b]GeoLite2-City.mmdb[/b]"
加个[/b]是撒意思?
免费的不怎么准确。
 

要回复问题请先登录注册