filebeat pipline处理timestamp 报错,求助
匿名 | 发布于2019年11月30日 | 阅读数:3140
不想使用logstash 来解析字段所以想直接使用filebeat
es 里面的时间想使用原始日志的时间,现在使用pipline 报错,:
019-11-30T13:36:04.545+0800 DEBUG [elasticsearch] elasticsearch/client.go:337 PublishEvents: 1 events have been published to elasticsearch in 1.332163ms.
2019-11-30T13:36:04.545+0800 DEBUG [elasticsearch] elasticsearch/client.go:544 Bulk item insert failed (i=0, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: unable to parse date [2019-11-25 10:02:27:901]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: unable to parse date [2019-11-25 10:02:27:901]","caused_by":{"type":"illegal_argument_exception","reason":"unable to parse date [2019-11-25 10:02:27:901]","caused_by":{"type":"illegal_argument_exception","reason":"Invalid format: \"2019-11-25 10:02:27:901\" is malformed at \" 10:02:27:901\""}}},"header":{"processor_type":"date"}}
2019-11-30T13:36:04.889+0800 ERROR pipeline/output.go:121 Failed to publish events: temporary bulk send failure
pipline
PUT /_ingest/pipeline/test-pipeline/
{
"description" : "test-pipeline",
"processors" : [
{
"grok" :{
"field" : "message",
"patterns" : ["%{TIMESTAMP_ISO8601:time}"]
},
"remove": {
"field": "@timestamp"
}
},
{
"date" : {
"field" : "time",
"target_field" : "@timestamp",
"formats" : ["ISO8601"],
"timezone" : "UTC"
}
}
]
}
es 里面的时间想使用原始日志的时间,现在使用pipline 报错,:
019-11-30T13:36:04.545+0800 DEBUG [elasticsearch] elasticsearch/client.go:337 PublishEvents: 1 events have been published to elasticsearch in 1.332163ms.
2019-11-30T13:36:04.545+0800 DEBUG [elasticsearch] elasticsearch/client.go:544 Bulk item insert failed (i=0, status=500): {"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: unable to parse date [2019-11-25 10:02:27:901]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: unable to parse date [2019-11-25 10:02:27:901]","caused_by":{"type":"illegal_argument_exception","reason":"unable to parse date [2019-11-25 10:02:27:901]","caused_by":{"type":"illegal_argument_exception","reason":"Invalid format: \"2019-11-25 10:02:27:901\" is malformed at \" 10:02:27:901\""}}},"header":{"processor_type":"date"}}
2019-11-30T13:36:04.889+0800 ERROR pipeline/output.go:121 Failed to publish events: temporary bulk send failure
pipline
PUT /_ingest/pipeline/test-pipeline/
{
"description" : "test-pipeline",
"processors" : [
{
"grok" :{
"field" : "message",
"patterns" : ["%{TIMESTAMP_ISO8601:time}"]
},
"remove": {
"field": "@timestamp"
}
},
{
"date" : {
"field" : "time",
"target_field" : "@timestamp",
"formats" : ["ISO8601"],
"timezone" : "UTC"
}
}
]
}
0 个回复