流程: filebeat读取java日志->es中配置了pipeline预处理->根据模板写入es
问题: filebeat做了multiline配置,可以多行合并,但是经过pipeline的grok预处理以后就只能读取第一行了.如果清空pipeline的processors配置则一切正常.
 
这是我filebeat的配置
这是我pipeline
 
这是我template
 
 
这是我的demo文件
 
 
 
 
 
 
																
																																																
															
														问题: filebeat做了multiline配置,可以多行合并,但是经过pipeline的grok预处理以后就只能读取第一行了.如果清空pipeline的processors配置则一切正常.
这是我filebeat的配置
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /Users/liule/Desktop/demo.log
  multiline:
    pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
    negate: true
    match: after
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: true
setup.ilm.enabled: false
output.elasticsearch:
  hosts: ["localhost:9200"]
  pipeline: "java_pipeline"
  index: "iotrta-%{+yyyy.MM.dd}"
setup.template.name: "log-java"
setup.template.pattern: "log-java-*"
queue.mem:
  events: 256
  flush.min_events: 128这是我pipeline
这是我template
这是我的demo文件
2019-10-17 11:29:45.011  INFO 12365 --- [main] c.d.AbnormalInterruptTaskFixRunner       : dddddddddddddd
2019-10-17 11:30:45.011  INFO 12365 --- [main] c.d.AbnormalInterruptTaskFixRunner       : test
2019-10-17 11:31:34.243 ERROR 12365 --- [XNIO-2 I/O-10] io.undertow.request.io                   : UT005090: Unexpected failure
java.dddddd.dddddddd: dddddddddddddd
	at aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
	at aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
	at aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
	at aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
	at aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
	at aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
	at aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
	at aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
	at aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
	at aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
2019-10-17 11:31:34.243 ERROR 12365 --- [XNIO-2 I/O-11] io.undertow.request.io                   : UT005090: Unexpected failure
java.lang.NoClassDefFoundError: Could not initialize class io.undertow.UndertowMessages
	at io.undertow.server.DefaultByteBufferPool$DefaultPooledBuffer.getBuffer(DefaultByteBufferPool.java:260) ~[undertow-core-1.4.25.Final.jar!/:1.4.25.Final]
	at io.undertow.protocols.ssl.SslConduit.doUnwrap(SslConduit.java:833) ~[undertow-core-1.4.25.Final.jar!/:1.4.25.Final]
	at io.undertow.protocols.ssl.SslConduit.doHandshake(SslConduit.java:648) ~[undertow-core-1.4.25.Final.jar!/:1.4.25.Final]
	at io.undertow.protocols.ssl.SslConduit.access$900(SslConduit.java:63) ~[undertow-core-1.4.25.Final.jar!/:1.4.25.Final]
	at io.undertow.protocols.ssl.SslConduit$SslReadReadyHandler.readReady(SslConduit.java:1127) ~[undertow-core-1.4.25.Final.jar!/:1.4.25.Final]
	at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:88) [xnio-nio-3.3.8.Final.jar!/:3.3.8.Final]
	at org.xnio.nio.WorkerThread.run(WorkerThread.java:561) [xnio-nio-3.3.8.Final.jar!/:3.3.8.Final]


 
	
1 个回复
CononYc
赞同来自:
参阅elasticsearch 官方文档 Ingest node ->grok-processor
表达式参阅:github.com/kkos/oniguruma/blob/master/doc/RE
匹配多行文本的表达式为(.m:.*)
在pipeline下配置
使用