我们是自定义的日志,日志内容大部分已经固定,但是留了一个叫做entry_detail的字段,可以再这个字段里面再加一个字典,供每个开发自行发挥。现在问题来了,因为entry_detail这个字段的不统一,导致es丢失数据,原因猜测是,entry_detail这个对象中会有名字相同,但是值不是相同类型的key:value,导致先入的"a":1在es中自动生成mapping后,"a":"c"这样和之前a数据类型不类似的就会被丢弃掉,产生丢数据问题。
例如:
{"entry_key":"ABC","entry_detail":{"a":1,"b":"2","c":"c"}}
{"entry_key":"AHJ","entry_detail":{"a":"c","h":"2","j":"c"}}
现在想问logstash是否可以进行字符串拼接,把日志转化成这样的格式在入到elastic中
{"entry_key":"ABC","entry_detail_ABC":{"a":1,"b":"2","c":"c"}}
{"entry_key":"AHJ","entry_detail_AHJ":{"a":"c","h":"2","j":"c"}}
谢谢大家。
例如:
{"entry_key":"ABC","entry_detail":{"a":1,"b":"2","c":"c"}}
{"entry_key":"AHJ","entry_detail":{"a":"c","h":"2","j":"c"}}
现在想问logstash是否可以进行字符串拼接,把日志转化成这样的格式在入到elastic中
{"entry_key":"ABC","entry_detail_ABC":{"a":1,"b":"2","c":"c"}}
{"entry_key":"AHJ","entry_detail_AHJ":{"a":"c","h":"2","j":"c"}}
谢谢大家。
0 个回复