使用 dmesg 来查看一些硬件或驱动程序的信息或问题。

es2.x版本升级到es6.1版本bulk插入速度变慢,从哪些方面定位?

Elasticsearch | 作者 greatnew | 发布于2018年03月19日 | 阅读数:5364

原配置
es2.x, 3台物理机集群,es默认配置,3个docker进行bulk写入,
es6.1,5台物理机集群,es默认配置,3个docker进行bulk写入。
原来写入速度在几百ms级别,现在写入速度在1-2秒左右。
已邀请:

greatnew

赞同来自: kennywu76 夏一跳

看到说6.0以后不支持一个索引下多个type了

greatnew

赞同来自:

[2018-03-19T11:38:36,022][DEBUG][o.e.a.b.TransportShardBulkAction] [20180319-09][4] failed to execute bulk item (index) BulkShardRequest [[20180319-09][4]] containing [52] requests
java.lang.IllegalArgumentException: Rejecting mapping update to [20180319-09] as the final mapping would have more than 1 type: [mall-trade-api, datacenter-spark-log]
at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:494) ~[elasticsearch-6.1.3.jar:6.1.3]
at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:350) ~[elasticsearch-6.1.3.jar:6.1.3]
at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:282) ~[elasticsearch-6.1.3.jar:6.1.3]
at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.applyRequest(MetaDataMappingService.java:311) ~[elasticsearch-6.1.3.jar:6.1.3]
at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.execute(MetaDataMappingService.java:230) ~[elasticsearch-6.1.3.jar:6.1.3]
at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:640) ~[elasticsearch-6.1.3.jar:6.1.3]
at org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:270) ~[elasticsearch-6.1.3.jar:6.1.3]
at org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:195) ~[elasticsearch-6.1.3.jar:6.1.3]
at org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:130) ~[elasticsearch-6.1.3.jar:6.1.3]
at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) ~[elasticsearch-6.1.3.jar:6.1.3]
at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) ~[elasticsearch-6.1.3.jar:6.1.3]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:568) ~[elasticsearch-6.1.3.jar:6.1.3]
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:247) ~[elasticsearch-6.1.3.jar:6.1.3]
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:210) ~[elasticsearch-6.1.3.jar:6.1.3]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_20]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_20]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_20]

看了日志发现报错了,先研究下这个错误

要回复问题请先登录注册