异常1:
2016-04-19 10:14:47,544 INFO org.elasticsearch.client.transport(398): [Shatter] failed to get node info for [FAILED toString()], disconnecting...
java.lang.OutOfMemoryError: GC overhead limit exceeded
at org.elasticsearch.client.transport.TransportClientNodesService$SimpleNodeSampler.doSample(TransportClientNodesService.java:391)
at org.elasticsearch.client.transport.TransportClientNodesService$NodeSampler.sample(TransportClientNodesService.java:315)
at org.elasticsearch.client.transport.TransportClientNodesService$ScheduledNodeSampler.run(TransportClientNodesService.java:348)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
警告2: 看这个警告应该是快要发生异常了
[2016-04-19 14:42:31,622][WARN ][monitor.jvm ] [node9-ssd] [gc][young][444][30] duration [2.4s], collections [1]/[3s], total [2.4s]/[3.3s], memory [648.1mb]->[558.6mb]/[11.8gb], all_pools {[young] [270.2mb]->[2mb]/[865.3mb]}{[survivor] [108.1mb]->[108.1mb]/[108.1mb]}{[old] [269.7mb]->[448.4mb]/[10.9gb]}
[2016-04-19 14:42:33,170][INFO ][monitor.jvm ] [node9-ssd] [gc][young][445][31] duration [935ms], collections [1]/[1.5s], total [935ms]/[4.3s], memory [558.6mb]->[689.5mb]/[11.8gb], all_pools {[young] [2mb]->[2mb]/[865.3mb]}{[survivor] [108.1mb]->[108.1mb]/[108.1mb]}{[old] [448.4mb]->[579.3mb]/[10.9gb]}
[2016-04-19 14:42:35,320][WARN ][monitor.jvm ] [node9-ssd] [gc][young][446][32] duration [1.4s], collections [1]/[2.1s], total [1.4s]/[5.7s], memory [689.5mb]->[891.1mb]/[11.8gb], all_pools {[young] [2mb]->[3mb]/[865.3mb]}{[survivor] [108.1mb]->[108.1mb]/[108.1mb]}{[old] [579.3mb]->[780mb]/[10.9gb]}
[2016-04-19 14:42:39,429][WARN ][monitor.jvm ] [node9-ssd] [gc][young][449][33] duration [1.4s], collections [1]/[2.1s], total [1.4s]/[7.2s], memory [1.4gb]->[1gb]/[11.8gb], all_pools {[young] [584.4mb]->[4mb]/[865.3mb]}{[survivor] [108.1mb]->[108.1mb]/[108.1mb]}{[old] [780mb]->[995mb]/[10.9gb]}
[2016-04-19 14:42:45,733][WARN ][monitor.jvm ] [node9-ssd] [gc][young][455][34] duration [1s], collections [1]/[1.3s], total [1s]/[8.2s], memory [1.8gb]->[1.2gb]/[11.8gb], all_pools {[young] [786.3mb]->[7mb]/[865.3mb]}{[survivor] [108.1mb]->[108.1mb]/[108.1mb]}{[old] [995mb]->[1gb]/[10.9gb]}
异常3:
org.elasticsearch.transport.NodeDisconnectedException: [][inet[/ip:9300]][cluster:monitor/nodes/info] disconnected
2016-04-19 10:28:41,269 ERROR com.zkdj.restserver.dao.IndexDao(121): booleanQuery: None of the configured nodes were available: [[node6-ssd][AdKDN-lCQEGtZZrU7q_Y3Q][es6][inet[/ip:9300]], [node7-ssd][khpIoponRtWw0pfbOjBmpQ][es7][inet[/10.45.20.202:9300]]]
异常4:
2016-04-19 10:06:36,283 INFO org.elasticsearch.client.transport(398): [Shatter] failed to get node info for [#transport#-2][es3][inet[/ip:9300]], disconnecting...
org.elasticsearch.transport.ReceiveTimeoutTransportException: [][inet[/ip:9300]][cluster:monitor/nodes/info] request_id [25562] timed out after [6129ms]
at org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:529)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-04-19 10:14:47,544 INFO org.elasticsearch.client.transport(398): [Shatter] failed to get node info for [FAILED toString()], disconnecting...
java.lang.OutOfMemoryError: GC overhead limit exceeded
at org.elasticsearch.client.transport.TransportClientNodesService$SimpleNodeSampler.doSample(TransportClientNodesService.java:391)
at org.elasticsearch.client.transport.TransportClientNodesService$NodeSampler.sample(TransportClientNodesService.java:315)
at org.elasticsearch.client.transport.TransportClientNodesService$ScheduledNodeSampler.run(TransportClientNodesService.java:348)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
警告2: 看这个警告应该是快要发生异常了
[2016-04-19 14:42:31,622][WARN ][monitor.jvm ] [node9-ssd] [gc][young][444][30] duration [2.4s], collections [1]/[3s], total [2.4s]/[3.3s], memory [648.1mb]->[558.6mb]/[11.8gb], all_pools {[young] [270.2mb]->[2mb]/[865.3mb]}{[survivor] [108.1mb]->[108.1mb]/[108.1mb]}{[old] [269.7mb]->[448.4mb]/[10.9gb]}
[2016-04-19 14:42:33,170][INFO ][monitor.jvm ] [node9-ssd] [gc][young][445][31] duration [935ms], collections [1]/[1.5s], total [935ms]/[4.3s], memory [558.6mb]->[689.5mb]/[11.8gb], all_pools {[young] [2mb]->[2mb]/[865.3mb]}{[survivor] [108.1mb]->[108.1mb]/[108.1mb]}{[old] [448.4mb]->[579.3mb]/[10.9gb]}
[2016-04-19 14:42:35,320][WARN ][monitor.jvm ] [node9-ssd] [gc][young][446][32] duration [1.4s], collections [1]/[2.1s], total [1.4s]/[5.7s], memory [689.5mb]->[891.1mb]/[11.8gb], all_pools {[young] [2mb]->[3mb]/[865.3mb]}{[survivor] [108.1mb]->[108.1mb]/[108.1mb]}{[old] [579.3mb]->[780mb]/[10.9gb]}
[2016-04-19 14:42:39,429][WARN ][monitor.jvm ] [node9-ssd] [gc][young][449][33] duration [1.4s], collections [1]/[2.1s], total [1.4s]/[7.2s], memory [1.4gb]->[1gb]/[11.8gb], all_pools {[young] [584.4mb]->[4mb]/[865.3mb]}{[survivor] [108.1mb]->[108.1mb]/[108.1mb]}{[old] [780mb]->[995mb]/[10.9gb]}
[2016-04-19 14:42:45,733][WARN ][monitor.jvm ] [node9-ssd] [gc][young][455][34] duration [1s], collections [1]/[1.3s], total [1s]/[8.2s], memory [1.8gb]->[1.2gb]/[11.8gb], all_pools {[young] [786.3mb]->[7mb]/[865.3mb]}{[survivor] [108.1mb]->[108.1mb]/[108.1mb]}{[old] [995mb]->[1gb]/[10.9gb]}
异常3:
org.elasticsearch.transport.NodeDisconnectedException: [][inet[/ip:9300]][cluster:monitor/nodes/info] disconnected
2016-04-19 10:28:41,269 ERROR com.zkdj.restserver.dao.IndexDao(121): booleanQuery: None of the configured nodes were available: [[node6-ssd][AdKDN-lCQEGtZZrU7q_Y3Q][es6][inet[/ip:9300]], [node7-ssd][khpIoponRtWw0pfbOjBmpQ][es7][inet[/10.45.20.202:9300]]]
异常4:
2016-04-19 10:06:36,283 INFO org.elasticsearch.client.transport(398): [Shatter] failed to get node info for [#transport#-2][es3][inet[/ip:9300]], disconnecting...
org.elasticsearch.transport.ReceiveTimeoutTransportException: [][inet[/ip:9300]][cluster:monitor/nodes/info] request_id [25562] timed out after [6129ms]
at org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:529)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
1 个回复
helloes
赞同来自:
异常3、4也有可能是在full gc,导致STW了(看你的异常1、2,可能性比较大),当然,网络也有关系,不过一般都不会有网络问题吧?