es5.4内存调优

本文基于elkstack日志平台。
本文仅作个人的学习笔记，不作任何形式的商用。

## 1.概述


### 1.1 es进程

#### 1.1.1. ps aux | grep 查看 

es默认设置为2G `-Xms2g -Xmx2g` . 然而主机内存为16G,官方推荐es最优配置为服务器内存的一半，即8G.

    

[elk54@MCLM-L0Q09 ~]$ ps axu | grep elasticsearch

    elk54    14568  2.0 15.8 8008272 2591572 ?     Sl   Jul19 242:09 /usr/bin/java     -Xms2g -Xmx2g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+DisableExplicitGC -XX:+AlwaysPreTouch -server -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -Djdk.io.permissionsUseCanonicalPath=true -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Dlog4j.skipJansi=true -XX:+HeapDumpOnOutOfMemoryError -Des.path.home=/opt/elkstack/elasticsearch -cp /opt/elkstack/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch -d -p process_es_log

#### 1.1.2. 服务器es日志信息
Kibana面板Discover经常出现`Discover: Request Timeout after 30000ms` ，查看es云服务器es日志信息如下：

   

 [2017-07-20T09:28:44,881][WARN ][o.e.m.j.JvmGcMonitorService] [node-255] [gc][565420] overhead, spent [3s] collecting in the last [3s]

    [2017-07-20T09:48:08,881][WARN ][o.e.m.j.JvmGcMonitorService] [node-72] [gc][567405] overhead, spent [2.8s] collecting in the last [2.8s]

    [2017-07-20T09:48:10,160][DEBUG][o.e.a.a.c.n.s.TransportNodesStatsAction] [node-72] failed to execute on node [qhRPXX7uQ4K1XNdKblM_NQ]

    org.elasticsearch.transport.ReceiveTimeoutTransportException: [node-255][30.4.64.255:9300][cluster:monitor/nodes/stats[n]] request_id [1636708] timed out after [15000ms]

        at org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:934) [elasticsearch-5.4.1.jar:5.4.1]

        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:569) [elasticsearch-5.4.1.jar:5.4.1]

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]

        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]

    [2017-07-20T09:48:10,161][WARN ][o.e.a.a.c.n.s.TransportNodesStatsAction] [node-72] not accumulating exceptions, excluding exception from response

    org.elasticsearch.action.FailedNodeException: Failed node [qhRPXX7uQ4K1XNdKblM_NQ]

        at org.elasticsearch.action.support.nodes.TransportNodesAction$AsyncAction.onFailure(TransportNodesAction.java:246) [elasticsearch-5.4.1.jar:5.4.1]

        at org.elasticsearch.action.support.nodes.TransportNodesAction$AsyncAction.access$200(TransportNodesAction.java:160) [elasticsearch-5.4.1.jar:5.4.1]

        at org.elasticsearch.action.support.nodes.TransportNodesAction$AsyncAction$1.handleException(TransportNodesAction.java:218) [elasticsearch-5.4.1.jar:5.4.1]

        at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1050) [elasticsearch-5.4.1.jar:5.4.1]

        at org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:933) [elasticsearch-5.4.1.jar:5.4.1]

        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:569) [elasticsearch-5.4.1.jar:5.4.1]

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]

        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]

    Caused by: org.elasticsearch.transport.ReceiveTimeoutTransportException:     [node-255][30.4.64.255:9300][cluster:monitor/nodes/stats[n]] request_id [1636708] timed out after [15000ms]

        at org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:934) ~[elasticsearch-5.4.1.jar:5.4.1]

        ... 4 more

    [2017-07-20T09:48:22,648][WARN ][o.e.m.j.JvmGcMonitorService] [node-72] [gc][567417] overhead, spent [2.5s] collecting in the last [2.5s]

#### 1.1.3. es内存需要调优
**结论**：es内存调优由默认的2G上调为8G.


## 2.修改配置文件


### 2.1 es配置文件

es安装主目录的config中的配置文件

    

[elk54@MCLM-L0Q09 ~ ]$ cd /opt/elkstack/elasticsearch/config/

[elk54@MCLM-L0Q09 config]$ ls

elasticsearch.yml  jvm.options  log4j2.properties  scripts

#### 2.1.1 jvm.options
* 默认设置为2G修改为8G
* 修改GC

        # Xms represents the initial size of total heap space

        # Xmx represents the maximum size of total heap space

 

        -Xms8g

        -Xmx8g

 

        ################################################################

        ## Expert settings

        ################################################################

        ##

        ## All settings below this section are considered

        ## expert settings. Don't tamper with them unless

        ## you understand what you are doing

        ##

        ################################################################

 

        ## GC configuration

        #-XX:+UseConcMarkSweepGC

        #-XX:CMSInitiatingOccupancyFraction=75

        #-XX:+UseCMSInitiatingOccupancyOnly

        -XX:+UseG1GC

        -XX:MaxGCPauseMillis=200

* jvm.options默认设置
        
        

# Xms represents the initial size of total heap space

        # Xmx represents the maximum size of total heap space



        -Xms2g

        -Xmx2g



        ################################################################

        ## Expert settings

        ################################################################

        ##

        ## All settings below this section are considered

        ## expert settings. Don't tamper with them unless

        ## you understand what you are doing

        ##

        ################################################################



        ## GC configuration

        -XX:+UseConcMarkSweepGC

        -XX:CMSInitiatingOccupancyFraction=75

        -XX:+UseCMSInitiatingOccupancyOnly

 
#### 2.1.2 elasticsearch.yml
* bootstrap.memory_lockb去除注释，设置为true
        

        # ----------------------------------- Memory -----------------------------------

        #

        # Lock the memory on startup:

        #

        bootstrap.memory_lock: true

        #

* jvm.options默认设置

        # ----------------------------------- Memory -----------------------------------

        #

        # Lock the memory on startup:

        #

        #bootstrap.memory_lock: true

        #

        

### 2.2 服务器配置文件
服务器文件主要是系统设置，注意用root用户修改。


#### 2.2.1 sysctl.conf

* 增添vm.max_map_count=262144,参照es官网

        [root@MCLM-L0Q09 ~ ] vi /etc/sysctl.conf

        # For OS Default.

        vm.min_free_kbytes = 409600

        vm.vfs_cache_pressure = 200

        vm.swappiness = 0

        fs.file-max = 6815744

        fs.aio-max-nr = 1048576

        kernel.sysrq = 1

        vm.max_map_count = 262144

* 执行命令，使配置文件生效

        [root@MCLM-L0Q09 ~ ] sysctl –p

#### 2.2.2 limits.conf
* 添加es启动用户的设置，若不添加启动报错，如下所示elk54为服务器用户

        [root@MCLM-L0Q09 ~ ] vi /etc/security/limits.conf

        elk54 soft memlock unlimited

        elk54 hard memlock unlimited