问题描述:
在使用logstash同步mysql数据库中的数据,在使用logstash7.4.1以前的版本,输出的sql日志如下:
[2019-12-09T10:50:01,585][INFO ][logstash.inputs.jdbc ] (0.009082s) SELECT version()
[2019-12-09T10:50:01,634][INFO ][logstash.inputs.jdbc ] (0.001152s) SELECT version()
[2019-12-09T10:50:01,841][INFO ][logstash.inputs.jdbc ] (0.002531s) SELECT count(*) AS `count` FROM (select * from t_test where updatetime > '1970-01-01 08:00:00' and updatetime < date_add(NOW(), interval -1 second) order by updatetime asc) AS `t1` LIMIT 1
[2019-12-09T10:50:01,892][INFO ][logstash.inputs.jdbc ] (0.009893s) SELECT * FROM (select * from t_test where updatetime > '1970-01-01 08:00:00' and updatetime < date_add(NOW(), interval -1 second) order by updatetime asc) AS `t1` LIMIT 1 OFFSET 0
[2019-12-09T10:50:02,032][INFO ][logstash.inputs.jdbc ] (0.001794s) SELECT * FROM (select * from t_test where updatetime > '1970-01-01 08:00:00' and updatetime < date_add(NOW(), interval -1 second) order by updatetime asc) AS `t1` LIMIT 1 OFFSET 1
[2019-12-09T10:50:02,043][INFO ][logstash.inputs.jdbc ] (0.001604s) SELECT * FROM (select * from t_test where updatetime > '1970-01-01 08:00:00' and updatetime < date_add(NOW(), interval -1 second) order by updatetime asc) AS `t1` LIMIT 1 OFFSET 2
[2019-12-09T10:50:02,060][INFO ][logstash.inputs.jdbc ] (0.001694s) SELECT * FROM (select * from t_test where updatetime > '1970-01-01 08:00:00' and updatetime < date_add(NOW(), interval -1 second) order by updatetime asc) AS `t1` LIMIT 1 OFFSET 3
[2019-12-09T10:50:02,076][INFO ][logstash.inputs.jdbc ] (0.001766s) SELECT * FROM (select * from t_test where updatetime > '1970-01-01 08:00:00' and updatetime < date_add(NOW(), interval -1 second) order by updatetime asc) AS `t1` LIMIT 1 OFFSET 4
使用logstash7.4.1以后的版本,输出的sql日志如下:
[2019-12-09T15:42:01,445][INFO ][logstash.inputs.jdbc ][main] (0.009087s) select * from t_test where updatetime < date_add(NOW(), interval -1 second) order by updatetime asc
logstash input 脚步如下:
jdbc {
jdbc_driver_library => "mysql-connector-java-5.1.47.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_connection_string => "jdbc:mysql://127.0.0.1:3306/test"
jdbc_user => "root"
jdbc_password => "test"
jdbc_default_timezone =>"Asia/Shanghai"
jdbc_paging_enabled => true
jdbc_page_size => 1
schedule => "* * * * *"
record_last_run => true
use_column_value => true
tracking_column => "updatetime"
tracking_column_type => "timestamp"
last_run_metadata_path => "/Users/future/work/logstash-7.4.2/lastrunmetadata/test"
statement => "select * from t_test where updatetime < date_add(NOW(), interval -1 second) order by updatetime asc"
}
请问为什么在7.4.1及以后的版本在输出的sql日志中没有LIMIT 1 OFFSET 0分页的相关信息?
在使用logstash同步mysql数据库中的数据,在使用logstash7.4.1以前的版本,输出的sql日志如下:
[2019-12-09T10:50:01,585][INFO ][logstash.inputs.jdbc ] (0.009082s) SELECT version()
[2019-12-09T10:50:01,634][INFO ][logstash.inputs.jdbc ] (0.001152s) SELECT version()
[2019-12-09T10:50:01,841][INFO ][logstash.inputs.jdbc ] (0.002531s) SELECT count(*) AS `count` FROM (select * from t_test where updatetime > '1970-01-01 08:00:00' and updatetime < date_add(NOW(), interval -1 second) order by updatetime asc) AS `t1` LIMIT 1
[2019-12-09T10:50:01,892][INFO ][logstash.inputs.jdbc ] (0.009893s) SELECT * FROM (select * from t_test where updatetime > '1970-01-01 08:00:00' and updatetime < date_add(NOW(), interval -1 second) order by updatetime asc) AS `t1` LIMIT 1 OFFSET 0
[2019-12-09T10:50:02,032][INFO ][logstash.inputs.jdbc ] (0.001794s) SELECT * FROM (select * from t_test where updatetime > '1970-01-01 08:00:00' and updatetime < date_add(NOW(), interval -1 second) order by updatetime asc) AS `t1` LIMIT 1 OFFSET 1
[2019-12-09T10:50:02,043][INFO ][logstash.inputs.jdbc ] (0.001604s) SELECT * FROM (select * from t_test where updatetime > '1970-01-01 08:00:00' and updatetime < date_add(NOW(), interval -1 second) order by updatetime asc) AS `t1` LIMIT 1 OFFSET 2
[2019-12-09T10:50:02,060][INFO ][logstash.inputs.jdbc ] (0.001694s) SELECT * FROM (select * from t_test where updatetime > '1970-01-01 08:00:00' and updatetime < date_add(NOW(), interval -1 second) order by updatetime asc) AS `t1` LIMIT 1 OFFSET 3
[2019-12-09T10:50:02,076][INFO ][logstash.inputs.jdbc ] (0.001766s) SELECT * FROM (select * from t_test where updatetime > '1970-01-01 08:00:00' and updatetime < date_add(NOW(), interval -1 second) order by updatetime asc) AS `t1` LIMIT 1 OFFSET 4
使用logstash7.4.1以后的版本,输出的sql日志如下:
[2019-12-09T15:42:01,445][INFO ][logstash.inputs.jdbc ][main] (0.009087s) select * from t_test where updatetime < date_add(NOW(), interval -1 second) order by updatetime asc
logstash input 脚步如下:
jdbc {
jdbc_driver_library => "mysql-connector-java-5.1.47.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_connection_string => "jdbc:mysql://127.0.0.1:3306/test"
jdbc_user => "root"
jdbc_password => "test"
jdbc_default_timezone =>"Asia/Shanghai"
jdbc_paging_enabled => true
jdbc_page_size => 1
schedule => "* * * * *"
record_last_run => true
use_column_value => true
tracking_column => "updatetime"
tracking_column_type => "timestamp"
last_run_metadata_path => "/Users/future/work/logstash-7.4.2/lastrunmetadata/test"
statement => "select * from t_test where updatetime < date_add(NOW(), interval -1 second) order by updatetime asc"
}
请问为什么在7.4.1及以后的版本在输出的sql日志中没有LIMIT 1 OFFSET 0分页的相关信息?
1 个回复
GLC
赞同来自:
jdbc_page_size => 1 分页数量也太少了。