这是一个公司要求logstash测试的配置,从oracle中读取数据,导入到mysql中去
input {
stdin {
}
jdbc {
jdbc_driver_library => "/usr/share/logstash/config/ojdbc6.jar"
jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
jdbc_connection_string => "jdbc:oracle:thin:@//192.168.10.5:1521/orcl"
jdbc_user => "sdystr"
jdbc_password => "shandong"
jdbc_paging_enabled => "true"
jdbc_page_size => "50000"
jdbc_default_timezone => "Asia/Shanghai"
statement => "SELECT ID FROM PERSON_INFO where rownum <= 10 and create_datetime >= :sql_last_value"
# schedule => "* * * * * Asia/Shanghai"
# type => ""#####LAB_SERVER_NO#####""
record_last_run => "true"
use_column_value => "true"
tracking_column_type => "timestamp"
tracking_column => "ID"
last_run_metadata_path => "/usr/share/logstash/config/last_datetime/PERSON_INFO"
clean_run => "false"
lowercase_column_names => "false"
}
}
filter {
json {
source => "message"
remove_field => ["message","@version","host","@timestamp"]
}
}
output {
jdbc {
driver_jar_path => "/opt/logstash-6.3.1/config/mysql-connector-java-8.0.11.jar"
driver_class => "com.mysql.jdbc.Driver"
connection_string => "jdbc:mysql://192.168.10.61:3306/ystr2?user=ystr2&password=ystr2"
statement => ["insert into PERSON_INFO(ID) values(?)","%{ID}"]
}
}
现在问题是,获取数据后发现,mysql上的ID字段是无限循环的,如附件所示,我只检索了10个数据,但是这些数据总是无限循环。
请教一下各位
input {
stdin {
}
jdbc {
jdbc_driver_library => "/usr/share/logstash/config/ojdbc6.jar"
jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
jdbc_connection_string => "jdbc:oracle:thin:@//192.168.10.5:1521/orcl"
jdbc_user => "sdystr"
jdbc_password => "shandong"
jdbc_paging_enabled => "true"
jdbc_page_size => "50000"
jdbc_default_timezone => "Asia/Shanghai"
statement => "SELECT ID FROM PERSON_INFO where rownum <= 10 and create_datetime >= :sql_last_value"
# schedule => "* * * * * Asia/Shanghai"
# type => ""#####LAB_SERVER_NO#####""
record_last_run => "true"
use_column_value => "true"
tracking_column_type => "timestamp"
tracking_column => "ID"
last_run_metadata_path => "/usr/share/logstash/config/last_datetime/PERSON_INFO"
clean_run => "false"
lowercase_column_names => "false"
}
}
filter {
json {
source => "message"
remove_field => ["message","@version","host","@timestamp"]
}
}
output {
jdbc {
driver_jar_path => "/opt/logstash-6.3.1/config/mysql-connector-java-8.0.11.jar"
driver_class => "com.mysql.jdbc.Driver"
connection_string => "jdbc:mysql://192.168.10.61:3306/ystr2?user=ystr2&password=ystr2"
statement => ["insert into PERSON_INFO(ID) values(?)","%{ID}"]
}
}
现在问题是,获取数据后发现,mysql上的ID字段是无限循环的,如附件所示,我只检索了10个数据,但是这些数据总是无限循环。
请教一下各位
1 个回复
laoyang360 - 《一本书讲透Elasticsearch》作者,Elastic认证工程师 [死磕Elasitcsearch]知识星球地址:http://t.cn/RmwM3N9;微信公众号:铭毅天下; 博客:https://elastic.blog.csdn.net
赞同来自:
确认方法:配置的文件会打印输出的