JDBC作为JAVA访问数据库的一套规范与标准,统一了数据库操作的API,大大简化了程序开发工作。不过由于历史原因,MySQL对JDBC默认的实现与规范定义或者说其它数据库如Oracle并不一致,为了更完整记录这些差异,计划抽空写个系列,整理下这些可能会误解的常用功能。第一篇首先聊聊PreparedStatement。

MySQL JDBC PreparedStatement

Prepare SQL的产生原因与实现原理

数据库SQL执行过程包括以下阶段: 词法分析->语法分析->语义分析->执行计划优化->执行。【词法分析->语法分析】这两个阶段称之为硬解析。词法分析识别SQL中每个词,语法分析解析SQL语句是否符合(SQL92、99、方言等)语法,并得到一棵语法树。

其实基于SQL的架构设计,基本都有这样一个处理过程,TDDL、ShardingJDBC、MyCAT都如此,当然这些产品都相比于数据库,支持的关键词、语法都只是其子集。另外SQL解析器可基于Yacc、Lex、Antlr、Javacc等构建,当然如果对解析性能要更好要求,则需要进行一个纯手工编写的解析器,例如阿里的Druid中的SQL解析器,应用可基于Vistor模式进行使用。

Prepare SQL也叫预编译SQL、Prepared Statements或者Parameterized Statements,就是将这类SQL中的值用占位符?替代,可以视为将SQL语句模板化或者说参数化。预编译语句的优势在于归纳为:一次编译、多次运行,省去了解析优化等过程。

Prepare的出现就是为了优化硬解析的问题,Prepare在服务器端的执行过程如下:

  • 【Prepare】 接收客户端带?的SQL, 硬解析得到语法树(stmt->Lex), 缓存在线程所在的PS cache中。此cache是一个HASH MAP. Key为stmt->id. 然后返回客户端stmt->id等信息。

  • 【Execute】接收客户端stmt->id和参数等信息(客户端不需要再发SQL过来)。服务器根据stmt->id在PS cache中查找得到硬解析后的stmt, 并设置参数,就可以继续后面的优化和执行。
    Prepare在execute阶段可以节省硬解析的时间。因此prepare适用于频繁执行的SQL。

Prepare的另一个作用是防止SQL注入,这个是纯客户端JDBC通过转义实现的。这也是一般更推荐使用PreparedStatement而不是Statement的主要理由。防SQL注入的具体实现可以参见MySQL驱动中com.mysql.jdbc.PreparedStatement.setString代码。

MySQL驱动中PrepareStament的实现逻辑

看完Prepare的功能原理后,我们看下JDBC操作MySQL时的PreparaStatement,
在com.mysql.jdbc.ConnectionImpl类中

public java.sql.PreparedStatement prepareStatement(String sql,int resultSetType, int resultSetConcurrency) throws SQLException {synchronized (getConnectionMutex()) {checkClosed();//// FIXME: Create warnings if can't create results of the given// type or concurrency//PreparedStatement pStmt = null;boolean canServerPrepare = true;String nativeSql = getProcessEscapeCodesForPrepStmts() ? nativeSQL(sql): sql;if (this.useServerPreparedStmts && getEmulateUnsupportedPstmts()) {canServerPrepare = canHandleAsServerPreparedStatement(nativeSql);}if (this.useServerPreparedStmts && canServerPrepare) {if (this.getCachePreparedStatements()) {synchronized (this.serverSideStatementCache) {pStmt = (com.mysql.jdbc.ServerPreparedStatement)this.serverSideStatementCache.remove(sql);if (pStmt != null) {((com.mysql.jdbc.ServerPreparedStatement)pStmt).setClosed(false);pStmt.clearParameters();}if (pStmt == null) {try {pStmt = ServerPreparedStatement.getInstance(getLoadBalanceSafeProxy(), nativeSql,this.database, resultSetType, resultSetConcurrency);if (sql.length() < getPreparedStatementCacheSqlLimit()) {((com.mysql.jdbc.ServerPreparedStatement)pStmt).isCached = true;}pStmt.setResultSetType(resultSetType);pStmt.setResultSetConcurrency(resultSetConcurrency);} catch (SQLException sqlEx) {// Punt, if necessaryif (getEmulateUnsupportedPstmts()) {pStmt = (PreparedStatement) clientPrepareStatement(nativeSql, resultSetType, resultSetConcurrency, false);if (sql.length() < getPreparedStatementCacheSqlLimit()) {this.serverSideStatementCheckCache.put(sql, Boolean.FALSE);}} else {throw sqlEx;}}}}} else {try {pStmt = ServerPreparedStatement.getInstance(getLoadBalanceSafeProxy(), nativeSql,this.database, resultSetType, resultSetConcurrency);pStmt.setResultSetType(resultSetType);pStmt.setResultSetConcurrency(resultSetConcurrency);} catch (SQLException sqlEx) {// Punt, if necessaryif (getEmulateUnsupportedPstmts()) {pStmt = (PreparedStatement) clientPrepareStatement(nativeSql, resultSetType, resultSetConcurrency, false);} else {throw sqlEx;}}}} else {pStmt = (PreparedStatement) clientPrepareStatement(nativeSql, resultSetType, resultSetConcurrency, false);}return pStmt;}}

从以上源代码中,我们看到其实MySQL的Prepare竟然有两种,分为是客户端(JDBC4PreparedStatement)与服务器端(ServerPrepareStatement),根据应用连接参数设置(useServerPrepStmts),选择不同的PreparedStatement。另外还会根据缓存参数设置(cachePrepStmts),选择是否从缓存重获取解析对象,该缓存是针对连接的,这对于应用端使用连接池的场景是比较适用的。

不同参数对应的Prepare区别

分别设置不同参数,查看服务器端操作日志。

  1. 使用客户端PreparedStatement,不开启缓存
public static void selectWithClientPs(int count) throws SQLException{Connection connection = DriverManager.getConnection("jdbc:mysql://localhost:3306/test", "root", "123456");long begin = System.currentTimeMillis();for(int i=0;i<count;i++){PreparedStatement statement = connection.prepareStatement("select * from test where id= ?");statement.setInt(1, i);ResultSet resultSet = statement.executeQuery();resultSet.close();statement.close();}System.out.println("selectWithClientPs span time="+(System.currentTimeMillis()-begin) + "ms");connection.close();}

MySQL服务器执行日志:

Time                 Id Command    Argument
181225 13:23:43     1 Connect   root@localhost on test1 Query /* mysql-connector-java-5.1.46 ( Revision: 9cc87a48e75c2d2e87c1a293b2862ce651cb256e ) */SELECT  @@session.auto_increment_increment AS auto_increment_increment, @@character_set_client AS character_set_client, @@character_set_connection AS character_set_connection, @@character_set_results AS character_set_results, @@character_set_server AS character_set_server, @@collation_server AS collation_server, @@init_connect AS init_connect, @@interactive_timeout AS interactive_timeout, @@license AS license, @@lower_case_table_names AS lower_case_table_names, @@max_allowed_packet AS max_allowed_packet, @@net_buffer_length AS net_buffer_length, @@net_write_timeout AS net_write_timeout, @@query_cache_size AS query_cache_size, @@query_cache_type AS query_cache_type, @@sql_mode AS sql_mode, @@system_time_zone AS system_time_zone, @@time_zone AS time_zone, @@tx_isolation AS transaction_isolation, @@wait_timeout AS wait_timeout1 Query SET NAMES latin11 Query SET character_set_results = NULL1 Query SET autocommit=11 Query select * from test where id= 01 Query select * from test where id= 11 Query select * from test where id= 21 Quit
  1. 使用客户端PreparedStatement,开启缓存
    public static void selectWithClientPsAndCache(int count) throws SQLException{Connection connection = DriverManager.getConnection("jdbc:mysql://localhost:3306/test?cachePrepStmts=true", "root", "123456");long begin = System.currentTimeMillis();for(int i=0;i<count;i++){PreparedStatement statement = connection.prepareStatement("select * from test where id= ?");statement.setInt(1, i);ResultSet resultSet = statement.executeQuery();resultSet.close();statement.close();}System.out.println("selectWithClientPsAndCache span time="+(System.currentTimeMillis()-begin) + "ms");connection.close();}

MySQL服务器执行日志:

            2 Connect   root@localhost on test2 Query /* mysql-connector-java-5.1.46 ( Revision: 9cc87a48e75c2d2e87c1a293b2862ce651cb256e ) */SELECT  @@session.auto_increment_increment AS auto_increment_increment, @@character_set_client AS character_set_client, @@character_set_connection AS character_set_connection, @@character_set_results AS character_set_results, @@character_set_server AS character_set_server, @@collation_server AS collation_server, @@init_connect AS init_connect, @@interactive_timeout AS interactive_timeout, @@license AS license, @@lower_case_table_names AS lower_case_table_names, @@max_allowed_packet AS max_allowed_packet, @@net_buffer_length AS net_buffer_length, @@net_write_timeout AS net_write_timeout, @@query_cache_size AS query_cache_size, @@query_cache_type AS query_cache_type, @@sql_mode AS sql_mode, @@system_time_zone AS system_time_zone, @@time_zone AS time_zone, @@tx_isolation AS transaction_isolation, @@wait_timeout AS wait_timeout2 Query SET NAMES latin12 Query SET character_set_results = NULL2 Query SET autocommit=12 Query select * from test where id= 02 Query select * from test where id= 12 Query select * from test where id= 22 Quit  
  1. 使用服务器端PreparedStatement,不开启缓存
public static void selectWithServerPs(int count) throws SQLException{Connection connection = DriverManager.getConnection("jdbc:mysql://localhost:3306/test?useServerPrepStmts=true", "root", "123456");long begin = System.currentTimeMillis();for(int i=0;i<count;i++){PreparedStatement statement = connection.prepareStatement("select * from test where id= ?");statement.setInt(1, i);ResultSet resultSet = statement.executeQuery();resultSet.close();statement.close();}System.out.println("selectWithServerPs span time="+(System.currentTimeMillis()-begin) + "ms");connection.close();}

MySQL服务器执行日志:

            3 Connect   root@localhost on test3 Query /* mysql-connector-java-5.1.46 ( Revision: 9cc87a48e75c2d2e87c1a293b2862ce651cb256e ) */SELECT  @@session.auto_increment_increment AS auto_increment_increment, @@character_set_client AS character_set_client, @@character_set_connection AS character_set_connection, @@character_set_results AS character_set_results, @@character_set_server AS character_set_server, @@collation_server AS collation_server, @@init_connect AS init_connect, @@interactive_timeout AS interactive_timeout, @@license AS license, @@lower_case_table_names AS lower_case_table_names, @@max_allowed_packet AS max_allowed_packet, @@net_buffer_length AS net_buffer_length, @@net_write_timeout AS net_write_timeout, @@query_cache_size AS query_cache_size, @@query_cache_type AS query_cache_type, @@sql_mode AS sql_mode, @@system_time_zone AS system_time_zone, @@time_zone AS time_zone, @@tx_isolation AS transaction_isolation, @@wait_timeout AS wait_timeout3 Query SET NAMES latin13 Query SET character_set_results = NULL3 Query SET autocommit=13 Prepare   select * from test where id= ?3 Execute   select * from test where id= 03 Close stmt    3 Prepare   select * from test where id= ?3 Execute   select * from test where id= 13 Close stmt    3 Prepare   select * from test where id= ?3 Execute   select * from test where id= 23 Close stmt    3 Quit  
  1. 使用服务器端PreparedStatement,开启缓存
   public static void selectWithServerPsAndCache(int count) throws SQLException{Connection connection = DriverManager.getConnection("jdbc:mysql://localhost:3306/test?useServerPrepStmts=true&cachePrepStmts=true", "root", "123456");long begin = System.currentTimeMillis();for(int i=0;i<count;i++){PreparedStatement statement = connection.prepareStatement("select * from test where id= ?");statement.setInt(1, i);ResultSet resultSet = statement.executeQuery();resultSet.close();statement.close();}System.out.println("selectWithServerPsAndCache span time="+(System.currentTimeMillis()-begin) + "ms");connection.close();}

MySQL服务器执行日志:

            4 Connect   root@localhost on test4 Query /* mysql-connector-java-5.1.46 ( Revision: 9cc87a48e75c2d2e87c1a293b2862ce651cb256e ) */SELECT  @@session.auto_increment_increment AS auto_increment_increment, @@character_set_client AS character_set_client, @@character_set_connection AS character_set_connection, @@character_set_results AS character_set_results, @@character_set_server AS character_set_server, @@collation_server AS collation_server, @@init_connect AS init_connect, @@interactive_timeout AS interactive_timeout, @@license AS license, @@lower_case_table_names AS lower_case_table_names, @@max_allowed_packet AS max_allowed_packet, @@net_buffer_length AS net_buffer_length, @@net_write_timeout AS net_write_timeout, @@query_cache_size AS query_cache_size, @@query_cache_type AS query_cache_type, @@sql_mode AS sql_mode, @@system_time_zone AS system_time_zone, @@time_zone AS time_zone, @@tx_isolation AS transaction_isolation, @@wait_timeout AS wait_timeout4 Query SET NAMES latin14 Query SET character_set_results = NULL4 Query SET autocommit=14 Prepare   select * from test where id= ?4 Execute   select * from test where id= 04 Execute   select * from test where id= 14 Execute   select * from test where id= 24 Quit  

另外对这四种情况分别进行5000次查询,执行时间对比

selectWithClientPs span time=232601ms
selectWithClientPsAndCache span time=231493ms
selectWithServerPs span time=233999ms
selectWithServerPsAndCache span time=231262ms

结论:

  1. 使用客户端PreparedStatement,无论是否开启缓存,服务器端都不使用prepare,即硬解析时间不会减少,而且客户端缓存执行时间差别并不大。
  2. 使用服务器端PreparedStatement,如果开启缓存,则会使用prepare,硬解析仅为一次;如果不开启缓存,每次PreparedStatement进行close后,都需要重新进行prepare。
  3. 从测试效果来看,如果SQL本身比较简单,服务器端prepare并没有太大优势,使用客户端prepare即可,如果SQL较为复杂,则可尝试开启服务器端prepare,网上也有文章做过测试称可提高7%性能,不过笔者本地测试提升不到1%,测试数据实际跟SQL的复杂程度有关。

另外还有两个参数

  • prepStmtCacheSize参数,控制缓存的条数,MySQL驱动默认是25,实际使用时一般会根据需要调整大些;
  • prepStmtCacheSqlLimit参数,控制长度多大的SQL可以被缓存,MySQL驱动默认是256,实际使用时如果SQL较大,可调整大些。

关于PrepareStament,MySQL还有一个参数max_prepared_stmt_count,默认值为16382。

mysql> show variables like "max_prepare%";
+----------------------------+----------------------+
| Variable_name              | Value                |
+----------------------------+----------------------+
| max_prepared_stmt_count    | 16382                | 

如果创建的PS数量超过这个数值,则会报以下错误:

ERROR 1461 (42000): Can’t create more than max_prepared_stmt_count statements

可根据需要需要调大该值,不过调整前应先检查应用是否正确关闭PreparedStament对象。

由上可见,出于历史版本的迭代,MySQL在实现JDBC规范接口时,有很多功能最开始并不算“真正”的实现,而是客户端类facade的设计,因此很多默认功能与常识并不一致,这些需要我们在实际使用中特别注意。

MySQL JDBC PreparedStatement相关推荐

  1. java.lang.AbstractMethodError: com.mysql.jdbc.PreparedStatement.setCharacterStream(ILjava/io/Reader;

    出现上述异常是在使用MySQL进行大文本数据的读写时,使用PreparedStatement中的 setCharacterStream(int parameterIndex,Reader reader ...

  2. mysql date_trunc_com.mysql.jdbc.MysqlDataTruncation: Data trunca...

    连接的是mysql数据库,插入数据时,控制台报: com.mysql.jdbc.MysqlDataTruncation: Data truncation: Data too long for colu ...

  3. Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Query was empty

    1 错误描写叙述 at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(Invocable ...

  4. mysql,jdbc、连接池

    show processlist; select * from information_schema.processlist; Command:The type of command the thre ...

  5. 问题记录——com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure...

    最近在搞一个Spring boot + Mybatis + Mysql的项目,用Mybatis访问数据库时,报了如下的错误,先在网上搜索了,试了各种办法都不行, 奇葩的是,连接另外1个数据库又没问题. ...

  6. Caused by: com.mysql.jdbc.MysqlDataTruncation: Data truncation: Incorrect datetime value:

    问题:ERROR JDBCExceptionReporter:72 - Data truncation: Incorrect datetime value: '' for column 'create ...

  7. Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown column 't.statis_date'

    1.错误原因 [ERROR:]2015-04-18 13:20:31,883 [异常拦截] com.skycloud.oa.exception.ExceptionHandler org.hiberna ...

  8. 解决com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException:

    解决com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: 1. 报错内容 com.mysql.jdbc.exceptions.jdbc4 ...

  9. FAQ(43): com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error in your SQL sy

    2018/1/3 spring对Mybatis整合, 看Log: org.springframework.jdbc.BadSqlGrammarException: ### Error querying ...

最新文章

  1. BEP 7:CUDA外部内存管理插件(上)
  2. An eventually consistent data model for Erlang (and Riak)
  3. vue-cli启动项目运行_SpringBoot2.0 基础案例(17):自定义启动页,项目打包和指定运行环境...
  4. STM32开发 -- 可调直流稳压电源
  5. B - Fibonacci Again
  6. FreeMarker 用户自定义指令@(3.4)
  7. Win8 64位安装Oracle 11g时错
  8. android 中如何监听按键的长按事件
  9. arcgis取反+掩膜提取
  10. NoSql数据库确实非常适合网站
  11. protues 仿真 12864转OLED接法
  12. 视频工具mencoder
  13. 如何删除电脑里的android驱动程序,驱动安装失败 如何手动清除旧驱动程序
  14. SSL Virtual Private Network的技术分析
  15. 趣图 | 念念不忘必有回响
  16. AR模型收敛:特征根在单位圆内
  17. 防止暴利破解,拒绝ip登陆
  18. 关于ping以及TTL的分析
  19. 审计需要掌握的计算机语言,审计人员应该具备的素质
  20. apache2.4开启GZIP压缩

热门文章

  1. 树莓派之树莓派系统安装
  2. 日历插件(项目总结)(包括mobiscroll.js LCalendar 和Calendar这三个日历插件)
  3. latex表格内容上下居中_LaTeX表格紧跟文字 (不影响下方文本对齐)
  4. 计算机控制键功能,电脑ctrl键的功能
  5. 安卓沉浸式状态栏_安卓平板也能有品质感,小新Pad Pro上手
  6. 19杭电计算机考研科目,2019杭电计算机考研初试科目、参考书目、报录比汇总
  7. 如何看待互联网公司 996 现象,是种什么样的体验?
  8. mybatis中resultMap和resultType的详细用法
  9. 浅谈栈(Stack)实现
  10. Mac 安装第三方来源软件