java - 如何在几毫秒内将数千条记录更新到 MySQL 数据库中

Question

我想在不到一秒的时间内将大约 10K 条记录更新到 MySQL 数据库中。我写了下面的代码，大约需要 6-8 秒才能将记录列表更新到数据库中。

public void updateResultList(List<?> list) {
            String user = "root";
            String pass = "root";
            String jdbcUrl = "jdbc:mysql://12.1.1.1/db_1?useSSL=false";
            String driver = "com.mysql.jdbc.Driver";
            PreparedStatement pstm = null;

            try {
                Class.forName(driver);
                Connection myConn = DriverManager.getConnection(jdbcUrl, user, pass);
                myConn.setAutoCommit(false);
                for(int i=0; i<list.size(); i++) {
                    Object[] row = (Object[]) list.get(i);
                    int candidateID = Integer.valueOf(String.valueOf(row[0]));
                    String result = String.valueOf(row[14]);
                    int score = Integer.valueOf(String.valueOf(row[19]));
                    String uploadState = (String) row[20];

                    String sql = "UPDATE personal_info SET result = ?, score = ?, uploadState = ? "
                                + " WHERE CandidateID = ?";

                    pstm = (PreparedStatement) myConn.prepareStatement(sql);
                    pstm.setString(1, result);
                    pstm.setInt(2, score);
                    pstm.setString(3, uploadState);
                    pstm.setInt(4, candidateID);
                    pstm.addBatch();
                    pstm.executeBatch();

                }
                myConn.commit();
                myConn.setAutoCommit(true);
                pstm.close();
                myConn.close();

            }
            catch (Exception exc) {
                exc.printStackTrace();
                try {
                    throw new ServletException(exc);
                } catch (ServletException e) {
                    // TODO Auto-generated catch block
                    e.printStackTrace();
                }
            }   
       }

请让我知道您对优化此代码以提高性能的意见。

Answer 1

首先，你只需要初始化一次prepareStatement，你需要在for循环之前初始化它

其次，你应该避免执行 pstm.executeBatch(); 每次循环都会消耗更多的资源，你需要执行指定的数量，比如 100,500 或更多，也做不要只在for循环外执行一次，因为它会消耗更多的内存资源

Class.forName(driver);
Connection myConn = DriverManager.getConnection(jdbcUrl, user, pass);
myConn.setAutoCommit(false);
String sql = "UPDATE personal_info SET result = ?, score = ?, uploadState = ? "
                + " WHERE CandidateID = ?";
pstm = (PreparedStatement) myConn.prepareStatement(sql);
for(int i=0; i<list.size(); i++) {
    Object[] row = (Object[]) list.get(i);
    int candidateID = Integer.valueOf(String.valueOf(row[0]));
    String result = String.valueOf(row[14]);
    int score = Integer.valueOf(String.valueOf(row[19]));
    String uploadState = (String) row[20];
    pstm.setString(1, result);
    pstm.setInt(2, score);
    pstm.setString(3, uploadState);
    pstm.setInt(4, candidateID);
    pstm.addBatch();
    if(i%500==0){//execute when it meet a specified amount
        pstm.executeBatch();
    }
}
pstm.executeBatch();
myConn.commit();
myConn.setAutoCommit(true);