mysql - Solr DIH 错误 : Unable to read: dataimport. 属性

Question

安装 Solr 4 后，我使用 DIH 进行了完全导入，并在错误日志中收到以下错误:

Catalina.out

22-Nov-2012 08:36:21 org.apache.solr.handler.dataimport.DataImporter maybeReloadConfiguration
INFO: Loading DIH Configuration: /opt/solr/collection1/conf/data-config.xml
22-Nov-2012 08:36:21 org.apache.solr.handler.dataimport.config.ConfigParseUtil verifyWithSchema
INFO: id is a required field in SolrSchema . But not found in DataConfig
22-Nov-2012 08:36:21 org.apache.solr.handler.dataimport.config.ConfigParseUtil verifyWithSchema
INFO: latlng is a required field in SolrSchema . But not found in DataConfig
22-Nov-2012 08:36:21 org.apache.solr.handler.dataimport.DataImporter loadDataConfig
INFO: Data Configuration loaded successfully
22-Nov-2012 08:36:21 org.apache.solr.handler.dataimport.DataImporter doFullImport
INFO: Starting Full Import
22-Nov-2012 08:36:21 org.apache.solr.core.SolrCore execute
INFO: [collection1] webapp=/solr path=/dataimport params={command=full-import} status=0 QTime=5 
22-Nov-2012 08:36:21 org.apache.solr.handler.dataimport.SimplePropertiesWriter readIndexerProperties
WARNING: Unable to read: dataimport.properties
22-Nov-2012 08:36:21 org.apache.solr.handler.dataimport.JdbcDataSource$1 call
INFO: Creating a connection for entity listings with URL: jdbc:mysql://localhost/mysite
22-Nov-2012 08:36:21 org.apache.solr.handler.dataimport.JdbcDataSource$1 call
INFO: Time taken for getConnection(): 15
22-Nov-2012 08:36:25 org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: [collection1] webapp=/solr path=/dataimport params={command=full-import} status=0 QTime=5 {deleteByQuery=*:*} 0 5
22-Nov-2012 08:36:25 org.apache.solr.core.SolrCore execute
INFO: [collection1] webapp=/solr path=/admin/logging params={since=1353591348955&wt=json} status=0 QTime=0 
22-Nov-2012 08:36:25 org.apache.solr.handler.dataimport.JdbcDataSource closeConnection
SEVERE: Ignoring Error when closing connection
java.sql.SQLException: Unknown character set index for field '3687' received from server.
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1074)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:988)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:974)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:919)
    at com.mysql.jdbc.ConnectionImpl.getCharsetNameForIndex(ConnectionImpl.java:2976)
    at com.mysql.jdbc.Field.<init>(Field.java:200)
    at com.mysql.jdbc.MysqlIO.unpackField(MysqlIO.java:758)
    at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:423)
    at com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:3096)
    at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:2266)
    at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2687)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2713)
    at com.mysql.jdbc.ConnectionImpl.rollbackNoChecks(ConnectionImpl.java:4893)
    at com.mysql.jdbc.ConnectionImpl.rollback(ConnectionImpl.java:4779)
    at com.mysql.jdbc.ConnectionImpl.realClose(ConnectionImpl.java:4388)
    at com.mysql.jdbc.ConnectionImpl.close(ConnectionImpl.java:1594)
    at org.apache.solr.handler.dataimport.JdbcDataSource.closeConnection(JdbcDataSource.java:400)
    at org.apache.solr.handler.dataimport.JdbcDataSource.close(JdbcDataSource.java:391)
    at org.apache.solr.handler.dataimport.DocBuilder.closeEntityProcessorWrappers(DocBuilder.java:291)
    at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:280)
    at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:382)
    at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:448)
    at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:429)
22-Nov-2012 08:36:25 org.apache.solr.common.SolrException log
SEVERE: Full Import failed:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.OutOfMemoryError: Java heap space
    at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:273)
    at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:382)
    at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:448)
    at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:429)
Caused by: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.OutOfMemoryError: Java heap space
    at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:413)
    at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:326)
    at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:234)
    ... 3 more
Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.OutOfMemoryError: Java heap space
    at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:542)
    at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:411)
    ... 5 more
Caused by: java.lang.OutOfMemoryError: Java heap space

22-Nov-2012 08:36:25 org.apache.solr.update.DirectUpdateHandler2 rollback
INFO: start rollback{flags=0,_version_=0}
22-Nov-2012 08:36:25 org.apache.solr.handler.dataimport.SolrWriter rollback
SEVERE: Exception while solr rollback.
java.lang.NullPointerException
    at org.apache.solr.update.DefaultSolrCoreState.rollbackIndexWriter(DefaultSolrCoreState.java:173)
    at org.apache.solr.update.DirectUpdateHandler2.rollbackWriter(DirectUpdateHandler2.java:150)
    at org.apache.solr.update.DirectUpdateHandler2.rollback(DirectUpdateHandler2.java:625)
    at org.apache.solr.update.processor.RunUpdateProcessor.processRollback(RunUpdateProcessorFactory.java:98)
    at org.apache.solr.update.processor.UpdateRequestProcessor.processRollback(UpdateRequestProcessor.java:72)
    at org.apache.solr.update.processor.LogUpdateProcessor.processRollback(LogUpdateProcessorFactory.java:170)
    at org.apache.solr.handler.dataimport.SolrWriter.rollback(SolrWriter.java:117)
    at org.apache.solr.handler.dataimport.DocBuilder.rollback(DocBuilder.java:319)
    at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:387)
    at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:448)
    at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:429)

令我困惑的是这一行从服务器收到的字段“3687”的未知字符集索引。。任何表中都没有字段 3687!

我基本上正在设置一台新服务器并尝试从旧服务器迁移到此服务器。我使用 Puppet 配置新服务器，并将所有数据库转移到新服务器上的 MySQL 数据库。

服务器是 Ubuntu 12.04，运行 MySQL 5.5.28-0ubuntu0.12.04.2、Tomcat 7.0.26、Solr 4.0.0.2012.08.06.22.50.47。旧服务器是运行 MySQL 5.5.24 的 Ubuntu 12.04

知道完全导入出了什么问题吗？谢谢!

Answer 1

这是由于 JVM 内存不足以及完全导入时使用的 SQL 查询错误