java - 为 DS-2542(XOAI 日期粒度)修补 Dspace 不会导致行为发生变化

标签 java maven tomcat tomcat6 dspace

我正在尝试向运行 Dspace 5.1(带有一些附加模块)的测试环境应用补丁，以修复由于日期粒度不正确而导致的 OAI 收集问题。问题是 DS-2542 XOAI does not support non granular YYYY-MM-DD harvesting properly

环境

  • Dspace 5.1(继承并随后从4.2升级)
  • Tomcat 版本 6
  • Oracle Java 1.7 更新 79
  • 红帽企业 Linux 版本 6.6

我如何应用补丁

首先，我需要从 https://github.com/DSpace/DSpace/pull/912 获取 Dspace (dspace-oai) 的补丁。这给了我以下补丁( as a diff )，我将其放入/tmp/

diff --git a/dspace-oai/src/main/java/org/dspace/xoai/filter/DateUntilFilter.java b/dspace-oai/src/main/java/org/dspace/xoai/filter/DateUntilFilter.java
index 1995fc0..cdb17d9 100644
--- a/dspace-oai/src/main/java/org/dspace/xoai/filter/DateUntilFilter.java
+++ b/dspace-oai/src/main/java/org/dspace/xoai/filter/DateUntilFilter.java
@@ -50,6 +50,11 @@ public boolean isShown(DSpaceItem item)
     public SolrFilterResult buildSolrQuery()
     {
         String format = dateProvider.format(date).replace("Z", ".999Z"); // Tweak to set the millisecon
+        if (format.substring(11, 19).equals("00:00:00"))
+        {
+            format = format.substring(0, 11) + "23:59:59" + format.substring(19);
+        }
+
         return new SolrFilterResult("item.lastmodified:[* TO "
                 + ClientUtils.escapeQueryChars(format) + "]");
     }
diff --git a/dspace-oai/src/main/java/org/dspace/xoai/util/DateUtils.java b/dspace-oai/src/main/java/org/dspace/xoai/util/DateUtils.java
index e968414..b73955d 100644
--- a/dspace-oai/src/main/java/org/dspace/xoai/util/DateUtils.java
+++ b/dspace-oai/src/main/java/org/dspace/xoai/util/DateUtils.java
@@ -31,8 +31,7 @@ public static String format(Date date)
     }
     public static String format(Date date, boolean init)
     {
-       SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.'000Z'");
-       if (!init) sdf = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.'999Z'");
+        SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'");
         // We indicate that the returned date is in Zulu time (UTC) so we have
         // to set the time zone of sdf correct.
         sdf.setTimeZone(TimeZone.getTimeZone("ZULU"));

我已使用以下命令序列应用了补丁:

sudo su - builder
  cd /usr/local/src/
    git clone 'https://github.com/lyncode/xoai.git' xoai
    cd xoai/
      git branch -a
      git checkout 3.x
      less pom.xml    <-- verify that com.lyncode version is 3.2.10-SNAPSHOT
      mvn package install
      cd ..
    cd dspace-src/
      patch -p1 < /tmp/473f2faaba99671b55372bcca1604aea2acf9601.diff    # (needed tweaking)
      vim dspace-oai/pom.xml   # change <xoai.version> from 3.2.9 to 3.2.10-SNAPSHOT
      mvn package -U -P \!dspace-jspui,\!dspace-lni,\!dspace-rest,\!dspace-swordv2,\!dspace-rdf,\!dspace-xmlui-mirage2
      exit
cd /usr/local/src/dspace-src/dspace/target/dspace-installer/
  ant update
  chown -Rc tomcat:tomcat /usr/local/dspace/
  service tomcat6 restart

测试方法

修补后的结果与修补前的结果相同。使用2015-04-07仍然失败，但修补后应该可以通过。 2015-04-07T00:00:00Z 工作原理相同并通过。

$ curl 'https://test-dspace.example.com/oai/request?verb=ListRecords&metadataPrefix=oai_dc&until=2015-04-07'
...<error code="badArgument">Invalid date given in until parameter</error></OAI-PMH>

$ curl 'https://test-dspace.example.com/oai/request?verb=ListRecords&metadataPrefix=oai_dc&until=2015-04-07T00:00:00Z'
... completeListSize="3731" cursor="0">oai_dc//2015-04-07T00:00:00.000Z//100</resumptionToken></ListRecords></OAI-PMH>

我已经看到并阅读了多次 DSpace - Tech email entitled 'Building DSpace after bug fix (DS-2542)' ，尽管我已经根据该电子邮件线索做了我应该做的一切(包括“正确的”——编辑你的 POM——和“快速而肮脏的” -- 替换其中列出的 jar -- 方法)，问题依然存在。

我尝试过的事情(都无济于事)

  • 确保 tomcat 正确停止然后启动。
  • 重复完整的构建过程(mvn -U clean package)
  • 确保 /usr/local/dspace/lib/ 中除了 xoai-3.2.10-SNAPSHOT.jar 之外没有 xoai-*.jar 和 /usr/local/dspace/webapps/oai/WEB-INF/lib/
  • 为 com.lyncode.xoai 和 org.dspace.xoai 启用调试日志记录
  • 替换了 jar - 认为 dspace 的构建过程(mvn 或 ant)可能正在用其他进程替换它。
  • 删除了 ~/.m2/repository 并从头开始重新构建(包括所有模块)。
  • 在 Tomcat 启动参数中添加了 -verbose:class 以查看加载内容的位置(请参阅附加附录)

我目前的想法/疑问

我确信该 jar 已按预期创建，并且安装良好，并且没有旧 jar 正在运行。

我不确信我认为应该运行的代码实际上正在运行，或者是否有其他东西掩盖了代码......并且我不知道如何诊断这一点 Tomcat 。

是否有一个命令(大概是 dspace dsrun ...)我可以用来在 Tomcat 之外复制此测试？

我应该指出，我是一名系统工程师，而不是 Java 开发人员。

附录

日志设置

添加到 dspace.log 的附加程序部分的末尾

log4j.logger.com.lyncode.xoai=DEBUG, A1
log4j.logger.org.dspace.xoai=DEBUG, A1

这让我可以看到如下日志:

2015-07-28 02:34:07,222 DEBUG org.dspace.xoai.services.impl.solr.DSpaceSolrServerResolver @ Solr Server Initialized

2015-07-28 02:34:07,233 DEBUG com.lyncode.xoai.dataprovider.OAIRequestParameters @ RootParameterMap 'until' = '2015-04-07'

如果我编辑/usr/local/src/xoai/src/main/java/com/lyncode/xoai/dataprovider/OAIDataProvider.java，我可以将其变成

2015-07-28 02:50:15,374 DEBUG com.lyncode.xoai.dataprovider.OAIRequestParameters @ CAMERON-RootParameterMap 'until' = '2015-04-07'

但是将 CAMERON 添加到看似唯一生成错误文本的位置的开头并不会显示测试输出的变化(这位于同一目录中的 OAIDataProvider.java 中):

throw new BadArgumentException("CAMERON Invalid date given in until parameter");

dspace-src 目录的 git diff(相对于预修补)

# git diff
diff --git a/build.properties b/build.properties
index 844cb93..09cb5c3 100644
--- a/build.properties
+++ b/build.properties
@@ -166,6 +166,7 @@ http.proxy.port = 3128
 loglevel.other = WARN
 # loglevel.other: Log level for other third-party tools/APIs used by DSpace
 # Possible values (from most to least info): DEBUG, INFO, WARN, ERROR, FATAL
-loglevel.dspace = INFO
+#loglevel.dspace = INFO
+loglevel.dspace = DEBUG
 # loglevel.dspace: Log level for all DSpace-specific code (org.dspace.*)
 # Possible values (from most to least info): DEBUG, INFO, WARN, ERROR, FATAL
diff --git a/dspace-oai/pom.xml b/dspace-oai/pom.xml
index 6e56a52..136a10f 100644
--- a/dspace-oai/pom.xml
+++ b/dspace-oai/pom.xml
@@ -16,7 +16,7 @@
         <!-- This is the path to the root [dspace-src] directory. -->
         <root.basedir>${basedir}/..</root.basedir>
         <spring.version>3.2.5.RELEASE</spring.version>
-        <xoai.version>3.2.9</xoai.version>
+        <xoai.version>3.2.10-SNAPSHOT</xoai.version>
         <jtwig.version>2.0.1</jtwig.version>
     </properties>

diff --git a/dspace-oai/src/main/java/org/dspace/xoai/filter/DateUntilFilter.java b/dspace-oai/src/main/java/org/dspace/xoai/filter/DateUntilFilter.java
index 1995fc0..cdb17d9 100644
--- a/dspace-oai/src/main/java/org/dspace/xoai/filter/DateUntilFilter.java
+++ b/dspace-oai/src/main/java/org/dspace/xoai/filter/DateUntilFilter.java
@@ -50,6 +50,11 @@ public class DateUntilFilter extends DSpaceFilter
     public SolrFilterResult buildSolrQuery()
     {
         String format = dateProvider.format(date).replace("Z", ".999Z"); // Tweak to set the millisecon
+        if (format.substring(11, 19).equals("00:00:00"))
+        {
+            format = format.substring(0, 11) + "23:59:59" + format.substring(19);
+        }
+
         return new SolrFilterResult("item.lastmodified:[* TO "
                 + ClientUtils.escapeQueryChars(format) + "]");
     }
diff --git a/dspace-oai/src/main/java/org/dspace/xoai/util/DateUtils.java b/dspace-oai/src/main/java/org/dspace/xoai/util/DateUtils.java
index e968414..b73955d 100644
--- a/dspace-oai/src/main/java/org/dspace/xoai/util/DateUtils.java
+++ b/dspace-oai/src/main/java/org/dspace/xoai/util/DateUtils.java
@@ -31,8 +31,7 @@ public class DateUtils
     }
     public static String format(Date date, boolean init)
     {
-       SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.'000Z'");
-       if (!init) sdf = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.'999Z'");
+        SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'");
         // We indicate that the returned date is in Zulu time (UTC) so we have
         // to set the time zone of sdf correct.
         sdf.setTimeZone(TimeZone.getTimeZone("ZULU"));

来自/usr/local/src/xoai 的 Git 日志(分支 3.x 的 github.com/lyncode/xoai checkout )

commit f6721be7ef5bf75d790e220ee81821e8eb70986e
Merge: dd9ef83 71ae14e
Author: João Melo <<a href="https://stackoverflow.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="fc9691999093bc9085929f939899d29f9391" rel="noreferrer noopener nofollow">[email protected]</a>>
Date:   Thu Oct 9 18:33:57 2014 +0100

    Merge pull request #34 from kosarko/dates_fix

    from and until granularity

commit dd9ef830a16209e624bd03c3e91e7c5d7bdbd449
Merge: 8e778c2 c04b0bc
Author: João Melo <<a href="https://stackoverflow.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="3b51565e57547b57425558545f5e15585456" rel="noreferrer noopener nofollow">[email protected]</a>>
Date:   Thu Oct 9 18:33:29 2014 +0100

    Merge pull request #33 from kosarko/filter_fix

    fixed typo

commit 71ae14ed5ee9e6a5a6e3d6253424c212ac98081a
Author: Ondřej Košarko <<a href="https://stackoverflow.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="e3888c908291888ca39685828fcd8e8585cd80968d8acd8099" rel="noreferrer noopener nofollow">[email protected]</a>>
Date:   Thu Oct 9 19:05:03 2014 +0200

    from and until granularity

    Regardles of the repository setting the from and until parameters must
    handle day granularity. Error message fixed

commit c04b0bcd177007f618f18008e53a61a81aabdb06
Author: Ondřej Košarko <<a href="https://stackoverflow.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="a2c9cdd1c3d0c9cde2d7c4c3ce8ccfc4c48cc1d7cccb8cc1d8" rel="noreferrer noopener nofollow">[email protected]</a>>
Date:   Thu Oct 9 19:01:08 2014 +0200

    fixed typo

xoai-*.jar 实例

$ sudo updatedb
$ locate xoai- | grep '\.jar$' | grep -v bak | xargs ls -l
-rwxr-xr-x 1 tomcat  tomcat   321775 Jul 28 03:04 /usr/local/dspace/lib/xoai-3.2.10-SNAPSHOT.jar
-rwxr-xr-x 1 tomcat  tomcat   321775 Jul 28 03:04 /usr/local/dspace/webapps/oai/WEB-INF/lib/xoai-3.2.10-SNAPSHOT.jar
-rw-rw-r-- 1 builder builder  321832 Jul 28 01:34 /usr/local/src/dspace-src/dspace/modules/oai/target/oai-5.1/WEB-INF/lib/xoai-3.2.10-SNAPSHOT.jar
-rw-rw-r-- 1 builder builder  321832 Jul 28 01:34 /usr/local/src/dspace-src/dspace-oai/target/dspace-oai-5.1/WEB-INF/lib/xoai-3.2.10-SNAPSHOT.jar
-rw-rw-r-- 1 builder builder  321832 Jul 28 01:34 /usr/local/src/dspace-src/dspace/target/dspace-installer/lib/xoai-3.2.10-SNAPSHOT.jar
-rw-rw-r-- 1 builder builder  321832 Jul 28 01:34 /usr/local/src/dspace-src/dspace/target/dspace-installer/webapps/oai/WEB-INF/lib/xoai-3.2.10-SNAPSHOT.jar
-rw-rw-r-- 1 builder builder  321775 Jul 28 03:04 /usr/local/src/xoai/target/xoai-3.2.10-SNAPSHOT.jar
-rw-rw-r-- 1 builder builder 1571092 Jul 28 03:04 /usr/local/src/xoai/target/xoai-3.2.10-SNAPSHOT-javadoc.jar
-rw-rw-r-- 1 builder builder  222305 Jul 28 03:04 /usr/local/src/xoai/target/xoai-3.2.10-SNAPSHOT-sources.jar

验证 tomcat 应该看到相同的内容(如果链接损坏，但似乎没有链接)

$ find -L /usr/share/tomcat6 -type f -name 'xoai-*.jar' -print | xargs ls -l
-rwxr-xr-x 1 tomcat tomcat 321775 Jul 28 03:04 /usr/share/tomcat6/webapps/oai/WEB-INF/lib/xoai-3.2.10-SNAPSHOT.jar

Tomcat -verbose:类结果

所有com.lyncode.xoai。类来自文件:/usr/local/dspace/webapps/oai/WEB-INF/lib/xoai-3.2.10-SNAPSHOT.jar

非常感谢您的阅读，

卡梅伦

最佳答案

您是否尝试过清除oai响应缓存？我不知道这种类型的东西是否会被缓存，但这是我能想到的唯一你还没有提到的东西。

[dspace]/bin/dspace oai clean-cache

https://wiki.duraspace.org/display/DSDOC5x/OAI+2.0+Server#OAI2.0Server-OAIManager%28SolrDataSource%29

关于java - 为 DS-2542(XOAI 日期粒度)修补 Dspace 不会导致行为发生变化，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/31667055/

上一篇：java - 使用 netbeans.deploy 时 Netbeans 取消部署

下一篇：java - 如何子类化 Guava 的 ImmutableList？

相关文章：

JavaFX 对话 - 游戏

java - 在 maven 本地仓库中看不到 sikuli-java lib

maven - 为 Maven Archetype 插件指定依赖版本

java - spring boot包不存在错误

java - 停止 Tomcat 以保存用户凭据

java - TomEE启动错误: Cannot create unique file,请将java.io.tmpdir设置为可写文件夹或创建工作文件夹

java - Eclipse 需要额外的 { }...为什么？

maven - 解决 Maven 无法检索插件描述符和构建失败

java - 使用插件扩展 Java Web 应用程序

security - 禁用沙盒 tomcat 9 ubuntu 20.04

©2024 IT工具网  联系我们