java - 上传到 Blobstore 会出现 Java 堆 OutOfMemoryError

Question

我正在为我的 Google App Engine 应用程序做一个非常简单的上传表单。在客户端 GWT 代码中，我有类似的内容:

final FormPanel uploadForm = new FormPanel();
uploadForm.setEncoding(FormPanel.ENCODING_MULTIPART);
uploadForm.setMethod(FormPanel.METHOD_POST);

uploadBtn.addClickHandler(new ClickHandler() {
        @Override
        public void onClick(ClickEvent event) {
            blobstoreUploadURLService.getBlobstoreUploadURL("/banzai/process-pdf", new AsyncCallback<String>() {

                @Override
                public void onFailure(Throwable caught) {
                    // TODO Auto-generated method stub
                    System.err.println("FAILURE DURING UPLOAD SERVICE");
                }

                @Override
                public void onSuccess(String result) {
                    uploadForm.setAction(result);
                    uploadForm.submit();
                }

            });
        }
    });

它使用 new FileUpload() 来选择文件。当我在本地或部署的实例上测试它时，我在日志中收到以下错误:

WARNING: Error for /_ah/upload/agdrYnNrYWFychsLEhVfX0Jsb2JVcGxvYWRTZXNzaW9uX18YAww java.lang.OutOfMemoryError: Java heap space
  at java.util.Arrays.copyOf(Arrays.java:2786)
  at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:71)
  at javax.mail.internet.MimeMultipart.readTillFirstBoundary(MimeMultipart.java:316)
  at javax.mail.internet.MimeMultipart.parse(MimeMultipart.java:186)
  at javax.mail.internet.MimeMultipart.getCount(MimeMultipart.java:109)
  at com.google.appengine.api.blobstore.dev.UploadBlobServlet.handleUpload(UploadBlobServlet.java:135)
  at com.google.appengine.api.blobstore.dev.UploadBlobServlet.access$000(UploadBlobServlet.java:72)
  at com.google.appengine.api.blobstore.dev.UploadBlobServlet$1.run(UploadBlobServlet.java:100)
  at java.security.AccessController.doPrivileged(Native Method)
  at com.google.appengine.api.blobstore.dev.UploadBlobServlet.doPost(UploadBlobServlet.java:98)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:713)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:806)
  at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
  at com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter(ServeBlobFilter.java:51)
  at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
  at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
  at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
  at com.google.appengine.tools.development.StaticFileFilter.doFilter(StaticFileFilter.java:122)
  at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
  at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
  at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
  at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
  at com.google.apphosting.utils.jetty.DevAppEngineWebAppContext.handle(DevAppEngineWebAppContext.java:70)
  at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  at com.google.appengine.tools.development.JettyContainerService$ApiProxyHandler.handle(JettyContainerService.java:349)
  at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
  at org.mortbay.jetty.Server.handle(Server.java:326)
  at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
  at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)

有趣的是，以前似乎有几个人遇到过这个问题(只需查看 this issue here，您还可以在 Google 上找到更多)但似乎没有人知道为什么会这样，有些人甚至暗示这是 App Engine 的错。在我仓促下结论之前，我想我应该先在这里问一下 :)

如果它很有趣，这里是尝试上传的 HTTP header (一个非常小的文件)，使用 HTTP Live Headers 捕获。

    POST /_ah/upload/AMmfu6ZyyhSgz9uOR5VX4QBZeYADTB-aSejVvfGaogl3E_E8yPOLgtX9-0mob17IYfsaRZg-YP7aZrp1D4pDAwuKKm9CoNjeVx1eN2PwBro9x0PqXPeBLpQ/ALBNUaYAAAAATDFOaLPIvuEEhSS6F4HxMmf9xOb8lp0y/ HTTP/1.1
    Host: kbskaar.appspot.com
    User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.6) Gecko/20100625 Firefox/3.6.6
    Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
    Accept-Language: en-us,en;q=0.5
    Accept-Encoding: gzip,deflate
    Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
    Keep-Alive: 115
    Connection: keep-alive
    Referer: http://kbskaar.appspot.com/

    Cookie: Lastname=Wong; Firstname=Ka%20Man%20Sophia; Username=kmswong%40uwaterloo.ca
    Content-Type: multipart/form-data; boundary=---------------------------168072824752491622650073
    Content-Length: 57
        -----------------------------168072824752491622650073--



    HTTP/1.1 500 Internal Server Error
    Server: Upload Server Built on Jul 1 2010 15:26:59 (1278023219)
    Content-Type: text/html; charset=UTF-8
    X-AppEngine-Estimated-CPM-US-Dollars: $0.375815
    X-AppEngine-Resource-Usage: ms=7103 cpu_ms=16217 api_cpu_ms=0
    Date: Mon, 05 Jul 2010 03:06:00 GMT
    Pragma: no-cache
    Expires: Fri, 01 Jan 1990 00:00:00 GMT
    Cache-Control: no-cache, no-store, must-revalidate
    Content-Length: 3211
        ----------------------------------------------------------

如果您有任何想法，请告诉我。谢谢!

Answer 1

好吧，这个非常愚蠢的问题原来是我在表单的 FileInput 元素上遗漏了一个“name”属性。这显然会使生成的流无法解析，从而导致 MimeMultipart 解析器内存不足。

尽管问题最终是我的，而且很容易修复，但我仍然认为这是 AppEngine 中的一个错误，因为这样一个简单的错误不可能导致 OutOfMemoryError 和崩溃；这也可能只是通过制作恶意 HTTP 请求而成为 DOS 漏洞的来源。我会向 Google 提交错误报告。