java - 使用缓冲区的 ReadLine() ？

Question

我使用 BufferedReader 逐行读取 CSV 文件的内容:

    for (int i=0; ((line=br.readLine())!=null) ; i++) {
        String[] split = line.split(",");
        ts[i]=Double.parseDouble(split[0]);
        amp[i]=Double.parseDouble(split[1]);
    }

这个循环运行大约 150k 次迭代，ts 和 amp 数组是预先分配的，每个数组占用的空间不超过 1mb，但我遇到了内存不足异常。一开始我认为是数组太大，但我开始认为是 readLine() 和 split() 通过分配许多 String 导致了这种情况对象并填充堆，这是运行此循环时的 logcat :

11-21 08:12:16.277: D/dalvikvm(2351): GC_FOR_MALLOC freed 8640 objects / 519088 bytes in 31ms
11-21 08:12:16.377: D/dalvikvm(2351): GC_FOR_MALLOC freed 13943 objects / 617264 bytes in 34ms
11-21 08:12:16.467: D/dalvikvm(2351): GC_FOR_MALLOC freed 13073 objects / 578176 bytes in 31ms
11-21 08:12:16.567: D/dalvikvm(2351): GC_FOR_MALLOC freed 13079 objects / 578168 bytes in 32ms
11-21 08:12:16.657: D/dalvikvm(2351): GC_FOR_MALLOC freed 13073 objects / 578144 bytes in 31ms
11-21 08:12:16.747: D/dalvikvm(2351): GC_FOR_MALLOC freed 13073 objects / 578128 bytes in 31ms
11-21 08:12:16.837: D/dalvikvm(2351): GC_FOR_MALLOC freed 13075 objects / 578104 bytes in 31ms
11-21 08:12:16.927: D/dalvikvm(2351): GC_FOR_MALLOC freed 13073 objects / 578176 bytes in 31ms
11-21 08:12:17.017: D/dalvikvm(2351): GC_FOR_MALLOC freed 13073 objects / 578152 bytes in 31ms
11-21 08:12:17.107: D/dalvikvm(2351): GC_FOR_MALLOC freed 13076 objects / 578176 bytes in 31ms
11-21 08:12:17.197: D/dalvikvm(2351): GC_FOR_MALLOC freed 13077 objects / 578176 bytes in 31ms
11-21 08:12:17.227: D/dalvikvm(2351): GC_FOR_MALLOC freed 1417 objects / 88168 bytes in 28ms
11-21 08:12:17.237: I/dalvikvm-heap(2351): Grow heap (frag case) to 4.463MB for 278976-byte allocation
11-21 08:12:17.277: D/dalvikvm(2351): GC_FOR_MALLOC freed 0 objects / 0 bytes in 41ms
11-21 08:12:17.317: D/dalvikvm(2351): GC_FOR_MALLOC freed 100 objects / 3912 bytes in 45ms
11-21 08:12:17.317: I/dalvikvm-heap(2351): Grow heap (frag case) to 4.726MB for 278976-byte allocation
11-21 08:12:17.377: D/dalvikvm(2351): GC_FOR_MALLOC freed 0 objects / 0 bytes in 53ms
11-21 08:12:17.627: D/dalvikvm(2351): GC_FOR_MALLOC freed 16432 objects / 572048 bytes in 40ms
11-21 08:12:17.907: D/dalvikvm(2351): GC_FOR_MALLOC freed 21312 objects / 779968 bytes in 48ms
11-21 08:12:18.187: D/dalvikvm(2351): GC_FOR_MALLOC freed 21582 objects / 791496 bytes in 45ms
11-21 08:12:18.527: D/dalvikvm(2351): GC_FOR_MALLOC freed 21918 objects / 805336 bytes in 52ms
11-21 08:12:18.767: D/dalvikvm(2351): GC_FOR_MALLOC freed 22258 objects / 818384 bytes in 50ms
11-21 08:12:19.017: D/dalvikvm(2351): GC_FOR_MALLOC freed 22550 objects / 830456 bytes in 52ms
11-21 08:12:19.277: D/dalvikvm(2351): GC_FOR_MALLOC freed 24160 objects / 887640 bytes in 50ms
11-21 08:12:19.547: D/dalvikvm(2351): GC_FOR_MALLOC freed 24386 objects / 900944 bytes in 53ms
11-21 08:12:19.827: D/dalvikvm(2351): GC_FOR_MALLOC freed 24575 objects / 908088 bytes in 53ms
11-21 08:12:20.137: D/dalvikvm(2351): GC_FOR_MALLOC freed 24785 objects / 917320 bytes in 58ms
11-21 08:12:20.597: D/dalvikvm(2351): GC_FOR_MALLOC freed 24958 objects / 924104 bytes in 59ms
11-21 08:12:20.937: D/dalvikvm(2351): GC_FOR_MALLOC freed 25145 objects / 932488 bytes in 62ms
11-21 08:12:21.227: D/dalvikvm(2351): GC_FOR_MALLOC freed 25336 objects / 940096 bytes in 64ms
11-21 08:12:21.837: D/dalvikvm(2351): GC_FOR_MALLOC freed 26830 objects / 993552 bytes in 256ms
11-21 08:12:22.477: D/dalvikvm(2351): GC_FOR_MALLOC freed 26934 objects / 1001584 bytes in 65ms
11-21 08:12:22.797: D/dalvikvm(2351): GC_FOR_MALLOC freed 27056 objects / 1005400 bytes in 65ms
11-21 08:12:23.127: D/dalvikvm(2351): GC_FOR_MALLOC freed 27159 objects / 1009248 bytes in 75ms
11-21 08:12:23.467: D/dalvikvm(2351): GC_FOR_MALLOC freed 28594 objects / 1058760 bytes in 75ms
11-21 08:12:23.817: D/dalvikvm(2351): GC_FOR_MALLOC freed 28643 objects / 1065424 bytes in 70ms
11-21 08:12:24.157: D/dalvikvm(2351): GC_FOR_MALLOC freed 28650 objects / 1063104 bytes in 71ms
11-21 08:12:24.507: D/dalvikvm(2351): GC_FOR_MALLOC freed 30001 objects / 1110840 bytes in 76ms
11-21 08:12:24.837: D/dalvikvm(2351): GC_FOR_MALLOC freed 29989 objects / 1114480 bytes in 76ms
11-21 08:12:25.197: D/dalvikvm(2351): GC_FOR_MALLOC freed 29928 objects / 1112088 bytes in 82ms
11-21 08:12:25.547: D/dalvikvm(2351): GC_FOR_MALLOC freed 31223 objects / 1157600 bytes in 80ms
11-21 08:12:25.918: D/dalvikvm(2351): GC_FOR_MALLOC freed 31156 objects / 1158728 bytes in 75ms
11-21 08:12:26.297: D/dalvikvm(2351): GC_FOR_MALLOC freed 32364 objects / 1200176 bytes in 80ms
11-21 08:12:26.667: D/dalvikvm(2351): GC_FOR_MALLOC freed 32245 objects / 1199544 bytes in 78ms
11-21 08:12:27.047: D/dalvikvm(2351): GC_FOR_MALLOC freed 33392 objects / 1237880 bytes in 73ms
11-21 08:12:27.457: D/dalvikvm(2351): GC_FOR_MALLOC freed 33223 objects / 1236256 bytes in 83ms
11-21 08:12:27.857: D/dalvikvm(2351): GC_FOR_MALLOC freed 34326 objects / 1274488 bytes in 90ms
11-21 08:12:28.267: D/dalvikvm(2351): GC_FOR_MALLOC freed 35443 objects / 1316328 bytes in 87ms
11-21 08:12:28.667: D/dalvikvm(2351): GC_FOR_MALLOC freed 35154 objects / 1308232 bytes in 90ms
11-21 08:12:29.167: D/dalvikvm(2351): GC_FOR_MALLOC freed 36188 objects / 1343032 bytes in 88ms
11-21 08:12:29.597: D/dalvikvm(2351): GC_FOR_MALLOC freed 37185 objects / 1382272 bytes in 90ms
11-21 08:12:30.027: D/dalvikvm(2351): GC_FOR_MALLOC freed 36818 objects / 1369776 bytes in 96ms
11-21 08:12:30.507: D/dalvikvm(2351): GC_FOR_MALLOC freed 37768 objects / 1403688 bytes in 108ms
11-21 08:12:30.967: D/dalvikvm(2351): GC_FOR_MALLOC freed 38717 objects / 1439096 bytes in 98ms
11-21 08:12:31.437: D/dalvikvm(2351): GC_FOR_MALLOC freed 39601 objects / 1471320 bytes in 102ms
11-21 08:12:31.897: D/dalvikvm(2351): GC_FOR_MALLOC freed 40432 objects / 1502408 bytes in 103ms
11-21 08:12:32.377: D/dalvikvm(2351): GC_FOR_MALLOC freed 41252 objects / 1532248 bytes in 102ms
11-21 08:12:32.857: D/dalvikvm(2351): GC_FOR_MALLOC freed 40715 objects / 1516256 bytes in 107ms
11-21 08:12:33.337: D/dalvikvm(2351): GC_FOR_MALLOC freed 41442 objects / 1540168 bytes in 109ms
11-21 08:12:33.837: D/dalvikvm(2351): GC_FOR_MALLOC freed 42223 objects / 1568280 bytes in 114ms
11-21 08:12:34.357: D/dalvikvm(2351): GC_FOR_MALLOC freed 42918 objects / 1595072 bytes in 109ms
11-21 08:12:34.857: D/dalvikvm(2351): GC_FOR_MALLOC freed 43604 objects / 1621272 bytes in 111ms
11-21 08:12:35.387: D/dalvikvm(2351): GC_FOR_MALLOC freed 44236 objects / 1643136 bytes in 119ms
11-21 08:12:35.917: D/dalvikvm(2351): GC_FOR_MALLOC freed 44833 objects / 1667152 bytes in 125ms
11-21 08:12:36.457: D/dalvikvm(2351): GC_FOR_MALLOC freed 45441 objects / 1689152 bytes in 123ms
11-21 08:12:36.997: D/dalvikvm(2351): GC_FOR_MALLOC freed 46002 objects / 1710696 bytes in 126ms
11-21 08:12:37.327: D/dalvikvm(2351): GC_FOR_MALLOC freed 20464 objects / 838800 bytes in 109ms
11-21 08:12:37.327: I/dalvikvm-heap(2351): Grow heap (frag case) to 8.357MB for 278752-byte allocation
11-21 08:12:37.417: D/dalvikvm(2351): GC_FOR_MALLOC freed 624 objects / 28192 bytes in 88ms
11-21 08:12:37.417: I/dalvikvm-heap(2351): Grow heap (frag case) to 8.595MB for 278752-byte allocation
11-21 08:12:37.507: D/dalvikvm(2351): GC_FOR_MALLOC freed 605 objects / 26136 bytes in 94ms
11-21 08:12:37.627: D/dalvikvm(2351): GC_EXTERNAL_ALLOC freed 1771 objects / 79408 bytes in 95ms
11-21 08:12:37.677: I/global(2351): Default buffer size used in BufferedReader constructor. It would be better to be explicit if an 8k-char buffer is required.
11-21 08:12:38.147: D/dalvikvm(2351): GC_FOR_MALLOC freed 47316 objects / 1682624 bytes in 125ms
11-21 08:12:38.607: D/dalvikvm(2351): GC_FOR_MALLOC freed 50691 objects / 1886456 bytes in 125ms
11-21 08:12:39.077: D/dalvikvm(2351): GC_FOR_MALLOC freed 50346 objects / 1867368 bytes in 125ms
11-21 08:12:39.557: D/dalvikvm(2351): GC_FOR_MALLOC freed 50983 objects / 1860976 bytes in 134ms
11-21 08:12:40.007: D/dalvikvm(2351): GC_FOR_MALLOC freed 49459 objects / 1851056 bytes in 131ms
11-21 08:12:40.517: D/dalvikvm(2351): GC_FOR_MALLOC freed 50486 objects / 1842352 bytes in 135ms
11-21 08:12:40.797: D/dalvikvm(2351): GC_FOR_MALLOC freed 24273 objects / 970136 bytes in 121ms
11-21 08:12:40.797: I/dalvikvm-heap(2351): Grow heap (frag case) to 10.441MB for 46508-byte allocation
11-21 08:12:40.897: D/dalvikvm(2351): GC_FOR_MALLOC freed 1206 objects / 89576 bytes in 98ms
11-21 08:12:41.367: D/dalvikvm(2351): GC_FOR_MALLOC freed 50141 objects / 1769760 bytes in 128ms
11-21 08:12:41.847: D/dalvikvm(2351): GC_FOR_MALLOC freed 50484 objects / 1853208 bytes in 133ms
11-21 08:12:42.317: D/dalvikvm(2351): GC_FOR_MALLOC freed 50492 objects / 1842496 bytes in 135ms
11-21 08:12:42.517: D/dalvikvm(2351): GC_FOR_MALLOC freed 12082 objects / 502744 bytes in 110ms
11-21 08:12:42.517: I/dalvikvm-heap(2351): Grow heap (frag case) to 10.589MB for 69752-byte allocation
11-21 08:12:42.627: D/dalvikvm(2351): GC_FOR_MALLOC freed 410 objects / 66160 bytes in 106ms
11-21 08:12:42.727: D/dalvikvm(2351): GC_FOR_MALLOC freed 408 objects / 19520 bytes in 102ms
11-21 08:12:42.727: I/dalvikvm-heap(2351): Grow heap (frag case) to 10.574MB for 69752-byte allocation
11-21 08:12:42.847: D/dalvikvm(2351): GC_FOR_MALLOC freed 1 objects / 46512 bytes in 117ms
11-21 08:12:43.317: D/dalvikvm(2351): GC_FOR_MALLOC freed 48763 objects / 1708184 bytes in 136ms
11-21 08:12:43.807: D/dalvikvm(2351): GC_FOR_MALLOC freed 52176 objects / 1909352 bytes in 139ms
11-21 08:12:44.287: D/dalvikvm(2351): GC_FOR_MALLOC freed 50568 objects / 1848768 bytes in 140ms
11-21 08:12:44.747: D/dalvikvm(2351): GC_FOR_MALLOC freed 49024 objects / 1793368 bytes in 134ms
11-21 08:12:45.157: D/dalvikvm(2351): GC_FOR_MALLOC freed 39117 objects / 1451904 bytes in 135ms
11-21 08:12:45.157: I/dalvikvm-heap(2351): Grow heap (frag case) to 11.120MB for 104620-byte allocation
11-21 08:12:45.267: D/dalvikvm(2351): GC_FOR_MALLOC freed 395 objects / 88912 bytes in 107ms
11-21 08:12:45.367: D/dalvikvm(2351): GC_FOR_MALLOC freed 1380 objects / 67848 bytes in 103ms
11-21 08:12:45.367: I/dalvikvm-heap(2351): Grow heap (frag case) to 11.070MB for 104620-byte allocation
11-21 08:12:45.467: D/dalvikvm(2351): GC_FOR_MALLOC freed 596 objects / 98320 bytes in 100ms
11-21 08:12:45.957: D/dalvikvm(2351): GC_FOR_MALLOC freed 51102 objects / 1812792 bytes in 140ms
11-21 08:12:46.457: D/dalvikvm(2351): GC_FOR_MALLOC freed 50849 objects / 1862544 bytes in 148ms
11-21 08:12:46.927: D/dalvikvm(2351): GC_FOR_MALLOC freed 49318 objects / 1802448 bytes in 140ms
11-21 08:12:47.407: D/dalvikvm(2351): GC_FOR_MALLOC freed 49392 objects / 1802936 bytes in 143ms
11-21 08:12:47.887: D/dalvikvm(2351): GC_FOR_MALLOC freed 49495 objects / 1805800 bytes in 150ms
11-21 08:12:48.377: D/dalvikvm(2351): GC_FOR_MALLOC freed 49539 objects / 1806848 bytes in 148ms
11-21 08:12:48.857: D/dalvikvm(2351): GC_FOR_MALLOC freed 49652 objects / 1810752 bytes in 142ms
11-21 08:12:49.117: D/dalvikvm(2351): GC_FOR_MALLOC freed 17452 objects / 696544 bytes in 132ms
11-21 08:12:49.117: I/dalvikvm-heap(2351): Grow heap (frag case) to 11.520MB for 156920-byte allocation
11-21 08:12:49.237: D/dalvikvm(2351): GC_FOR_MALLOC freed 510 objects / 129792 bytes in 122ms
11-21 08:12:49.357: D/dalvikvm(2351): GC_FOR_MALLOC freed 786 objects / 37712 bytes in 121ms
11-21 08:12:49.357: I/dalvikvm-heap(2351): Grow heap (frag case) to 11.510MB for 156920-byte allocation
11-21 08:12:49.487: D/dalvikvm(2351): GC_FOR_MALLOC freed 1 objects / 104624 bytes in 131ms
11-21 08:12:49.987: D/dalvikvm(2351): GC_FOR_MALLOC freed 49227 objects / 1721856 bytes in 165ms
11-21 08:12:50.547: D/dalvikvm(2351): GC_FOR_MALLOC freed 53182 objects / 1943240 bytes in 155ms
11-21 08:12:51.067: D/dalvikvm(2351): GC_FOR_MALLOC freed 51447 objects / 1875984 bytes in 155ms
11-21 08:12:51.607: D/dalvikvm(2351): GC_FOR_MALLOC freed 49938 objects / 1820528 bytes in 157ms
11-21 08:12:52.117: D/dalvikvm(2351): GC_FOR_MALLOC freed 48500 objects / 1768472 bytes in 156ms
11-21 08:12:52.637: D/dalvikvm(2351): GC_FOR_MALLOC freed 48629 objects / 1771176 bytes in 164ms
11-21 08:12:53.157: D/dalvikvm(2351): GC_FOR_MALLOC freed 48672 objects / 1773880 bytes in 163ms
11-21 08:12:53.678: D/dalvikvm(2351): GC_FOR_MALLOC freed 48755 objects / 1777616 bytes in 164ms
11-21 08:12:54.197: D/dalvikvm(2351): GC_FOR_MALLOC freed 48814 objects / 1780776 bytes in 165ms
11-21 08:12:54.718: D/dalvikvm(2351): GC_FOR_MALLOC freed 48915 objects / 1783368 bytes in 164ms
11-21 08:12:55.237: D/dalvikvm(2351): GC_FOR_MALLOC freed 47448 objects / 1733344 bytes in 168ms

所以我想我必须找到 readLine() 和 split() 的替代方法来读取以下格式的 csv 文件:

double , double \n

我猜它必须是始终使用相同缓冲区的缓冲行读取 对于已经实现的具有这种能力的类有什么建议吗？

Answer 1

你可以尝试(Java 7):

List<String> lines = Files.readAllLines(Paths.get(filepath), StandardCharsets.UTF_8);

或其他方式:com.googlecode.jcsv.reader.CSVReader。

类似于:

CSVReader<CSVFromat> sessionReader = new CSVReaderBuilder<CSVFromat>(csvFile).entryParser(new CSVEntryParser()).build();

List<CSVFromat> session = sessionReader.readAll();

它将文件转换为列表