我仍然是一个相对较新的程序员,我在

Java中遇到的一个问题是Out of Memory Errors.我不想使用-Xmx来增加内存,因为我觉得错误是由于编程不好造成的,我想改进编码而不是依靠更多的内存.

我所做的工作涉及处理大量文本文件,每个文本文件在压缩时大约为1GB.我在这里的代码是为了遍历一个新的压缩文本文件被删除的目录.它打开了第二个最新的文本文件(不是最新的,因为它仍然被写入),并使用Jsoup库来解析文本文件中的某些字段(字段用自定义分隔符分隔：“| nTa |”表示一个新列,“| nLa |”表示一个新行.

我觉得应该没有理由使用大量内存.我打开一个文件,扫描它,解析相关位,将解析后的版本写入另一个文件,关闭文件,然后移动到下一个文件.我不需要将整个文件存储在内存中,我当然不需要存储已经在内存中处理过的文件.

当我开始解析第二个文件时,我遇到错误,这表明我没有处理垃圾收集.请查看代码,看看你是否能发现我正在做的事情,这意味着我使用的内存比我应该的多.我想学习如何做到这一点,所以我不会出现内存错误！

import java.io.File;

import java.io.FileInputStream;

import java.io.FileOutputStream;

import java.util.ArrayList;

import java.util.Collections;

import java.util.Scanner;

import java.util.TreeMap;

import java.util.zip.GZIPInputStream;

import java.util.zip.GZIPOutputStream;

import org.jsoup.Jsoup;

public class ParseHTML {

public static int commentExtractField = 3;

public static int contentExtractField = 4;

public static int descriptionField = 5;

public static void main(String[] args) throws Exception {

File directoryCompleted = null;

File filesCompleted[] = null;

while(true) {

// find second most recent file in completed directory

directoryCompleted = new File(args[0]);

filesCompleted = directoryCompleted.listFiles();

if (filesCompleted.length > 1) {

TreeMap timeStamps = new TreeMap(Collections.reverseOrder());

for (File f : filesCompleted) {

timeStamps.put(getTimestamp(f), f);

}

File fileToProcess = null;

int counter = 0;

for (Long l : timeStamps.keySet()) {

fileToProcess = timeStamps.get(l);

if (counter == 1) {

break;

}

counter++;

}

// start processing file

GZIPInputStream gzipInputStream = null;

if (fileToProcess != null) {

gzipInputStream = new GZIPInputStream(new FileInputStream(fileToProcess));

}

else {

System.err.println("No file to process!");

System.exit(1);

}

Scanner scanner = new Scanner(gzipInputStream);

scanner.useDelimiter("\\|nLa\\|");

GZIPOutputStream output = new GZIPOutputStream(new FileOutputStream("parsed/" + fileToProcess.getName()));

while (scanner.hasNext()) {

Scanner scanner2 = new Scanner(scanner.next());

scanner2.useDelimiter("\\|nTa\\|");

ArrayList row = new ArrayList();

while(scanner2.hasNext()) {

row.add(scanner2.next());

}

for (int index = 0; index < row.size(); index++) {

if (index == commentExtractField ||

index == contentExtractField ||

index == descriptionField) {

output.write(jsoupParse(row.get(index)).getBytes("UTF-8"));

}

else {

output.write(row.get(index).getBytes("UTF-8"));

}

String delimiter = "";

if (index == row.size() - 1) {

delimiter = "|nLa|";

}

else {

delimiter = "|nTa|";

}

output.write(delimiter.getBytes("UTF-8"));

}

}

output.finish();

output.close();

scanner.close();

gzipInputStream.close();

}

}

}

public static Long getTimestamp(File f) {

String name = f.getName();

String removeExt = name.substring(0, name.length() - 3);

String timestamp = removeExt.substring(7, removeExt.length());

return Long.parseLong(timestamp);

}

public static String jsoupParse(String s) {

if (s.length() == 4) {

return s;

}

else {

return Jsoup.parse(s).text();

}

}

}

我怎样才能确保当我完成对象时,它们被销毁并且不使用任何资源？例如,每次我关闭GZIPInputStream,GZIPOutputStream和Scanner时,我怎样才能确保它们被彻底销毁？

为了记录,我得到的错误是：

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

at java.util.Arrays.copyOf(Arrays.java:2882)

at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)

at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:572)

at java.lang.StringBuilder.append(StringBuilder.java:203)

at org.jsoup.parser.TokeniserState$47.read(TokeniserState.java:1171)

at org.jsoup.parser.Tokeniser.read(Tokeniser.java:42)

at org.jsoup.parser.TreeBuilder.runParser(TreeBuilder.java:101)

at org.jsoup.parser.TreeBuilder.parse(TreeBuilder.java:53)

at org.jsoup.parser.Parser.parse(Parser.java:24)

at org.jsoup.Jsoup.parse(Jsoup.java:44)

at ParseHTML.jsoupParse(ParseHTML.java:125)

at ParseHTML.main(ParseHTML.java:81)