Comments on Code Instructions: A Memory Problem With Java IO

I'm interested in why the JVM copies the byte ...

2009-08-28T17:03:06.604-04:00

I'm interested in why the JVM copies the byte array, instead of using the original one.
since the IO call doesn't return till all of the data was written to the IO pipe, what the copy is needed for? Another thread might tamper with the byte array? that's the responsibility of the user to avoid from such acts. Any idea?

This is really useful, thanks :-)

2009-02-02T08:21:00.000-05:00

This is really useful, thanks :-)

William: about the majority of the OOM errors bein...

2009-01-31T20:07:00.000-05:00

William: about the majority of the OOM errors being caused by memory capacity issues: you are totally right :)

William: the problem was exactly that we weren't writing the array in chunks. Since we had the whole array at hand, it was simpler to just call out.write(array) only once, which caused the memory error. The solution was to do exactly as you suggested.

By the way I still do not understand what exactly ...

2009-01-30T08:28:00.000-05:00

By the way I still do not understand what exactly the cause is. Even if the byte array is relatively large (5M) , which is a problem (waste) in itself, every IO class allows one to specify a range in the writing.

Kind regards,

William

"The process of fixing it usually involves profili...

2009-01-30T08:24:00.000-05:00

"The process of fixing it usually involves profiling the memory in search for memory leaks, which can be very time consuming."

Not all out memory errors are related to memory leaks. In fact a significantly large portion are just memory capacity issues due to a relative high number of concurrent memory hungry (alloc) requests.

William

Pablo: we were actually using it, but the Buffered...

2009-01-29T19:43:00.000-05:00

Pablo: we were actually using it, but the BufferedOutputStream bypasses itself if the data being written is larger than its internal buffer, which kind of makes sense :) If its purpose is to save the number of writes to the underlying stream, why should it make more calls when it receives a large input? :)

vrg: it could work in a general case but probably ...

2009-01-29T19:02:00.000-05:00

vrg: it could work in a general case but probably in my situation it would also cause problems because the process address space was almost exhausted, but it is something worth considering!

Meant 8KB, sorry

2009-01-29T13:59:00.000-05:00

Meant 8KB, sorry

Haven't you tried using a BufferedOutputStream? Th...

2009-01-29T13:58:00.000-05:00

Haven't you tried using a BufferedOutputStream? That way you can write as much bytes as you want, but only be written to the file at 9KB chunks

I think there is another solution: the nio MappedB...

2009-01-29T03:24:00.000-05:00

I think there is another solution: the nio MappedByteBuffer. What do you think?