Name: Cudder !cXCudderUE 2015-07-22 4:58
I had to convert from one multipage vector document format to another, and found an opensource application that claimed to be able to do it. It was written in Java, which is a bad sign already, but I just wanted something for this conversion - I only needed to convert a few files. The first file I tried it on, which was ~20MB and several hundred pages, caused it to run out of memory. Smaller ones (<1MB, around 10 pages) were OK.
I narrowed down the problem to it trying to keep the entire generated output file in memory and allocating a bitmap to render each page.
Both the input and output document formats are designed precisely so you don't have to keep the whole file in memory, and they're primarily vector formats so there should be no bitmap rendering either. (Each page in the first document I tried is roughly 20-30KB in vector format, including small embedded bitmaps.)
By limiting it to a small range of pages and pushing the JVM memory limits to their max I got it to generate an output file. 6 seconds for 10 pages of medium complexity, which take much less than a second for the original document to render in the native viewer. That's not a good sign either, but what's worse is when I opened the output file, all the pages were blank! Each one had nothing more than a single embedded bitmap with all white pixels.
WTF? This is pathetic. Do Java programmers these days not even think carefully about the algorithms they're using anymore? These are very close formats, basically all that's necessary is to open the input file, parse headers, write translated version to the output, then in the main loop read a vector command from input (several tens of bytes), translate coordinates and units, and write the equivalent command to the output. Embedded bitmaps are supported in both formats and the raw data is essentially identical, so just stream it through. It's definitely doable in <1MB of constant memory, and yet the idiot(s) who wrote this shit thought they needed to read the entire fucking file into memory, parse and create millions of objects, render them into bitmaps (again, all allocated in memory, one for every page) and then create an output file, again in memory, consisting entirely of embedded bitmaps!? Even with enough memory it was broken.
I'm not going to fix it because the source is typical ENTERPRISE QUALITY JAVA with several hundred classes each of which does very little. Writing my own converter would probably take much less time.
I narrowed down the problem to it trying to keep the entire generated output file in memory and allocating a bitmap to render each page.
Both the input and output document formats are designed precisely so you don't have to keep the whole file in memory, and they're primarily vector formats so there should be no bitmap rendering either. (Each page in the first document I tried is roughly 20-30KB in vector format, including small embedded bitmaps.)
By limiting it to a small range of pages and pushing the JVM memory limits to their max I got it to generate an output file. 6 seconds for 10 pages of medium complexity, which take much less than a second for the original document to render in the native viewer. That's not a good sign either, but what's worse is when I opened the output file, all the pages were blank! Each one had nothing more than a single embedded bitmap with all white pixels.
WTF? This is pathetic. Do Java programmers these days not even think carefully about the algorithms they're using anymore? These are very close formats, basically all that's necessary is to open the input file, parse headers, write translated version to the output, then in the main loop read a vector command from input (several tens of bytes), translate coordinates and units, and write the equivalent command to the output. Embedded bitmaps are supported in both formats and the raw data is essentially identical, so just stream it through. It's definitely doable in <1MB of constant memory, and yet the idiot(s) who wrote this shit thought they needed to read the entire fucking file into memory, parse and create millions of objects, render them into bitmaps (again, all allocated in memory, one for every page) and then create an output file, again in memory, consisting entirely of embedded bitmaps!? Even with enough memory it was broken.
I'm not going to fix it because the source is typical ENTERPRISE QUALITY JAVA with several hundred classes each of which does very little. Writing my own converter would probably take much less time.