Using "divide and conquer" for large image processing

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
Post Reply
anyoneis
Posts: 19
Joined: 2011-08-08T12:29:34-07:00
Authentication code: 8675308
Contact:

Using "divide and conquer" for large image processing

Post by anyoneis »

I have some large tiff files that I am trying to process and I'm looking for performance improvements. Each file has multiple pages, and I am interested in extracting a single page, then breaking it up into 16 sub-images for processing outside of IM. The steps I am currently following are:

1) convert theFile.tif[2] image.mpc

Assuming dimensions of the image are 4W and 4H:
2) convert image.mpc -crop WxH+0+0 +repage -scene 0 -define PNG:color-type-2 theFile_00.png
...
convert image.mpc -crop WxH+3W+3H +repage -scene 0 -define PNG:color-type-2 theFile_15.png

Currently, I am executing two "step 2" processes concurrently, trying to eak out a little more performance.

Does anyone have experience with rearranging things to improve performance? For example, I could try a divide and conquer approach, as in:

1) convert theFile.tiff[2] -crop 4Wx2H +repage imageH%d.mpc // split the image into two

2) convert imageH0.mpc -crop 2Wx2H +repage imageH0Q%d.mpc // split the half into two quarters
convert imageH1.mpc -crop 2Wx2H +repage imageH1Q%d.mpc // split the half into two quarters

3) convert imageH0Q0.mpc -crop 2WxH +repage imageH0Q0O%d.mpc // split the quarter into 2 eights
convert imageH0Q1.mpc -crop 2WxH +repage imageH0Q1O%d.mpc // split the quarter into 2 eights
convert imageH1Q0.mpc -crop 2WxH +repage imageH1Q0O%d.mpc // split the quarter into 2 eights
convert imageH1Q1.mpc -crop 2WxH +repage imageH1Q1O%d.mpc // split the quarter into 2 eights

4) and finally, split each eighth into 16ths, assuming that is the ultimate tile size needed.

Since TANSTAAFL, and there are a lot more operations with this approach over a simple image to 16 tile crop operation, I suspect that I will not see any improvement in wall-clock times unless and until I get to the point where I have an operation at the leaf level that "fits in memory," at which point the improvement could be considerable.

Am I thinking about this problem correctly? Anyone try something like this? any other ideas?

Thanks,
David
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA
Contact:

Re: Using "divide and conquer" for large image processing

Post by fmw42 »

see http://www.imagemagick.org/Usage/crop/#crop_equal


convert image.tif[0] -crop "4x4@" +repage -define PNG:color-type=2 image_p0_%02d.png

This will give you 16 approx equal sized crop sections.

Note, it is color-type=2 not color-type-2

By default the numbering starts at 0 (or in this case 00) so -scene is not needed
anyoneis
Posts: 19
Joined: 2011-08-08T12:29:34-07:00
Authentication code: 8675308
Contact:

Re: Using "divide and conquer" for large image processing

Post by anyoneis »

That's a handy beast! I missed that.
convert image.tif[0] -crop "4x4@" +repage -define PNG:color-type=2 image_p0_%02d.png
"%02d" gives me a neater "ls" or "dir"!
By default the numbering starts at 0 (or in this case 00) so -scene is not needed
Just a tiny little "gotcha" there - by default, it starts at the scene you have extracted, which in the case of your example above, is 0, but in my case of extracting scene 2, I need the "scene 0" to reset the value.

Thanks for the hints and comments!
David
anyoneis
Posts: 19
Joined: 2011-08-08T12:29:34-07:00
Authentication code: 8675308
Contact:

Re: Using "divide and conquer" for large image processing

Post by anyoneis »

This is a naive "pipe dream." As we used to say in the microcomputer industry about "negative incremental return" schemes, "We lose a little bit on each one but we make it up in volume." ;-)

I will be better off just using the "-crop XxY@" syntax pointed out by Fred to produce the output tiles directly from the input file without any intermediate files.

Thanks again!
David
Post Reply

Who is online

Users browsing this forum: Amazon [Bot], Bing [Bot], Google [Bot] and 41 guests