Results 1 to 10 of 10

Thread: Batch Processing for Everyone

  1. #1
    Join Date
    May 2005
    Location
    Amissville VA
    Posts
    3,827

    Batch Processing for Everyone

    (Note, image example links at bottom)

    For anyone who takes a lot of digital images and wants to pass them along to folks, batch processing is a way to automate tasks that could become extremely tedious when applied to several images.

    Take for instance, compressing a JPEG file. If your camera saves images in JPEG format (or higher end/dslr in RAW format) the size of the image may be very big. This is good - you get the BEST quality. That can be handy when working with the image for things such as blowing it up, printing large format, etc.

    But, say you want to store them online - well, 1,000 3mb images = 3gb. Over time, your images can take up some space. While it may not matter locally thanks to large hard drives and DVD backup media, storing them online or emailing large amounts of photos, you will reach some online limitations faster.

    So, compressing and resizing images are two VERY common operations done to the images that come directly off a camera, prior to sharing.

    I like to do both. (I've actually switched to only compress, then upload to a Coppermine image gallery that auto-processes resizes based on parameters I previously inputted on the server side...but that's another story...)

    So if you take 200 images, and want to compress them, you may use something like the GIMP (a great open source image editor) that can resize a digital photo.

    So you'd:

    1.) Open image
    2.) Save image as .jpg
    3.) Set .jpg quality
    4.) Save

    That's four steps. Do it 200 times, and it's 800 steps. At least 800 mouse clicks - OUCH!

    How about resising?

    1.) Open image
    2.) Set image size parameters and apply
    3.) Save as (Rename to save original high res version)
    4.) Set .jpg quality
    5.) Save

    That's 1,000 clicks!

    Batch processing would allow you to setup a single operation, and let the batch process apply that to all image you wish.

    Batch processing can be VERY VERY powerful, or, relatively simple.

    These examples are relatively simple but very time-saving operations.

    First, you'll need an image editor that supports Batch processing. Many high end image editors support batch processing, some are done by recording a "Macro", i.e. doing the operation while "Recording" the steps, then playing it back automatically.

    The GIMP has a script editor called script-fu, but it can be confusing to use.

    A program I like to use, is called "ImageMagick" and is certainly an odd beast being a "command" line image editor - that's right, you can create and edit images fully by text commands! ImageMagick support batch, gives high quality images in it's output, AND it's free!

    Now, I know command line can sound scary, but I don't think you'll have a problem with these simple operations.

    First, download and install ImageMagick:

    Most users will want to install from a binary. Cool folks and other Linux/Unix/BSD users may choose to install from source. Let's not get into that, let's just install Windows version via Binary.

    http://www.imagemagick.org/script/bi...es.php#windows

    Per the text, 99% of you will want this release: ImageMagick-6.4.0-11-Q16-windows-dll.exe
    (Note: version # may change over time, but this is for the dynamic Windows 16bit per pixel)

    Get it - install it.

    The cool thing about image magick is that it will integrate commands into your user environment. So........image magick commands will work from Windows command line (or .bat files if you wanted to)!

    So let's keep it simple for now.

    We will assume:

    1.) You are "converting" images meaning the ORIGINALS will be saved and NEW VERSIONS will be created with each command.

    2.) Operations are to be run on an ENTIRE directory. Place ONLY the images you wish to convert in this directory. (HINT: I like to use a "temp" directory, for instance, create a dir on root (C:\) called "IMTEMP", use it to stage and process files, and move them to their permanent homes when processing is done.)

    3.) We'll do two separate operations to compress and resize

    Compressing Images

    To create compressed versions of images using ImageMagick, move all the originals you wish to make compressed versions of into your temp directory (C:\IMTEMP if you followed my lead)

    Open your command line tool, in Windows click Start, then Run... and type "cmd" You will be presented with a command line sitting the your home directory. Follow the commands, first step you will change to the C:\IMTEMP dir


    Code:
    \[Homedir]\> cd \IMTEMP
    Next, we run the convert command, set to compress the images. What we are doing here, is "converting" the images using the convert command, tell it which images to do (all images in the dir) what quality level to save them as (90% is a good benchmark) and finally the new file name schema (ImageMagic will automatically add numbering to the end of the file name)

    Code:
    C:\IMTEMP> convert *.jpg -quality 90% sample_image.jpg
    Running this command will convert all images to the new file name, saving them at 90% quality, which is about half the original file size. (Due to how jpg compression works)

    When you hit enter, depending on the amount of files you have to process, you will not see the command prompt reappear until the operation is done. (Your command shell window will remain open, but the prompt will not be available until the operation is done)

    TIPS:

    1.) You can see the convert process in Task Manager. Go to task manager, processes tab, and find "convert". I have noticed it does use a fair amount of memory if you process alot of images.

    2.) If you want to background process alot of images with minimal slowdown to the rest of you work, right click the "convert" image name, go to "Set Priority" and set it to "Low". This will make it a low prioirty process that will slow down before higher priority tasks you may be running, such as your net browser, DVD player, etc.

    When done the command prompt will reappear.

    So, in your temp dir, if you started with IMG_1.jpg and IMG_2.jpg you will now see four files:

    IMG_1.jpg
    IMG_2.jpg
    sample_image-1.jpg (compressed image)
    sample_image-2.jpg (compressed image)

    So there you have it, you batch compressed using open source software, and yes, the image quality you will get will be very good!

    Resizing

    Another common batch is resizing. Emailing a 3000 pixel wide image may make it hard to view for the receiver. (Most monitors are only 1024 pixels to 1920 pixes wide depending on size and pixel density). Same goes for putting them on websites, smaller images will usually fit better. Yes, it is true that modern browsers support scaling of images (Automatic Image Resizing) but these are very crude tools and make big images look pretty bad when scaled down.

    My favorite image width is 780 pixels. It just seems to work for 95% of monitors, yet still be pretty big even on high resolution displays. (Anyway, if you REALLY want to send or post that high res version, you have that niced compressed yet full pixel-size vresion we created above!)

    So, again, let's use our temp directory. Clean it out - otherwise you'll resize all four images in there now, the originals AND the compressed ones. That's redundant, as resizing will save compressed anyway!

    Move any images in the directory that were created during the past processing. Keep or replace the original versions you wish to resize. Again, use the originals off the camera for best quality...

    The hard part of this will be getting the pixels you want to resize. For the same resolution, you only have to get this ONCE and then just tuck it away somewhere, i.e a sticky note, notepad note, etc! Use your normal resize technique like GIMP to resize the image keeping the scaling constraints.

    So...my D40's images come off the camera at 3008x2000

    When I resize to 780 pixel width, I get 518 height keeping the image ratio (H vs W) the same. Your image editor such as GIMP will tell you this.

    So, I will use 780x518 to get my pics looking right at 780 pixel width.

    Open your command line, go to your temp directory, and run this command:

    Code:
    C:\IMTEMP\> convert -resize 780x518 *.jpg -quality 90% sample_image_rs.jpg
    What you'll end up with in your dir would be...

    IMG_1.jpg
    IMG_2.jpg
    sample_image_rs-1.jpg
    sample_image_rs-2.jpg

    TIP: You can name your resultant images whatever you'd like. You can make it descriptive, or match the camera's naming convention. I like to use "rs" simply to denote a resized version.

    The two "rs" images in the directory, if you open them, will be 780x518 in dimension, and were saved at 90% jpg quality. They will look nice too!

    So there you have it, batch processing compress and resize for everyone!

    So you like that and want to do more with ImageMagick???

    Have at it!: http://www.imagemagick.org/Usage/

    Example...original, compressed, and resized. Click the filename to view/compare the quality and pixel size of each image.

    Original image from Nikon D40 (ISO 800) - File Size: 2,731kb, Dimensions 3008x2000 pixels
    DSC_0600.JPG

    ImageMagick Batch Compressed to 90 - File Size: 1,244kb, Dimensions 3008x2000 pixels
    sample_image-0.jpg

    ImageMagick Batch Resize (and compressed to 90) - File Size: 183kb, Dimensions 799x518 pixels
    sample_image_rs-0.jpg
    Last edited by Breadfan; 04-30-2008 at 11:27 AM.
    Trilogy #112 (self installed)

    - Trilogy S/C (Stock pulley/tune) - 4.10's - JLT - SS Inserts - 4" Megs -

    12.99@106.2mph (for now)

  2. #2
    Join Date
    Nov 2004
    Location
    Derby City, KY
    Age
    23
    Posts
    6,484
    THIS is an excellent write-up!

    I have often wondered how to setup a bat file to accomplish what you have outlined above.

    Now I know. And will certainly give it a try.

    I will add caution here that if you like saving your original images and a copy of the original after enhancing as I do, you will go thru disk space rather quickly by copying/moving image files around on your local HDD.

    Check your disk space prior to copying/moving and have fun!

    Thanks Breadfan!

    BTW, this thread is now 'stuck' as this can benefit many of us.

    Mike.
    Last edited by MM2004; 04-29-2008 at 03:20 AM.

  3. #3
    Join Date
    May 2005
    Location
    Amissville VA
    Posts
    3,827
    Thanks! Wow a Sticky

    A few other notes...

    - I chose ImageMagick due to it being a truly Freeware image editor that supports batch. There are other "shareware" image editors and some that look shady - remember not all free software is "free" often times you get trojans or spyware along with it... ImageMagic has their source code available which makes it more comforting to use. And, unlike expensive image tools, it's batch processing is free and still good quality.

    - I have not benchmarked ImageMagick's processing speed to any other applications, but if you process a large amount of big files, it can take awhile to run. You can view the run in Task Manager, otherwise you may not see other activity, for instance, you may not see files being placed in the directory until well into the run, and the command line will not show anything. (Do not know if there is a verbose command for image magick...)

    - Also, on that note, ImageMagick is memory intensive. If you don't have a ton of RAM (memory) you may see a warning about virtual memory space, I had this happen when compressing 670 3mb images in one shot. Windows will manage the virtual memory and you shouldn't have any issues.

    - As Mike said above I would copy your originals to your temp directory to work on them, saving the first copy of the originals eslewhere. That way if you mis type a command and lose your files, you'll have the originals as a backup.

    - If you want a powerful image editor with a graphical interface that can do many cool things try the free and OpenSource "GIMP" : http://www.gimp.org Remember to thank the open source development community!
    Last edited by Breadfan; 04-29-2008 at 07:00 AM.
    Trilogy #112 (self installed)

    - Trilogy S/C (Stock pulley/tune) - 4.10's - JLT - SS Inserts - 4" Megs -

    12.99@106.2mph (for now)

  4. #4
    Join Date
    Feb 2005
    Location
    Parkton, MD
    Age
    57
    Posts
    1,552

    Huh?

    Mike, I'll make sure to contact you so YOU can post a bunch of my pics. Just reading the first 3 paragraphs made my head spin...
    Just another clown on a bike...

    As good as any man, better than none.

    Former owner of a highly modified and very fast Marauder.


    "You CAN fit a square peg into a round hole, all you need is a big enough hammer." -Original

  5. #5
    Join Date
    Nov 2004
    Location
    Derby City, KY
    Age
    23
    Posts
    6,484
    Mike and I talked over the phone and have agreed to share ideas while creating a batch file(s) for people wanting to take advatage of this tool.

    The instructions would be straight forward and simplifed down to a few mouse clicks while the .bat file does the work for you.

    Stay tuned...

    Mike.

  6. #6
    Join Date
    Feb 2003
    Location
    Doin time in the Universal mind
    Age
    58
    Posts
    10,837
    Very nice write up...and waaaayyy too busy busy for my short attention span.

    Thank goodness Nikon Capture has a built in batch process in the task bar.
    2017 Niro

  7. #7
    Join Date
    May 2005
    Location
    Amissville VA
    Posts
    3,827
    Quote Originally Posted by CRUZTAKER View Post
    Very nice write up...and waaaayyy too busy busy for my short attention span.

    Thank goodness Nikon Capture has a built in batch process in the task bar.
    Yeah, being command line I didn't wanna skimp on the details.

    BTW the above works great on Linux too.
    Trilogy #112 (self installed)

    - Trilogy S/C (Stock pulley/tune) - 4.10's - JLT - SS Inserts - 4" Megs -

    12.99@106.2mph (for now)

  8. #8
    Join Date
    Nov 2004
    Location
    Derby City, KY
    Age
    23
    Posts
    6,484
    Quote Originally Posted by Breadfan View Post
    Yeah, being command line I didn't wanna skimp on the details.

    BTW the above works great on Linux too.
    Cool!

    I just pulled down an .iso of Ubuntu Linux yesterday!

    With a spare partition of 30GB, I am about to have a triple-boot system going on.



    Mike.

  9. #9
    Join Date
    May 2005
    Location
    Amissville VA
    Posts
    3,827
    Cool! Good luck! Nothing like triple booting!
    Trilogy #112 (self installed)

    - Trilogy S/C (Stock pulley/tune) - 4.10's - JLT - SS Inserts - 4" Megs -

    12.99@106.2mph (for now)

  10. #10
    Join Date
    Aug 2004
    Location
    S.E. of Cleveland
    Posts
    1,525
    Here's a great program I found on the net for correcting date/time photo (exif) information.

    http://www.relliksoftware.com/pages/...te-changer.php

    It will perform a date/time offset to all the photos in a directory and output them to a "output" directory.

    I had 6 different people with four different date/time settings on their cameras and was looking to merge all photos into a single directory chronologically, thus this discovery.

    TIPs:
    You have to put each users photos in their own directory. Get one directory you know the date/time is correct and try to match up the other persons photo time if possible by comparing images. Then calculate the date/time difference and plug into the program. The program will batch process the whole directory.

    Under the Options tab, the program can also rename the files by putting in date/time/other characters for the whole directory. I added the date/time as a prefix to all my files so they can be sorted chronologically by non-geeks.

    You may want to check first if the filenames may overlap from other's cameras prior to merging all files into a single directory. Therefore the rename function can add a character or whatever to id the filename differently.

    I named mine as follows "yyyymmddhhmm_filename.jpg"
    yyyy = year
    mm = month
    dd = day
    hh = hour
    nn = minute

    MAKE SURE YOU BACK UP PHOTOS BEFORE EXPERIMENTING, JUST IN CASE.
    ______________________________ _______________
    2003 BLACK 300A 06/19/2002 Build 1665 of 7839


Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. New Batch of pens
    By Slowpoke in forum Community Discussion
    Replies: 15
    Last Post: 03-15-2003, 12:07 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •