Well, well, well. This “I-need-a-GUI-to-make-sure-I’m-not-hosing-files” girl was forced to dive off into Terminal/UNIX land to fix a rather daunting bug. Due to the successful outcome, I feel more than obligated to document it and save it for [hopefully unnecessary] later use.
Long story short, I had to compress and properly format thousands of large photos being used on my server without renaming/moving them. A recent update to a camera system caused photos to upload to the server as TIFFs and not JPEGs and this has been going on since November. Awesome! So here I am with thousands of enormous photos that are being used by a live site. Sweet setup, right?
So after pacing about for a while and considering the daunting task of manually fixing all the files i.e. my personal hell, I ‘hit the books’ and started sniffing out bash scripts and image conversion tools.
Fortunately, my coworker is rather familiar with the fun of bash scripting thus he had immediate tips to get me on the right path.
Step 1: ImageMagick (link to Mac installer)…Whew, does this thing do some work! Soooo many tools and tricks for playing around with image files. The ‘convert’ command is likely the ideal means of altering files, as you can maintain original files. Buuuuut we, too, like to live dangerously. Bring in Mogrify! I promise I remained calm while reading that “this tool is similar to ‘convert’ except that the original image file is overwritten.” ::cough:: No going back? GREAT! Sounds like a solid idea for a newbie. Yeesh. Unfortunately, it’s exactly what I want haha advocacy for heavy pre-testing? Sure, why not.
Step 2: Get a testing area and play! Get that mogrify up and running and see what happens.
Immediate issue #1: Our image filename convention is simple for dynamic functionality. Basically indexing six images sequentially with the unique product number for quick pulling. This seems to clash with the ‘update-in-place’ effect of mogrify. It did the trick for most files, but had a tendency of renaming the reformatted files to [filename]-0.jpg and creating a preview file called [filename]-1.jpg. So it’s clearly indexing, too; not ideal. I definitely needed the image names to stay the same! Meh, time for some extra script lines.
Immediate issue #2: We don’t want to alter ALL the files in this giant directory (100+GB of files), but just the ones that are large ‘n in charge. SO! We need to single out the files that are over 1.3MB and THEN run some script magic. This would essentially single out the mixed up files dating back to November. We have our friend “-size” to do the trick. Perfect!
The rest was just cleaning up the indexing mess from mogrify by deleting/chopping the files. Nothing some -exec and -delete commands couldn’t handle. After some tinkering with this guy, we managed to get a final, working script:
#!/bin/sh
cd /Path/To/Stuff/You/Wanna/Change
find . -size +1300000c -name "*.jpg" -exec mogrify -format jpg {} \;
find . -name "*-1.jpg" -delete
find . -name "*-0.jpg" -exec bash -c 'mv "$1" "${1//\-0}"' -- {} \;
Line 1: Get to the main photo directory.
Line 2: Find files that are over 1.3 MB that are JPGs and reformat them to true JPGs (again, these mixed up files are TIFFs with JPG extensions; nice ‘n messed up, right?).
Line 3: Find all the extra files that now have the mogrify “-1.jpg” convention and delete them.
Line 4: Find all the reformatted, desired files with “-0.jpg” convention and simply chop the “-0” from the name. Voila!
We setup a few test runs on the Desktop to play it safe. Turned out nicely! Dropped files from 1.5MB to ~100kb in one swoop while maintaining the original filename. Awesome.
Time to saddle up and run the real deal, right? Wait!
Step 3: BACK UP EVERYTHING AND ITS MOTHER. Despite my surge of confidence and blatant excitement over this script, I knew things could still hit the fan. A simple download of Carbon Copy Cloner did the trick for us! After a nice ‘n safe back-up, NOW we can pull the trigger. If you are used to bash scripting, you can skip to Step 5. Otherwise…
Step 4: UNDERSTAND how bash scripts execute! This is an important step I missed prior to all of this fun. I really haven’t done anything drastic (or at least multi-lined) with shell scripting and thus missed the fact that each line loops through the entire directory before moving to the next line. I’m so used to languages looping through all steps before moving to the next index, but not the case here. This tidbit of info would have likely saved me a lot of unnecessary anxiety when I watched it run for real!
Step 5: HIT IT! Well, okay…make sure your ‘cd’ line is correct but then pull the trigger! Stop waiting and worrying; just do it. You have the right script lines, you did the testing, you have the clone…you’re good to go! Do it!
Finally: Sit in fear for the duration unless, of course, you went through Step 4 haha. I nervously watched the system churn its way through thousands of files. Skipping some and drastically and irreversibly altering others. All in hopes that it’ll come out smoothly in the end. Keep an eye on what it’s doing and where it’s venturing, as you may have some slight cleanup for directories that fit the mold of those you wanted to fix. Plus, you should watch out for any permissions clashes, which could easily hose all alterations. Fortunately, the script covered 95% of the necessary changes and I was able to clean-up the last 5% within a few minutes.
So that was it! My first true-blue run with a hefty bash script. My website is happy again with its proper images restored and I can relax knowing that everything is in working order. Back to laid-back PHP for now; I need to decompress!
Happy coding!
