For years I have wanted a central, searchable and consistent photo archive/storage system in place for organizing my photography (along with other digital assets like video and audio). I wanted to be able to easily search, organize and store all of my past work and insure that it was scaleable so I could add all my future work. Like many photographers, when I made the transition from film to digital I started out treating my digital images much like I did negatives. After completing a shoot I would download the card(s) and name the folder using some keyword that described the shoot. I would then drop that folder into a larger folder that contained all my shoots. This was reminiscent of putting negatives into a PrintFile page and putting it into a binder that would then go into a shelf with all the other binders.
Throughout the years I had tried every method of storing my digital images imaginable. I used DVDs, CDs, Zip disks (from way, way back) and a variety of external hard drives. I would sort my archive by date and then change my mind and sort by subject or job and then change back to sorting by date. I would separate all my film scans from all my digital work. It basically added up to a big mess spread over countless drives and disks.
Several years ago I began figuring out the solution with the discovery of the usefulness of metadata. Because I did more commercial/freelance work rather then newspaper work, I didn’t know much about, or have very much concern over metadata and how it would make my life easer. It wasn’t until I worked at a newspaper myself that I began to understand the value of adding metadata to my digital assets, especially when searching through tens of thousands of images from multiple photographers. Everyone knows that adding a caption to a photo helps to identify the image and the photographer but, just as valuable, every piece of data you add to a photo (or video and audio) provides another way to quickly search out a specific photo.
Two big decisions for getting my archive started were 1) What hard drive I needed to consolidate all my digital assets to and 2) what software was I going to use to organize it. I ended up choosing a Drobo as my hard drive. Drobo is not a perfect system but for my purposes it works fine. It provided RAID redundancy so if one hard drive failed there would be enough data stored on the others in system to recover everything, and it is expandable so initially I only bought enough drives for 2TB of storage but over the years I can expand it to 16TB.
As for database software, after weighing all the pros and cons I decided to go with Adobe Lightroom. The reason I picked Lightroom over software like Aperture and Cumulus was because I was familiar with Adobe products and knew there would be frequent updates and improvements. The price was also much better than other similar software. The most important part was that it had to be able to handle both my old images and all the ones that I will take in the future.
Getting it done
In a nutshell, here is what I did to build my archive: On my new Drobo drive I created a nested folder structure based on date. The top level is the year, which inside breaks down to month and inside the month are the folders with individual shoots. Each shoot begins with a reverse date (year/month/date) followed by a job or project name (example: 20130113_JohnDoeWedding). I sometimes also add information to the folder title like camera type, if I’m shooting with multiple cameras. I then copied all my images into that framework and imported them into a Lightroom catalog.
After everything was on the drive and in catalogs I backed up everything to additional external hard drives and moved them to an off site location so that if something happened to the Drobo (crash, fire, flood, etc.) I would still have access to my photographs.
And that is it!…Ummmm Not really.
The Dirty Work.
Selecting the equipment and adding images to the catalog are the easy parts. The true major ingredients of this process are TIME and PATIENCE. Putting together a central, searchable and consistent photo archive was my New Years Resolution in 2011. I’m just finishing up now in early 2013. That was a total of TWO YEARS of work.
If someone is reading this and is thinking about taking on the project of creating a centralized photo archive, it is important to note that not everyone will have the same amount of work that I did. Some may have started using Lightroom or Aperture years ago and have much of their catalog already organized. I started from scratch. Some may have added metadata to their photos from the start; I didn’t start until about 2007.
It’s also important to mention that I did not work continually for two straight years on this project. I would do some work in the evenings if I had some free time. Sometimes I would work on it for a few weeks and then not touch it for 2 -3 months. I simply worked on it when I could. Some evenings I just thought, “I could watch something on TV or I could work on my archive.”
If you don’t already have a strong photo archive, taking time to prepare your images is key. I had years of images that had no metadata applied to them at all (other then the EXIF data from the camera on digital files) so I used the program PhotoMechanic to caption and keyword every photo. I found that PhotoMechanic is much easier to work with in terms of batch captioning then Lightroom. Adding this metadata was, by far, the most time consuming part of this project. Imagine going through tens of thousands of images taken over the course of years and adding caption information. I had to track down old notebooks, emails and other documents to find information on when, where and why I did the images. I also spent hours on Google finding information about events, people and places that I photographed.
I often debated about whether I should take the time to add this metadata, but to me it was important because it made every image consistent and allowed everything I’ve ever shot to be searchable within the database. Now that all this data is written to the image file I will never have to look it up again, it will always be with the file. Not everyone will have the stomach to caption years of old photographs but luckily I made it through and will never have to do it again.
The bulk of my existing, loosely organized archive was on 2-3 external hard drives but I also had piles of old DVDs and CDs all with images that I need to get into my nested folder framework on my new drive. Part of the problem was that since I didn’t have a central archive since the beginning, I often had to deal with duplicate images on multiple media because of my attempts to store and back up my files up in the past, so I had to check DVD, CD and hard drives thoroughly before I deleted or threw away anything.
I had to many photos to put them all in one Lightroom catalog so I separated them into multiple catalogs. I was able to separate them according to certain life events. For instance, I was at a job for several years then I moved on to another location where I had a job that I was at for several more years. There was an obvious split right there that I knew I could easily remember. From what I have found, most people say that on a newer computer a Lightroom catalog will run smoothly with up to about 100,000 images. While technically a Lightroom catalog can hold an infinite amount of images, eventually the computer’s processor and memory get bogged down with that much data that has to be processed.
Dealing with Analog Images
Another time consuming aspect of building a centralized archive is bringing together both digital images scanned from film and digital images produced directly from the camera. Luckily when I switched to digital from film it was a pretty solid break; there wasn’t much overlap. While I put all my digital and scanned images in the same nested folder framework on the hard drive I didn’t want to mix them in my catalog so I put all my scanned photos in their own catalog.
While on the subject of scanned images, another part of this project was scanning images from my negative library that I wanted to add to my digital library. Over the years I have made a list of images that, if I had time, I would like to scan. Luckily I have a Nikon film scanner so this was easy to do but it added a significant amount of time to the project. To make a high resolution scan it takes about 5-7 minutes a frame and I scanned several hundred photos so you can see how long that might take.
Since I was taking the time to organize my digital files I thought I had better take this opportunity to organize my analog files as well. Fortunately my negatives files were organized fairly well but over the years some of my binders had been scattered out between because of moves. There was also a decent amount of negatives and slides that never made it into binders. Much like with my digital archive, I brought all my negatives and transparencies in one consistent, organized location and filed them chronologically into binders. Now I don’t have to track down multiple boxes in multiple locations if on old client needs a frame scanned.
The Final Product
After countless hours of work I finally have my archive nearly complete. There are still a few things that I want to tweak. The ability to easily add GPS data in Lightroom 4, for instance, may come in handy for some of my past travel work. Looking back after all this work I have been asking myself, was all this work worth it? I would have to say, definitely! My entire library is finally consistent and searchable and I have peace of mind knowing that everything is organized and backed up in multiple locations. If someone wants to buy an older stock photograph from me I won’t have to dig through random hard drives or DVDs. I will find it along with all my other images. It will be captioned and my copyright will be attached to it. Most importantly, I made it so it would be scalable so as long as there isn’t some drastic change in technology, I can keep adding to it completely in the framework that I have established. This will save a huge amount of time moving forward because now that I have a process I can follow as I go and I won’t have to spend huge amounts of time entering large chunks of images in the future.