Does Library Folder Organization Matter?

by George on November 7, 2012

Glass

Photographs © George A. Jardine

I keep reading from all the usual industry pundits that if you’re going to conquer the elusive goal of truly effective library organization, then you will have to adopt this or that folder structure. And although I’ve written a few myself, I’m beginning to rethink my position.

Even if folder organization does matter, there is very little consensus on the issue. Every photographer seems to make up their own library structure as they go along, and the world keeps right on turning. Photographers succeed by following their instincts and finding their own voice, so why would you expect them to behave any differently when it comes to organizing their digital libraries?

The important thing (in the short run…) is that you can find your photos. And so that should be the starting point for a discussion. Think about how you generally try to find any given photo or group of photos, once you have tens of thousands of them, and that might lead you to a surprising answer. (More about the long run later.)

pull quote

This goes back to an idea that I’ve been writing about for some time now; the way that we’ve learned to organize things in the real world tends to shape our thinking about how we should be organizing our digital world. Which is only natural. But does that approach make sense, when it comes to digital photos?

Think for a minute about where that thinking has taken us. In the real world, of course we organize by subject. We have drawers for specific kitchen utensils. And drawers for envelopes and stamps. We have places for our sox, and places for our underwear. And in the physical world, that makes sense.

Japanese Kitchen Knives

When we sit down at a computer, our natural instinct is to begin organizing by subject there, too. And this goes back to the beginning.

When desktop computers came along, a graphical interface with file and folder icons that you could drag around to “organize” was a huge innovation. It gave us a way to organize our computer documents that emulated the way we do things in the real world. And it worked great, at least for a while. It helped “the rest of us” get up to speed with computers. But once you had thousands of files, the system started to break down. The virtual mess inside of our computers became larger than the real mess in our offices.

I don’t know how many people will remember this, but there came a time when both Apple and Microsoft declared that future operating systems would eventually be just one big database. That files and folders would be going away, and when you needed something you would “search” for it.

Despite the fact that this probably was a very good idea, getting rid of the file system never got off the ground. (Until iPads came along, but that’s another story.) By the turn of the century, files and folders were just too ingrained to be taken away from us. Eventually Apple was able to give us a better way to work with certain types of digital media by using a database, and iTunes is probably the best example of that. In the case of your music, iTunes takes care of where it goes, and basically, you don’t care. When you want to find a song, you find it by artist, or album, or in a playlist. Which are all just methods of searching a database.

Random Materials

The logic of that seems like it should be inescapable. Once you have thousands upon thousands of “assets” in the computer, they essentially all look the same. And so good metadata becomes useful—even required. But photographers still tend to think in terms of organizing their digital photos into “places” (folders) by subject. After all, when you walked into a film library, you would know where to find a given piece of film, because you had given it a place. You gave it a place in the library, because in the physical world, you didn’t have any choice! The film has to go somewhere, and if you didn’t organize by subject in some form, you would never be able to find anything.

When it comes to computers, the inherent limitations of that approach should be pretty clear. But somehow that doesn’t prevent most photographers from following the same instinct when building a digital library.

345 Park Ave

Back in 2004 working at Adobe Systems, I was busy interviewing photographers, and thinking about their film libraries. We were in the early stages of Lightroom development, and wanted to understand how photographers thought about photo organization. The approach of one prominent underwater photographer left a lasting impression on me. He had shot 35mm Kodachrome all over the world, covering dozens of subjects. Each slide was filed in poly slide sheets, in stored in filing cabinets, by subject. The determination of that subject was of course, subjective, with no pun intended. If it was a photo of sharks, it was filed with other shark photos. But if the photo was taken in the Caribbean, it might get filed in that folder, by location. (Location is frequently treated as a subject by photographers, further complicating the issue.) If it was shot in the Caribbean and the photo had sharks in it, he had to make a choice. The original might go into the Sharks folder, while a film duplicate of it would also go into the Caribbean folder, and another might go into the Coral folder. Each with hand-written notes on the slide mount indicating where the original was. (Metadata!)

pull quote

One obvious downside to this strategy is that each roll of film was subject to being split up across dozens of filing cabinets, but such were the necessities of cataloging real objects in the real world. (One of the ideas that we get to in a moment, is that preserving the context of shoot order and chronology in digital libraries becomes very easy, and gives you a huge advantage over film libraries that are organized by subject.)

Anyway, back to digital photo libraries. Earlier I said that the folder organization you use doesn’t matter, as long as you can find what you’re looking for. And that truly is the heart of the matter. Because I’ll bet that if you can put your eyes on any given photo that you need in a moment’s notice, it won’t be by digging around in file folders. You will find it by using a search of one form or another. Even if you do sort your photos into folders by subject or location, once you have thousands upon thousands of them, the idea of where those bits are stored becomes meaningless. (Aside from the computer UI trick of file and folder icons, “where” is a meaningless concept in the computer anyway. But I digress.)

Things

So if that’s the case, why do most photographers still sort their digital photos into folders by subject? Nearly every one of my friends, and every photographer I meet, still has top-level folders in their library, separating photos by subject or location very much the same way you would in a film library. Stock photos in this folder. Commercial assignments in this folder. Photos taken in NYC go into this folder. And photos of their family, or their partner, into another folder.

But what happens when you take a photo of your wife or husband while traveling to NYC? Which folder does that go into? And even though nearly every photographer working in digital has figured out how to use keywords to solve this rather than duplicating actual pixels, most digital libraries are still segregated by subject.

pull quote

I believe this is true simply because most photographers haven’t thought through why they do it this way. A set of requirements for your library structure can easily be created by looking at your personal workflow, and asking yourself how you typically set out to find your pictures.

Pumpkins

For me, it’s all about keywords, or collections. If I’m looking for one of my “best” photos, that’s almost always going to mean one that I’ve used in a presentation, or published in a blog posting, or in a web gallery of some sort. And so I can always find my best photos within seconds, by looking in a short list of collections. (I stopped using star-ratings to try and create qualitative rankings a long time ago, once I realized how subjective they were. A 5-star photo in the context of any one day’s shoot, would almost never be a 5-star photo in the context of my larger library.)

Then, if a visual search in my collections doesn’t get me to the exact photo that I’m looking for, it usually leads me to a photo that I know to be from the same group of photos, or the same “shoot”. In this case I right-click on the related photo, choose Go to Folder in Library, and this takes me to the shoot folder. Once I’m there I see all the photos from the shoot, outtakes and all, in the exact order in which they were shot. With this context I can always get to the one photo I’m looking for, in just seconds.

If I am looking for a photo that isn’t in a collection, I’ll search by keyword. When I do a keyword search, it’s going to be a subject like “bridge”, or “wine”, or it’s going to be a location (“Mendocino”, or “Bangkok”), or maybe a person’s name. If a text search returns more than about 100 photos from my library, then chronology becomes important. At this moment the search again becomes visual, and seeing search results in chronological order gives me the context I need to be able to very quickly scan the grid, and zero in on the photo that I’m looking for. Usually within seconds, and even if it’s an outtake.

pull quote

And so in the end, what purpose does segregating actual photo files by subject really serve? (Other than to help you feel that you have everthing in “place”?) Once you’ve built a real library with tens of thousands of photos, do you ever really go into a folder looking for a photo by name? Or do you ever really need to copy or back up bits and pieces of your library by subject, as opposed to shoot folder?

I’m sure some photographers might answer yes to those questions. But for my library requirements, all I need is that my file and folder names give me enough metadata to sort chronologically, both in Lightroom, and in the file system (for archival and legacy purposes). For context in visual search mode, shoots are kept together one folder per shoot. And this is the smallest unit of folder organization in my library. Derivative files (panoramas, composites, etc.) are also managed by Lightroom, right in the same parent shoot folder as the source photo.

Does the fact that you have different file types or project types (such as panoramas or layered composite files) intermingling with the raw files in the shoot folder bother you? I say get over it. When do you ever need to look in there anyway, once you have a more logical way to find the bits that you’re looking for?

Do the pants in the back of your closet that no longer fit you, or the knives in your kitchen that you no longer use cause you to lose sleep at night?

End

{ 18 comments… read them below or add one }

John King November 7, 2012 at 1:03 PM

Good points George. I agree 100%. The Library isn’t so much about storage as it is about retrieval. I don’t have big commercial or wedding shoots to manage but I do have a system that is working for me.

I organize folders by year and month. End of story. Except for film scans which are in folders by film size.

Where I do invest some time with every import is the assignment of keywords that allow me to retrieve files based on location, subject or project theme. With a hierarchical keyword structure this is now a very easy task. Like you say, a subject folder is merely a keyword search away. And Collections are an ongoing and dynamic set of subject folders.

I do have a use for the star ratings. Using a binary system of 0 and 1, I know which photos are worth a second look either now or possibly later. And they may get promoted with more stars after editing. The zeroes will be reviewed for possible compression or deletion after a year when they show up in the “Review for Compression” collection I created for this purpose. The original files and their catalogs are safely on an external drive dedicated to Photo backup if I ever change my mind.

jdb November 7, 2012 at 10:07 PM

+1 on tagging and collections George. When I switched to LR recently, I went with your advice and have had a lot of fun back-tagging my library and building up smart collections. Good times.

As a hobbyist, I find it funny that folks spend time worrying about star ratings 0 and 1. All my zeros and ones get deleted on camera, or from the camera while reviewing on my TV. What gets imported onto my computer, and into LR begin as at least a 2 (without bothering to actually star them all). Then I promote them beginning with 3 or more. Perhaps photo businesses have a greater need for 5-star systems when working with high volume and production assistants…

Paul Beiser November 10, 2012 at 7:48 AM

Thanks, George – an excellent set of thoughts and some great best practices. I too remember when file system based OS’s were gonna be swept under the rug and everything was going Object Oriented… ah the good old days :-). I have had ongoing debates with people about Mail folder organization; some people just put everything in one linear folder and grep/google to find everything; other’s have elaborate systems. At the end of the day for me in Lightroom, I do pretty much as you recommend, seems to work great for me. I do like the idea of the collections as places for ‘keepers’ and the technique in LR to get back to the folder where ‘like’ photos will reside.

Thanks again!

George DeWolfe November 12, 2012 at 7:28 AM

George This is the most intelligent written essay on LR Library organization I’ve read. I personally categorize heavily with collections and collection naming, but every photographer has to find their own way here. A wonderful lengthy piece of advice and analysis of this thorny problem. George DeWolfe

John Belmuda November 13, 2012 at 3:12 PM

Hi George, makes a lot of sense, I am always looking to find a better way to retrieve my photos, started out with the standard database method you describe then was convinced by others that I needed to use catalogues for every assignment and ended up confused, I think that my past experience in database building holds true and your explanation is exactly along those lines, it is all about your search criteria etc. But I am interested in how you work this with multiple archived hard drives? any experience with that model? thanks for your hard work and thoughts. John Belmuda

George November 13, 2012 at 8:48 PM

Hi John,

My model assumes that you are able to create one, large Lightroom catalog, regardless of how many hard drives (for photo storage) it must access to be complete. Unless you literally have several hundred thousand photos, LR can handle it with the storage spread out across multiple devices.

Additionally, once you’ve built the catalog, all the devices do not need to be online for search and management. Just for Develop, Printing, and other types of output.

Hope that helps! George

Simon King November 14, 2012 at 3:34 AM

Good article George. I floundered for a long time trying to use my old model of place or subject. Now Smart Collections are they way I go

Initially I resisted keywording, until I realised the potential of smart Collections.

Mit der Smart Collections I can organise how I like, by rating, or date, or place , or concept etc

My, ah feel so liberated! ;)

David Bump November 16, 2012 at 6:37 AM

Fantastic article! Since Lightroom offers an easy way to organize pictures by date-stamped folder during import, I just let it do so. The one advantage I see to this is if my database ever becomes corrupt, or years from now, there is no program that can read that database (unlikely, but it’s happened to many other really good programs in the past), I can still access my photos manually, if needed–or import them into a different program. Even if that means losing my rich stream of metadata, I still have the raw captures. iTunes is a good example–if you decide to stop using iTunes and want to use some other program to organize and play your existing library of music, you need to use a 3rd party tool to read the proprietary database and rename all the GUID filenames (or hope that you can export them from iTunes). Databases are awesome, but they do limit you to working with the specific program that understands that database. Open format databases make it easier, but the average end-user would still need a program that can import that open format.

Andrew Rodney November 27, 2012 at 2:07 PM

>Once you’ve built a real library with tens of thousands of photos, do you ever really go into a folder looking for a photo by name?

I might, if LR no longer functions. Poop happens . Or if I know for a fact that the image of Sydney opera house is in the folder called Sydney and maybe I’m in Photoshop or InDesign and want it. I don’t like the idea of totally relying on any database if I have to, I’ll resort to finding stuff in the finder if there’s no other option. For that reason I believe a well structured folder system is better than a non structured folder system. Until Apple (in my case) decides I can’t use the finder this way, I’ll stick with this approach.

The good news is we have lots of ways to find these images within LR and I use such tools whenever possible.

The point David makes about iTunes is the reason I don’t want to totally rely on a LR proprietary method of finding images although the good news is, if you work a lot with metadata, any other decent DAM should find the images using the same methods. Smart Collections should be quite easy to recreate in another DAM that we’d expect has the same ability to find metadata.

I want to easily find my images. But I also don’t want to paint myself into a proprietary corner, anymore than the OS restricts me at this time. Popping 30K images into a single folder then assuming I can find 1 image with metadata is a bit scary to me. Popping 30K images into a few dozen folders that make sense to me (and probably only me) has thus far worked.

Bottom line is, whatever process makes sense to you, and one that’s hopefully flexible, is the best system. When I view other people’s desktops compared to mine, I wonder how they find anything. But they often do find one file in a cluttered mess as that’s how their minds work. For me, a date makes no sense (I can’t remember what I ate for breakfast). I can’t recall the dates, let alone the years I went to Sydney, but I know my images of Sydney are in a folder called Sydney. And the keyword Sydney works too.

George November 27, 2012 at 2:28 PM

Thanks for the comment, Andrew.

I think we pretty much agree, but two points that I keep trying to make seem to be missing the target.

One is that I can’t remember a date a specific photo was taken on either. The point of chronological folder organization is not so you can go in there digging to find a particular photo file by date. Remembering a shoot date is always the last way I will attempt to find anything! Chronological folder organization is about simplicity, logical groupings (what I call a “shoot”), and legacy. I believe that in the long run…. shoot date will be the most important grouping to archeologists…. should your or my library ever be deemed useful after we’re pushing up daisies.

Two (and perhaps much more important… again going back to that ‘logical groupings’ thing…) is that I’ll bet you have a folder on your system named “Karen”. When you and Karen go traveling to Sydney, do your photos of her go into the Sydney folder? Or into the Karen folder? It’s the same quandary that the underwater photographer faced when sorting film originals.

But now that we have keywords and databases that can be searched, why split up a shoot (or your entire library) breaking the coherency of chronology, just so that you can have your sox in one drawer, and your shirts in another? Because of the way Lightroom abstracts your output from your library and your workflow, we no longer have to organize computer files by usage, either. (Portfolio photos go in this folder, printing files into this folder…. high-res here, low-res here.) Following this legacy workflow (essentially the Bridge workflow) leads to duplicate files, lost corrections, and all sorts of other library management headaches.

George

Andrew Rodney November 29, 2012 at 1:16 PM

>>When you and Karen go traveling to Sydney, do your photos of her go into the Sydney folder? Or into the Karen folder?

There’s no Karen folder, don’t tell her

I’d put it into the Sydney folder only because my old brain should be able to remember I did indeed go there with her on one trip. And I’d have a Karen Keyword too. Even if I had a Karen folder, I’d still pop the image into Sydney and maybe have a Smart Collection to find the Karen keyword if I wanted to only find images of her regardless of location.

Another example. I have a folder called Santa Fe. I also have a folder called Dogs as I’m crazy about shooting images of my kids. They live in Santa Fe. And they have a Santa Fe keyword. But if I’m looking for dog images, I’ll go to that folder if I had to.

It works for me but I can see how this “logic” might not for others.

I was thinking today about this idea that where documents live on an OS based on not having to depend on the user putting items in folders. If you sent me an email with an image or attachment, where would it go? If I wanted to open it in Photoshop or LR weeks later, would I need to still search around to find it? IOW, in something like an Aperture of LR which could suck images into a database, what if I just want to open that image with another app? I’d hate to think I have to always go into a database to get access to that image. Maybe all app’s will have access to this area but I’d still have a lot of data to sift through to find the image.

Tim January 6, 2013 at 3:31 PM

Aloha George When I got my first DSLR in 2005 (sooo long ago!), my first decision was how to store my pics on my pc. Following the KISS principle I organized by date since the DSLR created the folder with a year/month/day title. Plus no “killer app” existed. I had tried several different programs to process the pics during that time and I wanted to keep a consistent organization on my hard drive no matter what program I used. Even with LR, I’ve kept the same physical organization of my files. After all,a “better” program might come along someday? :-) Of course I’ve been willing to use keywording since day one. Over the years, I’ve read some Pro blogging about the drugery of keywording. Without doing some minimal keywording a database is useless. Automation can reduce it but it still needs to be done.

imajez January 6, 2013 at 6:34 PM

I normally file my images by year/month/date-description and if it was a busy day I will split the the different shoots up, so 2013-01-07 Rhosilli Beach, 2013-01-07 King Arthur’s Stone for say locations or 2013-01-07 Joanne Blue Dress, 2013-01-07 Joanne White Suit, 2013-01-07 Pauline Green Scarf, 2013-01-07 Pauline Red trousers for example with fashion. This however is not because I’m into filing things by subject as I’ve always thought that to be a broken system – trying to file my physical music that way never really worked satisfactorily and the same applies to imagery and then some.

The reason I do it by a heirachical date-description is that the files have to go somewhere and organising them that way means any OS or any programme [or indeed any human] can look through the images and fairly easily find things. Also if you file by date, you are become better at remember when a shot happened I found and for some things I’ve documented I know that certain things usually happen on say a Tuesday, so can find the shots more easily than looking through all 13 weeks of the project. This separation on busy days also means I don’t end up with folders with thousands of images in them. Obviously keywording can makes life much easier, other than the fact that keywording properly is in itself is quite hard and very, very time consuming work. And also if you repeatedly photograph certain subjects it kind of breaks down as for example the keyword ‘lindy hop’ for me will probably bring up 80,000 images as it’s something I’m spent many years documenting. And the other gotcha with relying on keywording to find things is that if you don’t recall the keyword you are stuffed.

Heirachical structures like year/month/date-description and also putting keywords into heirachies are often underestimated in their usefulness. As those who tend to recommend keywords as THE solution never seem to consider forgetting names or places for example, so if I can’t quite remember the exact date or the name of that dancer I shot in New York I can drill down the heirachy to find the shot. Assuming I had time to keyword – sadly often I do not. So I file images using a belt and braces approach [braces=suspenders for the Americans]. A robust date–description method allied with keywords when I have time. But most of my library is still not keyworded as I’ve simply not had time to do it as a lot of it predates LR, which is still a bit clunky with the 9 word set limitation anyway. But as my files are all named YYY-MM-DD Description 001.RAW they are effectively keyworded by the filename to a certain extent.

There are times when professional work however can at times be better served by subject filing. Say you do weddings, then place shots in folder called Wedding with the folder having the couple’s name and that’s maybe all you need for that sort of work as the client tends to know their name. ;-) Plus you’d be well served by stars, colours and keywords to pick out shots form that sort of shoot to then add to website/portfolio etc

I should say also I love smart playlists for sorting out music, though dumb playlists/collections also have their place. Sadly as LR doesn’t allow manual sorting of smart folders, something I find absolutely essential for output purposes, I tend to use dumb collections far more often than smart ones. Unlike some of the posters above I do use stars – though it took me a while to work out my current system. Currently when looking through a folder anything that catches my eye gets a single star, I then filter folder down to just starred shots and if I then process it it gets 2 stars, if I like it enough to add to my website, it gets 3 stars, once uploaded it gets 4 stars and if I really, really like it it then gets a 5 star to go in my uber favourites folder. Though this may soon change to 3 stars when finished for job, 4 for website and 5 uploaded to website and then use a colour for top shots.

Matt February 12, 2013 at 12:51 PM

Very nice blog post, thank you. I’ve been using a State/City structure, but it’s been getting unwieldy as I have many many shots in certain locations. So I’ve been thinking of going the chronological route and using keywords for location. One thing: my software lets me enter IPTC fields, as well as keywords (I use City/State/Headline primarily). I’m curious if you or others use the IPTC fields or just the keywords, or both.

George February 12, 2013 at 5:10 PM

Thanks, Matt.

Yeah…. sorting by “State/City” is essentially sorting by subject, and works as long as your library doesn’t get too large. The problem I’ve had in promoting my chronological structure thinking is that people naturally think that’s how you’re going to “find” your photos. Of course you don’t remember what day—or even what month or year—you shot a given photo! That line of thinking is entirely missing the point.

But I digress. Now that I keep a Canon GPS unit turned on more or less all the time that I’m shooting, I just let LR fill in the IPTC location fields based on the GPS data. So that becomes a valid search item. But it also works pretty well to just go into the Map module and draw a selection around a location to “find” every photo you’ve shot in any given city or location…. maybe Venice, for example. It becomes just like a keyword search… doesn’t matter where in your library the file sits, it is returned by location via the map.

G.

Floyd Gustafson December 13, 2013 at 1:05 PM

Hi George, was the clutter you photographed owned by someone named Jay?

George December 13, 2013 at 1:07 PM

Perhaps. :-)

Mike January 8, 2014 at 1:30 PM

OK, I’ve been struggling with rearranging my “My Pictures” folder on my computer. After some agonizing, I’ve come up with:

YYYY\MM month_name\{project_name(s)}\file_name(s)

However, why do I care? All the photos could be in one big fat folder for all it matters because the software will sort everything by date and I can use tagging for other categorizations. Even Windows Explorer can sort by date.

I guess there’s the belt and suspenders approach that says if the software fails, there is a manual system fallback, but one can always sort by date using the OS’s file system browser. OK, the tagging could be blown if the tags are not imbedded in the files, but then there is the backup catalog…

Leave a Comment

{ 1 trackback }

Previous post:

Next post: