Backups - What's your system/method of backing up scans?

General Film Photography
1

I was wondering what others’ methods are for backing up their film scans. I’m currently scanning everything with a Fuji X-Pro 1, and these RAF files need to be converted to DNG. After doing so, they’re very large (70-90mb.) I’ve gotten a lot better at organizing things in Lightroom using smart collections, tags etc., but I’m eventually going to run out of space.

I could only keep “the good shots” since it’s film and I will have a physical negative. But that will eventually take up space, anyway. Also thinking about only running Enhance Details (converts to DNG) on the photos that I’ll actually export for use. But again, that only gets me so far.

So, just curious about what others use to keep their backups going smoothly.

  • Pre-built NAS solution
  • Multiple external drives
  • Build a server
  • Convert everything to TIF/delete DNGs
  • Do you even archive an entire roll?

I’d love to hear solution ideas from anyone, thanks!

2

I work as a digital tech on fashion and advertising shoots all over - there are many ways to skin a cat as they say, and multiple ways you could do this. For simplicity sake, maybe the workflow I use personally can be of some help. Here’s the condensed version of what you could do at home.

Hard drives are cheap and your time and work is valuable. Make yourself a set-it-and-forget-it backup system

Assuming you’re starting from scratch, buy yourself 4 drives, and some backup software (Chronosync)

One will be a clone of your laptop - so get one that’s the same size, or larger (to future proof it) than the internal drive you’ve got inside. Use Chronosync or some other backup software, and set it to mirror/clone your laptop onto it every time it gets plugged in - that way it’s automatic and you don’t have to think about it

Second, buy three external drives. Two identical, and a third from a different manufacturer - make sure they’re all the same size. For example, all 8TB drives, one from HGST, two from WD.

The HGST one would stay on your desk and could be your working drive with all your scans on it

The other two are clones of this working drive above. One stays on-site where it gets synced with Chronosync every week. The other is the same, though it lives off-site - either at a mates house or your office/house depending on where you work from.

Those two drives get rotated every two weeks and this prevents fire/flood/explosion scenarios where you’d lose everything otherwise.

If you have a fast internet connection, you could also look into backing up your scans online via something like Backblaze - anything goes haywire with your stuff at home, they’ll even FedEx you a physical hard drive with your backup on it so you can recover from it that way. Send them the bard drive back when you’re done and they’ll refund you the cost of the drive too

That should sort you out completely. :+1:t2:

Just in case you were curious about how digital-techs do this for photographers professionally. We just do the above on a slightly larger scale, backing up as we go along, and multiple copies for redundancy, each given to a different person.

At the end of the day, the photographer will have a drive with his copy of what we’ve shot, I’ll have a copy, the client will have a copy, the producer will get one, art-director etc. Sometimes when shooting abroad, we’ll even get a copy that gets FedExed to the photographers’ studio/home.

At the level of some of these jobs, we absolutely have to know our shit. You hear those stories of wedding photographers getting robbed and losing all the files from the wedding because they only had one copy in one place? Imagine that happening on a shoot with a couple of million riding on the ad-buy, with A-list talent being flown in from all over, a set-build into the tens or hundreds of thousands, crews of 50+, all for one day on set that can’t be rescheduled or re-shot. Now imagine only having one copy of that shoot… it just doesn’t happen.

Imagine the photographer’s laptop gets stolen? No problem, - there’s a backup of his laptop, and the job he shot on a small SSD on his person. He gets robbed and completely stripped of everything? Insurance covers the gear stolen, he can restore his laptop from the cloud or his backup at home, and I still have a copy of the job on me. We’re together and both somehow get robbed of everything? No problem - the client, art-director etc all have copies of the job on them too. Everyone’s on a flight that goes down killing everyone? No problem, We’ve FedExed a copy to the client before getting the plane, and even though we’re all dead, we go out as legends. All good!

3 Likes
3

Thanks for the detailed response! This is awesome information and I’m sure it’ll help others as well. I actually use Backblaze to back up my laptop but I didn’t think to use it for an external drive(s). I may use a bit of your advice with multiple drives + Backblaze but also make TIF versions of my scans to archive. There’s really no need to keep the DNG files once I’m done with them. I can always scan some again if necessary. I’m mostly trying to figure out a way to archive all of this stuff and keep it in Lightroom for organizing/sorting the ones that I do want to grab quickly.

Maybe I’ll archive each year and start a new collection per year, for example start a 2021 collection January 1st and just archive anything I’ve exported from scans in 2020 to keep it clean.

I really appreciate the level of detail, you got me thinking in a better direction :+1:

4

Why do you feel you need to convert your RAF files to DNG? If you leave them in RAF format, they will occupy less drive space and you have full flexibility if better raw conversion options appear in the future. (There’s nothing magical about DNG, it’s just a TIFF 4 image file with some additional metadata…)

5

DNG uses lossless zip compression and typically saves 10-30% on file size, and as much as 70% on Sony ARW files. It’s not the only reason I convert to DNG, but it’s worth noting.

6

That’s just a quirk of Sony’s porky raw file format. Raw files from the systems I use (Fuji, Olympus) invariably get larger after conversion to TIFF or DNG even when applying lossless compression, based on my own tests.

7

Hi,

If you do not convert your RAF files to DNG via Lightroom’s Enhance Details or some other way (there are other apps, I just keep my workflow in Lightroom), you will get very strange, squiggly/wormy pixel patterns due to the way Lightroom handles Fuji’s raw files. I do plan on switching to a newer, non-Fuji camera for scanning in the future. Until then, this is what I got.

Let’s talk about backup/organization of film scans, though.

8

That hasn’t been my experience, but I try to stay out of “religious” debates. Can we agree on advising the original poster to test his/her/their/its own files to determine which conversion/compression method, if any, produces the best results for his/her/their/its purposes?

9

Hi, again. It’s me, original poster. It isn’t something up for debate, it’s just an issue that Lightroom has never really addressed for RAF files. Here are three random google search results with more information:

Anyway, am more interested in peoples back up and organization methods, routine etc. Specifically for film scans.

Thanks!

10

As Chico Marx’ character says to Margaret Dumont’s character in Duck Soup: “Who ya gonna believe, me or your own eyes?” I’ve been shooting with Fuji for a long time, I’m already familiar with those articles and a lot of others like them, and I still believe my own results over random Google search results. But never mind, do whatever you want… I’m done here…

11

A short foreword: I’m in the convenient position of not having to convert my RAW files to DNG, because my Pentax DSLR allows me to choose DNG as its native storage format right away. I see that some people want to convert their RAW files to DNG, because there is some assumption it would be something like a long-term standard supported by most software in the future. Well, on the other hand it has happened in the past that the conversion of specific RAW formats was flawed and people, who got rid of their original RAW files had no chance to recover what the DNG converter has mixed up… I would rather stick with the RAW format that comes right out of the camera, but I see the issue with the worm patterns of Fuji cameras in LR. However, when you are talking about such big file sizes for the DNGs: are you embedding the original RAW files in the DNGs? If so, this will of course eat up a lot of hard disk space.

Now to the topic, and there are many options as already mentioned by others. My approach is as follows:

  • I generally delete files that don’t pass my culling process. For me there is no point in keeping those. For images I have captured from film negatives or positives however I’m a bit more reluctant with deleting. I need to be 100% sure that a photo won’t serve any purpose in the future before I delete the digital version of a film based photo. The reason is quite simple: there is some significant effort I have spent in setting up everything, cleaning the film, ensuring proper exposure and even light distribution etc. I’m keeping the physical films for as long as I have space for them, but I could imagine getting rid of them once I’m certain that the digital captures are flawless.

  • My backup setup is quite complex - some people would say. :wink: But I spent enough time on images and music that I want to make sure that everything is secured. First of all I have a secondary hard drive in my PC tower. There is a scheduled script that will create versioned backups of regular files, but for media files as big as RAW photos, videos etc. I came to the conclusion that versioned backups will take up too much space. I have configured LR so that metadata changes are written back to the DNG files. In case my LR catalog would be corrupted at some point (a phenomenon that happens to some people) all development steps and metadata would be part of each DNG file. If you’re using any other RAW format, LR will create these tiny XMP sidecar files for each RAW file. As you can see: as soon as I’m “touching” a RAW file by adding keywords or developing it, the file will change and a versioned backup would add the whole file to my backup medium. Therefore I have decided to just synchronize media files using the Windows tool “Robocopy” (robust copy), which is provided by Microsoft. My script will mirror my media folder to the backup HD inside my PC. Since the data transfer happens inside of the PC via SATA it’s quite fast. But I have an option to sync that backup hard drive to another disk connected to my NAS system (actually a simple Raspberry Pi) and I’m also syncing the whole thing (versioned backups and mirrored media files) to an external USB disk that is stored outside of my home. I do that every 2 to 4 weeks. There is another option in my backup setup: I have a Corsair Survivor Stealth USB stick that is stored inside a small fireproof safe at home. It holds another copy of the versioned backups and only the whole Lightroom catalog with all the previews. I let LR generate smart previews, which are part of these files. Smart previews contain a “limited” RAW file with smaller size and reduced dynamic range, but in the worst case I could work with these versions of my photos and get some good results in an acceptable size. Surely not ideal, but for the much smaller space requirements it’s a really nice 3rd level recovery option.

  • There was a time, when I would synchronize all my RAW files to my Nextcloud storage as well. It’s hosted at my web hosting provider, so the transfer is not as quick as it should be for the amount of data. When I come back from a trip or vacation I will import thousands of photos and that surely took too long before everything was synced to the cloud. And webspace was limited even though it was quite a lot… I stopped doing that.

I hope this was helpful in any way. If needed I could share more details about the individual steps.

12

Thanks for going over your setup! I didn’t mean for the DNG conversion part to become such a hot topic of this discussion. Maybe it’s how I worded the original post, my bad. It wasn’t even that important. But I really appreciate you taking the time to write out your process!

1 Like