I've been synchronizing collections spanning up to 60GB and ten thousands of small and large files for years now. This includes my LR catalogs and pictures since I started photography. I'll be honest though, this is not a tool a lot of people use, it is not trivial to setup but it will solve your problem. Once it is up and running, it is very easy to work with and it will perform its duty quickly and effortlessly. As a software engineer, this is one of the top 3 pieces of software I use daily and can't live without.

The tool is Unison File Synchronizer.

Unison

Unison is a file-synchronization tool for Unix and Windows. It allows two replicas of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to date by propagating the changes in each replica to the other.

• It's free, run on both Windows and Unix, and even works across platforms.
• It can deal with updates to both replicas of a distributed directory structure.
• Updates that do not conflict are propagated automatically.
• Conflicting updates are detected and displayed (more about that below).
• Unison is resilient to failure. It is careful to leave the replicas and its own private structures in a sensible state at all times.
• It works between any pair of machines connected to the internet. It is careful with network bandwidth. Transfers of small updates to large files are optimized using a compression protocol similar to rsync.

Large files

This later point is relevant to your problem. If a large file has been updated (say, your LR catalog), it will not copy the whole file over. It will instead transmit a smart, compressed subset (a delta) representing the difference between both files. For example, if I modify the Artist name on a MP3 or add some keywords to a JPEG file, it will not send the whole song or photo over, but only a very small portion of it. Granted, bigger changes happen in your LR catalog, but never quite big enough that they require to send the whole file over. This makes sense because your catalog is a SQLite database (a SQL db on disk); for performance reasons it is in the database engine's best interest not to shuffle every bits around.

Renaming

Unison has no problem renaming files and directories. Files are not identified by names actually, but by a unique signature. This is very convenient.

Compared to Microsoft Live Sync

Similar to Microsoft Live Sync, Unison does not keep an online copy of your files (thank God), which means both machines need to be connected to the internet.

Unlike Microsoft Live Sync (from what I understand), it does display potential conflicts and its user interface will let you decide which replica should "win". That is, if you made changes on both sides, it will let you mark if the 'left' side should have priority over the 'right' side (see screenshot below), or vice versa. You can do so file by file, replica by replica, on a subset of files, etc. You can even, for text files, have Unison launch a tool that will let you resolve the conflict manually at the contents level (say, using WinMerge).

Thumbnails and Previews

Now what about thumbnails and previews? Do not synchronize them. You do not need to, I never did, and LR is smart enough to regenerate a preview if it's missing. This has never been a problem for me in 2 years.

Besides your RAW and JPEG files of course, which piece of data should you synchronize to make sure you can work with LR on both sides? There are 3 components to consider, to my knowledge:

• Your LR catalog file (Lightroom 3 Catalog.lrcat), which is where LR stores its metadata, keywords, rating, labels and develop settings. Remember, LR is non-destructive, it doesn't touch your original files but store every single change you made to a photo in its catalog instead (unless you instruct LR to dump these settings back to your XMP/DNG/PSD files, but that's another story and not a problem anyway). Your catalog is the important file to sync.
• Your Previews directory (Lightroom 3 Catalog Previews.lrdata), which is where LR stores intermediate representation of your photos for faster viewing (1:1, low res, thumbnails, etc). Do not synchronize this folder.
• Your Adobe Camera RAW Cache (ACR Cache). Every time you open an image in Camera Raw the full resolution of the image must be loaded into Camera Raw. As you can imagine, this can be pretty processor intensive. The Camera Raw cache will cache recently opened images to make re-opening them faster. This cache is actually used by LR and Photoshop if you have both. Do not synchronize this folder either, its content will be regenerated automatically.

You can change the default location of your catalog, previews and Camera RAW Cache from the Preferences. This is how I moved my catalog to a specific user directory (my HOME directory, with personal data), and this is the very directory I sync. You don't have to do that, but it makes my life easier.

Installation

As I mentioned earlier, installing Unison is tricky, check the user manual online, and comment if you are still in the blue. It does involve running a SSH server on one of the computer (if you sync between two), traditionally using Cygwin or a subset of it. This is the tricky part, but SSH is an open, robust, safe and secure solution. I trust this set up more than running Microsoft Live Sync or Dropbox, which are essentially client/servers too.

Once your SSH server is running, you are pretty much all set. You only need to write a small text file called a 'profile' that will tell Unison which folders need to be sync'ed. I realize it's not very user friendly, but it does have a powerful syntax that lets you ignore files using regular expressions or simple patterns. Here is what the top of a profile looks like, for instance:

root = d:\users\barre
root = ssh://[email protected]/d:\mystuff

The first line specifies where my local replica is, and the second where my remote replica can be found, on the foo.net (fictitious) machine at home for example, in the mystuff directory. This is even simpler if you stay within a local network. I use Unison to sync between 4 machines at 3 different sites.

ignore = Path {Pictures/Lightroom/Lightroom 3 Catalog Previews.lrdata}
ignore = Regex Pictures/Lightroom/.*\.lock
ignore = Regex Pictures/Lightroom/.*-journal

These 3 lines instruct Unison to ignore my LR thumbnails/previews, as well as any temporary lock/journal files found in the Pictures/Lightroom sub-directory (relative to the root directory specified above).

And that's about it.

Running Unison

The first time you run Unison takes quite a bit as it computes hash signatures for all the files under its new jurisdiction. This is key to help computing differences quickly. The next time you run it, it's astonishingly fast. The user interface is a bit rough because Unison is an exotic beast written in OCaml + GTK. Just pick your profile from a list and Unison will check your local replica for changes. At the same time it will call itself through SSH on the other replica to compute remote changes. Wait a few seconds and it will display a list of changes and potential conflicts to help you reconcile the two, if needed. If there is nothing to resolve, just click "Go" and sync happens automagically.

In this screenshot below I’m synchronizing my local machine (first column), with my remote machine named KANEDA. The second column, in between, is what Unison thinks should be done.

As you can tell:

• I deleted one file on my local machine (First Friday…jpg), and the change will be propagated from left to right (the arrow), thus deleting the file on the remote.
• I changed both my LR catalog and a DNG on the remote. The arrow is pointing from right to left, i.e. these changes will be propagated to my local replica.
• I changed my .emacs.el file on both replicas, in a way that can’t be merged. Unison tells me at the bottom when the modification happened on both sides, the files size, etc. The question mark in the middle denotes that it’s up to me to make a choice. This line is selected, so I can just press the left or right arrow key to decide which side should win. I can even override any other row.

This situation doesn’t happen very often to me anymore, 99% of the time I don’t have any choice to make, I just review the situation quickly, press “Go”, and Unison synchronizes both replicas.

In Conclusion

I've used this tool for years and I swear by it. I synchronize and backup music, photos, documents and personal data using Unison, a task that would be impossible to do efficiently manually. It supports very large collections and thousands and thousands of file. It handles renaming, smart sync of large files and works quickly. It never corrupted a single one of my files.

This is still a file-level synchronization tool. It is not aware of the subtleties of your catalog, only Lightroom does. If you made changes to your catalog on both sides, you will have to make a choice between the two. At least the whole file is not copied over, and the user interface makes it easy to tell Unison which one should win. As somebody pointed out in this post, a bit of discipline should take care of this problem. Your catalog is one complex file. Even if a general-purpose tool used high-level SQL commands to inspect your catalog’s database and try to synchronize it with another, I still wouldn’t trust it over Lightroom.

Good luck.