14 December 2009
Synchronize two folders on a Mac with Automator and Rsync
Power of Automator and Unix combined
I really miss some good Automator tutorials online. It's such a great and I guess underrated and underused tool. Here I'll show you how to combine it with rsync to easily synchronise two folders.
You can find the finished workflow from the tutoria on the right so you can use it right away and see how it's done for yourself.
And here it is for the patient ones, step by step...
You'll start by opening automator.app, you can find it in your Application folder it has a nice icon which reminds me of eve from Wall-E. Once you start it, it will ask you what kind of workflow you're going to make, we'll choose Workflow:
First action we're going to add is "Ask for Finder Items" (it's inside Files & Folders category, but best way to find actions is to use search), that action simply opens finder window where you can choose file(s) or folder(s).
And here we can write some custom text in "Prompt" field, you can specify which folder will it open as a starting point (Start at), and we'll specify that we want only folders and we do not want multiple selection option.
Next we'll add "Set Value of Variable" action which will get the folder that you chose and write it's name into a variable that we'll use later with Rsync.
Next, we'll repeat it, we need another pair of Ask for Finder Items and Set Value of Variable actions. First we add Ask for Finder Items:
The important thing here, is to ctrl + click (right click) on the Ask for Finder Items text and choose "Ignore Input" option, if you do not do this the whole thing will not work.
Then we'll add Set Value of Variable
And we'll click on Variable drop down and choose New variable and name it for example "target_folder".
Next step is optional, we'll add confirmation window (Ask for Confirmation action) that will ask us if we chose the right folders and print out the names of the folders we're going to sync:
Here you start writing something like "You'll be syncing from" than you drag the variable "source_folder" from variable list below, continue by writing " to " and than drag "target_folder" variable from the list, and finish your confirmation text, for example " is that right?".
Here you're also have to say to automator that you want to ignore input from previous action.
Next we'll add two Get Value of Variable actions that will pick up the values of source and target variables:
Again, ignore the input from the Ask for Confirmation action.
After that, add Run Shell Script action which will tell rsync what to do:
You have to change "Pass input" to "as arguments" and than write rsync -va --delete "$1/" "$2/" $1 represents first variable that you're sending in, and $2 second one (first one being source_folder and second one target_folder). You can play with rsync options here and maybe exclude some files from syncing.
And here is another optional action, we'll add New TextEdit Document at the end. That will open new TextEdit window with output from Rsync, so you can see what Rsync actually did.
And that's it! Now we're going to save it, the best way I think is to save the workflow into your Scripts folder (you go to your home folder and there you'll find Scripts folder). I named mine workflow "RsyncTutorial". Then if you have Script Menu turned on in menu bar (If you don't you can enable it by running AppleScript Utility, located at /Applications/AppleScript/, and checking the "Show Script Menu in menu bar" checkbox) you'll see something like this when you click on the Script Menu:
And if you click on it your workflow will start by asking you for folder to sync from.
Comments
What a nice Workflow. Same people with brains gotten figure this out, well done.
Marvelous !
I was searching such tutorial a long time ago, thanks so much :-)
This works great! I was wondering if there is a way to set it up so that once you choose to run the script from the menu bar, it does not ask you to choose folders at all.
Sure, if you want to have it with predefined folders you delete everything from automator workflow except "Run Shell Script" and "New TextEdit Document" (if you want to see the output). And in "Run Shell Script" enter something like this:
rsync -va --delete "/path/to/folder1/" "/path/to/folder2/"
Great tutorial, thanks!
I too have been looking for some time for an easy solution for this. So thank you! However, I keep generating a shell script error. I started with exactly what was cited, leaving the values "$1/" and $2/" then tried replacing the text within the quotes to "Source_Folder" and "Target_Folder" respectively. Still an error. What am i doing wrong? Thank you!
Hi Eric. I'm not sure I understand it right. You should leave "$1/" and "$2/" in the run shell script window. It basically should look just as it is on the screenshot above. Folder names (paths) are passed as arguments to shell script ($1 is first argument we pass to it and $2 is second one) so these names we use in automator itself are irrelevant to bash script.
Hvala puno puno puno i pozdrav iz Pule!
Really nice tutorial, thank you very much
Hi, like Eric I keep getting a shell script error. The folder copies, but the contents do not. Help needed!
Hi, Nivesh, could you send me your automator workflow? If you send me your e-mail over the contact form I have on http://www.bananica.com/Contact/ I'll reply with mine, so you can send me the files and I'll take a look.
Hi there,
your idea of using automator is great!
Do you know if is possible to do a bydirectional syncronization? meaning a mirror sync where the most recent file version (no metter where is stored if in target or destination) is copyed to the other folder.....
That's what I'm looking for....
thanks
Luca
Hi Luca. Yes it's possible. You can read more about it here
Really awesome. Thanks for posting. Just a quick question though. With this script, the original "Source" will not be altered so only the "Target" will be altered?
Yes, that's right. Source folder is left intact.
Also my thanks, this will help me a lot.
Actually, my second "ask for finder items" always get TWO items. the first one again, and the second one. Even if I drag your first three commands into a new window, this happens, whereas it does NOT happen in your script. some hidden details?
oops read read directions ... the bit about ignore input ...
last comment. It might improve the tutorial if you say what the "ignore input" does even graphically. E.g. breaking the link between the above panel and the panel one is editing. Thanks again.
Awesome tutorial. Love the layout for your blog :D was super easy to follow and make sense of automator!
Thank you Aaron! Glad you liked it, and the layout as well, some find it weird with right/left alignments :)
Very clear tutorial. Thanks.
Is it possible to use this or something similar to do a bidirectional sync between two Macs over an network?
Thanks! Now I don't have to buy special software to do my backups. :-)
Is there an easy way to schedule running of the workflow in OS X Lion?
All my files and folders from source_code got lost. I can't find them anywhere, they didn't copy to destination_folder and they were deleted. Is there any way to recover that? Please help, im desperate.
Thanks - just what i needed. copies my scanned photos to an external disk once run.
This may be a bit confusing though on step no 8. I think should be on step number 7. 'Again, ignore the input from the Ask for Confirmation action.'
You were right! I changed it. Thanks!
Great tutorial, thanks! I am looking for a way to do this over a network between my laptop and my MacPro. I saw your tut on bidirectional synchronization on one machine, but is this possible between two machines? I've got about 8GB of files that I need to keep synced.
Thank you!
Jason A.
it is possible but this workflow won't support it without some editing. Quite frankly I haven't figured it out yet but for that I just manually do it in terminal or iTerm with the I.P. addy of the 2 computers.
Hi Jason!
I am myself syncing some folders on my laptop and desktop macs. I'm using rsync over ssh to do it. I'm gonna try to write a tutorial how to do it. There might be some other ways to do it, but this one works for me.
This tutorial is great, I h ave been looking for something like this.
However I have a question. Once the folder sync is set up, will it automatically then sync everything between the folders in the future. Without having to manually run it every time?
Thanks!
Hi there, this is an amazing idea, and was actually the first time I have ever used Automator; so go you for educating me there.
I was wondering if there was a way of making this script run permanently, so it updates the target from the source automatically?
i.e. I drop a file into 'documents' on my system drive, and it automatically backs up on my backup drive.
Thank you so much for this tutorial.
I tried the modified version of your instructions found on practically efficient dot com.
I have not used the terminal nor the Automator before, so I apologize in advance for my total lack of knowledge.
When I type this command in Terminal:
rsync -aE --delete ~/Desktop/testerfolder/ "/Volumes/My Passport/"
I am getting the error:
rsync: opendir “/Volumes/My Passport/.Trashes” failed: Permission denied (13)
rsync error: some files could not be transferred (code 23) at /SourceCache/rsync/rsync-42/rsync/main.c(992) [sender=2.6.9]
The files actually did copy though.
When I type this command in Terminal:
rsync -va --delete "~/Desktop/testfolder/" "Volumes/My Passport"
I get these error messages:
building file list ... rsync: link_stat "/Users/Jan/~/Desktop/testfolder/." failed: No such file or directory (2)
done
rsync: push_dir#3 "/Users/Jan/Volumes" failed: No such file or directory (2)
rsync error: errors selecting input/output files, dirs (code 3) at /SourceCache/rsync/rsync-42/rsync/main.c(580) [receiver=2.6.9]
rsync: connection unexpectedly closed (8 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at /SourceCache/rsync/rsync-42/rsync/io.c(452) [sender=2.6.9]
And no files are copied.
When I wrote and saved this in a RunShellScript in Automator:
rsync -va --delete "~/Desktop/testfolder/" "Volumes/My Passport/"
it didn’t work. I tried with both -va and -aE and with and without quotes around the source folder.
I got this message:
The action “Run Shell Script” encountered an error. Check the action’s properties and try running the workflow again.
Thanks again, so much.
I hope it isn't a bother to help me sort this out.
Hi Jan!
The problem with this one:
rsync -aE --delete ~/Desktop/testerfolder/ "/Volumes/My Passport/"
Is that you're syncing to root of your attached disk, and there are always some hidden folders in roots of hard disks (one is .Trashes for deleted files) and these are usually locked. And rsync is trying to delete .Trashes because there's no such folder in your testerfolder/ folder.
So I would suggest that you make a folder on you My Passport disk and then run it, for example:
rsync -aE --delete ~/Desktop/testerfolder/ "/Volumes/My Passport/test_target/"
Here:
rsync -va --delete "~/Desktop/testfolder/" "Volumes/My Passport"
you're missing "/" before Volumes, when you put it like this without "/" you're telling it to search for a folder Volumes in the folder you're currently in, so if you're in your home folder ("/Users/Jan/") it will look for /Users/Jan/Volumes/My Passport/ and there's probably no such folder.
And there's another problem. "~" is a shortcut for home folder, so instead of writing /Users/Jan/ you put just ~, but, if you use that shortcut than you can't put the whole path under quotes.
So the simple solution would be to create some folder on your My Passport disk and than sync it to that. Like I wrote above:
rsync -aE --delete ~/Desktop/testerfolder/ "/Volumes/My Passport/test_target/"
Btw. you need quotes around My Passport path because you have space in the volume name (My Passport), you could do without quotes, but than you have to escape space character like this:
/Volumes/My\ Passport/test_target/
Does not work here (Mac OS 10.6.5, MacBook Pro): Shell script seems to run in an endless loop.
Running the corresponding rsync script from the shell works flawlessly though.
May I send you my automator workflow for examination?
Does this also subfolders? if i have a subfolder with files in the source_folder, does it sync that to the target_folder?