ftp, how to mirror many websites via ftp with just one great program
posted on TUTORIAL
website mirror via ftp, lftp, scheduled ftp, multi website mirror ftp
maybe you have this question in mind,
how do i backup (mirror) a list of websites, automatically and maybe scheduled, via ftp ?
imagine, you have no ssh, no direct server control, no other better way but FTP, to mirror one or hundreds of websites,
i figured it out, to my needs, and after having tried core ftp pro (paid version), the one program for windows that has a command line ftp, that can easily be batched to your needs, and having seen that the performance are too poor, (? maybe command line has no parallel file download ?), i tried one other solution, open source solution, LFTP.
LFTP is a great ftp client, with tons of features for your ftp work, but it needs linux, or you can do in windows with CYGWIN.
so those among you that use succesfully windows xp or any windows server and want a serious way to backup websites the linux way can count on LFTP for cygwin.
so first you have to get cygwin, latest build, and install, then on the package selection search for LFTP and add to the installation, install et voilà you have your environment ready to be used.
now just start cygwin and you can start to use LFTP
LFTP is a powerful command line ftp, so you have plenty of commands to explore in the man page, but i want to show you the easy way to mirror you website and have a fresh backup of your precious work.
let's start with the syntax
lftp -u username,password -e "mirror --parallel=10 --verbose /websitefolder /cygdrive/c/backup/website1" hostname
with this command, replacing username, password, hostname with your data, you just create a mirror copy of your ftp in your local directory, with downloading 10 files in parallel, so you can enjoy maximum speed and full use of your dsl line.
choosing how many parallel files to download at the same time, is a personal preference based on your line speed and what kind of files you have on your sites, if for example in the ftp you are full of small files then it's ok to have a high value for parallel file, if you have big files then you can choose less.
so now, how to do if you have a dozen website or more ?, very simple first you prepare a file with one command per line "backupftp.sh" is fine
then make sure to use this string with ;quit command to permit ftp to close connection on each command in the file
lftp -u username1,password1 -e "mirror --parallel=10 --verbose /websitefolder /cygdrive/c/backup/website1;quit" hostname1
lftp -u username2,password2 -e "mirror --parallel=10 --verbose /websitefolder /cygdrive/c/backup/website2;quit" hostname2
lftp -u username3,password3 -e "mirror --parallel=10 --verbose /websitefolder /cygdrive/c/backup/website3;quit" hostname3
then when ready save the file in the /bin folder and just run on the consolle of cygwin "backupftp.sh"
for scheduling the backup, you can use cron and setup as you wish, per day, per week or multiple. here a good cron tutorial