I originally wrote most of this article while I was working in an IT shop at UCSB. I wanted to mirror it here as I no longer work in that shop and would be bummed to lose the article. You can find it here if you want.

If you’re not familiar with, LFTP it bills itself as a “sophisticated file transfer program”. It supports common protocols like FTP, HTTP/s, FXP, SFTP, and BitTorrent. It can also download multiple files in parallel, or download one file as multiple segments in parallel. If you’re on Linux, you should find LFTP in your distribution’s package repository. It’s also easily available on OS X via brew. If neither of those work for you, the LFTP site linked above contains source code and rpm binaries.

LFTP consists of a number of commands. LFTP will give you an interactive shell to call them, like an FTP or shell session. You can also pass commands directly to LFTP with no interactivity, more like rsync. Each of these commands does different things and takes different options. Some can be strung together to do great things, but unless you use LFTP frequently you’ll never remember them. That’s why I wrote down these common example use cases so you can get going quickly:

Connecting

  • Connect to an FTP server with authentication, get dropped into an LFTP shell:
      lftp -p21 username ftp://ftp.ucsb.edu
    
  • Connect to an SSH server, get dropped into an LFTP shell:
      lftp user@ucsb.edu
    


Downloading

Once you’re connected to the remote server, you can start pulling down files:

  • Download a file with resume support in 8 segments:
      lftp> pget -c -n 8 ./debian-6/debian-6.i386.iso
    

  • Download a directory with resume support, 4 concurrent files and 8 concurrent segments per file:
      lftp> mirror -c -P 4 --use-pget-n=8 ./debian-6
    

  • Download multiple files with a glob expression, resume, pget, and 12 segments:
      lftp> glob -- pget -c -n 12 ./debian-6/*
    

  • Download multiple directories with a glob, resume, pget for 12 segments, and mirror for 4 concurrent files:
      lftp> glob -d -- mirror -c -P 4 --use-pget-n=12 ./debian*
    


New Miscellaneous (June 2020)

  • Execute some commands and keep the shell open with -e flag:
      lftp -e "cd A; pget A.txt; cd ../B; pget B.txt" ftp://ftp.ucsb.edu
      lftp> 
    
  • Execute a script file with -f flag:
      lftp -f ./script.lftp
    

    Our script downloads remote_dir to local_dir with pget:

      # script.lftp
      open sftp://10.0.0.1
      mirror -c -P 2 --use-pget-n=4 ./remote_dir /mnt/local_dir
      quit