lftp: multiprotocolfile transfer program

Contents

  1. Overview of package
  2. Overview of package
    1. General usage
  3. Availability of package by cluster
  4. General usage
  5. Using with UMD Box

Overview of package

General information about package
Package: lftp
Description: multiprotocolfile transfer program
For more information: https://lftp.tech/
Categories:
License: OpenSource (GPLv3)

General usage information

LFTP is a sophisticated file transfer program supporting a number of network protocols (ftp, http, sftp, fish, torrent).

This module will add the lftp and lftpget commands to your PATH.

In case you need to link against this library in your code, the following environmental variables have been defined:

  • \$LFTP_ROOT has been set to the root of the lftp installation
  • \$LFTP_LIBDIR points to the directory containing the libraries
  • \$LFTP_INCDIR points to the directory containing the header files

You will probably wish to use these by adding the following flags to your compilation command (e.g. to CFLAGS in your Makefile):

  • -I\$LFTP_INCDIR
and the following flags to your link command (e.g. LDFLAGS in your Makefile):
  • -L\$LFTP_LIBDIR -Wl,-rpath,\$LFTP_LIBDIR

Available versions of the package lftp, by cluster

This section lists the available versions of the package lftpon the different clusters.

Available versions of lftp on the Deepthought2 cluster (RHEL8)

Available versions of lftp on the Deepthought2 cluster (RHEL8)
Version Module tags CPU(s) optimized for GPU ready?
4.8.1 lftp/4.8.1 ivybridge Y

Available versions of lftp on the Deepthought2 cluster (RHEL6) [DEPRECATED]

Available versions of lftp on the Deepthought2 cluster (RHEL6) [DEPRECATED]
Version Module tags CPU(s) optimized for GPU ready?
4.8.4 lftp/4.8.4 x86_64 N

General Usage

Lftp operates rather similarly to the old command line ftp utility. You should do a module load lftp first, and then connect to a remote site using lftp -u REMOTE_USERNAME REMOTE_SITE. You will be prompted for a password if needed, and then you will enter the interactive lftp prompt. At this point you can enter any of the many commands supported. The help command shows a quick overview of all the commands, we will discuss a few of the most commonly used ones below: (in the following, local site means the system on which you invoked the lftp command, and remote site is the site you specified with with REMOTE_SITE argument to the lftp command):

  • help: will print out a short list of all available commands. You can use help CMD for some more information on a specific command CMD
  • ls: will list the files in the current directory on remote site, or specific files
  • cd DIR: will change the current directory on the remote site to DIR
  • lcd DIR: will change the current directory on the local site to DIR
  • get FILE: will download the file FILE from the remote site to the local site (using current directories on each if not otherwise specified)
  • mget FILE_PATTERN: like get, will will download all files matching FILE_PATTERN, which can contain wild cards (*, etc).
  • put FILE: will upload the file FILE from the local site to the remote site.
  • mput FILE_PATTERN: like put, but uploads all files matching the FILE_PATTERN.
  • mirror REMOTE_DIR LOCAL_DIR: this will mirror (i.e. copy/delete/etc files to make the target directory look like the source directory). This subcommand has lots of options controlling various aspects, including whether the source/target directories are the remote/local directories or the local/remote directories. I suggest reading the man page before using it.
  • quit: exits lftp

The following example shows some of the commands for accessing Box:

login-2:~> module load lftp
login-2:~> lftp -u USERNAME@umd.edu ftp.box.com
Password: Enter your Box external application password here
lftp USERNAME@umd.edu@ftp.box.com:~> help
    !                     (commands)
    alias [ []]             attach [PID]
    bookmark [SUBCMD]                    cache [SUBCMD]
    cat [-b]                      cd 
    chmod [OPTS] mode file...            close [-a]
    [re]cls [opts] [path/][pattern]      debug [OPTS] [|off]
    du [options]                   edit [OPTS] 
    exit [|bg]                     get [OPTS]  [-o ]
    glob [OPTS]               help []
    history -w file|-r file|-c|-l [cnt]  jobs [-v] []
    kill all|                    lcd 
    lftp [OPTS]                    ln [-s]  
    ls []                          mget [OPTS] 
    mirror [OPTS] [remote [local]]       mkdir [OPTS] 
    module name [args]                   more 
    mput [OPTS]                   mrm 
    mv                     mmv [OPTS]  
    [re]nlist []                   open [OPTS] 
    pget [OPTS]  [-o ]     put [OPTS]  [-o ]
    pwd [-p]                             queue [OPTS] []
    quote                           repeat [OPTS] [delay] [command]
    rm [-r] [-f]                  rmdir [-f] 
    scache []                set [OPT] [ []]
    site                       source 
lftp USERNAME@umd.edu@ftp.box.com:~> ls
drwx------   1 owner group            0 Apr  4 12:24 testDirectory
-rw-------   1 owner group           24 Apr  4 13:26 test.file
-rw-------   1 owner group           24 Apr  4 13:51 test.file2
-rw-------   1 owner group           23 Apr  4 13:51 test.file3
lftp USERNAME@umd.edu@ftp.box.com:~> mget test.file*
71 bytes transferred in 1 second (50 B/s)
Total 3 files transferred
lftp USERNAME@umd.edu@ftp.box.com:~> mput newdata.2019-03-*.csv
12 bytes transferred in 2 seconds (5 B/s)
Total 2 files transferred
lftp USERNAME@umd.edu@ftp.box.com:~> quit
login-2:~>

Using with UMD Box

One of the protocols supported by UMD Box is ftps, making lftp a good choice for connecting to box. We discuss a few aspects of using lftp with Box here. See the UMD Box service catalog entry for more information on the Box service, including restrictions on what data can be stored.

When connecting to UMD Box over the ftps protocol, the standard campus directory name/password single-sign-on cannot be used, so you will need to set up in Box a password for external applications. To do this:

  1. From a web browser on your desktop/laptop/etc, go to umd.box.com and log in using the UMD single sign-on (your standard UMD campus directory ID and password).
  2. In the upper right is an icon (usually either your image or initials) with a drop down menu; open the menu and select "Account Settings".
  3. Under the "Account" tab, there is a section called "Authentication". There should be a link to the right entitled "Create Password" (if you already created your external password, the link will be called "Change Password". You can use this if you forgot what your external password is).
  4. Click the "Create Password" link and enter your password for accessing Box. Use a different password than your main UMD and other passwords.
  5. Click on "Save Changes" near the top right.
  6. WARNING
    Do not re-use your main UMD directory single-sign-on or any other password for your UMD Box external application password. Re-use of passwords is a security risk, since if you reuse passwords and hackers compromise one site they can then leverage that to compromise additional sites.

    After your external application Box password is set up, you can then access Box with a command like lftp -u USERNAME@umd.edu ftp.box.com.