Data Transfer

Efficient and reliable data transfer is a critical component of research computing. A variety of useful tools is available for rapid data transfer, whether you are transferring data from an external site or within different computing environments at UVA.

Selecting a Transfer Method

The data transfer method you choose heavily relies on the data sensitivity classification, where the data are currently located and to where you want to transfer the data. Click on a row in the table below to learn more about the data transfer methods available for a specific scenario.

System 1 System 2 Example Scenario

Lab Workstation
Storage for Public and Moderatively Sensitive Data
  • Copy data from a lab workstation to a /project storage share.

  • Copy result files from a /scratch directory on Rivanna.
Expand

Local Computer

  • Laptop
  • Lab Workstation

Remote System

Data Transfer Methods
System 1 System 2 Example Scenario

Secure Lab Workstation
Storage for Highly Sensitive Data
  • Transfer HIPAA data from a Health Systems workstation to Ivy Central Storage.

 

Expand

Sources

  • Lab Workstation
  • Health Systems Workstation

Destination

Data Transfer Methods


System 1 System 2 Example Scenario

External Institution
University of Virginia
  • Transfer sensitive data collected at another institution to an Ivy Central Storage share at UVA.

  • Transfer results from an analysis carried out on a remote supercomputer at a national lab or supercomputing center.
Expand

Sources

Any institution that uses Globus, such as:
  • Other universities
  • Supercomputing facilities

Destinations

Data Transfer Methods

Data Transfer Methods

Globus Connect

Large-scale research data transfer
Globus

Transferring large amounts of research data is fast and simple with Globus Connect. Globus gives researchers unified access to their data through an easy-to-use web interface, and can be used to transfer data between your laptop and storage systems mounted on the Rivanna HPC system. Globus can also be used to transfer data from other universities or supercomputing facilities.

 

For transferring highly sensitive data such as HIPAA or CUI data to the Ivy secure computing environment, researchers _must_ use the secure Globus data transfer node (DTN).

Graphical SFTP Clients

Graphical-SFTP
Secure file transfer protocol (SFTP)

Programs such as MobaXterm, Filezilla, and Cyberduck provide a graphical user interface to transfer data between a local computer and a remote storage location that permits scp or sftp. These applications allow drag-and-drop file manipulation.

Command Line Tools

Transferring Files from a Terminal

Researchers who are comfortable with the command line can use a variety of command line tools to transfer their data between their laptops and storage systems. Programs such as `scp`, `sftp`, and `rsync` can be used to quickly transfer files.

Local Data Transfer with the Command Line

When using a Linux file system, users can invoke generic Linux commands to manage files and directories (mv, cp, mkdir), manage permissions (chmod) and navigate the file system (cd, ls, pwd). If you or your collaborators are unfamiliar with some of these commands, we encourage you to take time to review some of the material below:

For more help, please feel free to contact RC staff to set up a consultation or visit us during office hours.

|  userinfo    , ,