Efficient and reliable data transfer is a critical component of research computing. A variety of useful tools is available for rapid data transfer, whether you are transferring data from an external site or within different computing environments at UVA.
Selecting a Transfer Method
The data transfer method you choose heavily relies on the data sensitivity classification, where the data are currently located and to where you want to transfer the data. Click on a row in the table below to learn more about the data transfer methods available for a specific scenario.
System 1
System 2
Example Scenario
Lab Workstation
Storage for Public and Moderatively Sensitive Data
Copy data from a lab workstation to a /project storage share.
Copy result files from a /scratch directory on Rivanna.
</td>
<td>
<p align="center"><b>Data Transfer Methods</b></p>
<ul>
<li><a href="#globus">Globus Connect</a></li>
<li><a href="#sftp">Graphical SFTP Clients</a></li>
<li><a href="#command-line">Command Line Tools</a></li>
</ul>
</td>
</tr>
<tr data-toggle="collapse" data-target="#accordion2" class="clickable">
<td align="center" vertical-align="middle">
<i class="fas fa-laptop fa-4x" aria-hidden="true" style="padding:20px 5px;"></i>
<br></br>
<b>Secure Lab Workstation</b>
</td>
<td align="center" vertical-align="middle">
<i class="fas fa-lock fa-4x" aria-hidden="true" style="padding:20px 5px;"></i>
<br></br>
<b>Storage for Highly Sensitive Data</b>
</td>
<td vertical-align="middle" style="padding:25px 10px">
<ul>
<li>Transfer HIPAA data from a Health Systems workstation to Ivy Central Storage.</li>
</ul>
<p> </p>
<span style="float:right;font-size:85%;margin-bottom:-8px;"><i class="fas fa-angle-double-down"></i> Expand</span>
</td>
</tr>
<tr id="accordion2" class="collapse">
<td>
<p align="center"><b>Sources</b></p>
<ul>
<li>Lab Workstation</li>
<li>Health Systems Workstation</li>
</ul>
</td>
<td>
<p align="center"><b>Destination</b></p>
<ul>
<li><a href="/userinfo/storage/sensitive-data/#ivy-central-storage/">Ivy Central Storage</a></li>
</ul>
</td>
<td>
<p align="center"><b>Data Transfer Methods</b></p>
<ul>
<li><a href="#globus">Globus Connect</a></li>
</ul>
</td>
</tr>
<tr data-toggle="collapse" data-target="#accordion3" class="clickable">
<td align="center" vertical-align="middle">
<i class="fas fa-building fa-4x" aria-hidden="true" style="padding:20px 5px;"></i>
<br></br>
<b>External Institution</b>
</td>
<td align="center" vertical-align="middle">
<i class="fas fa-home fa-4x" aria-hidden="true" style="padding:20px 5px;"></i>
<br></br>
<b>University of Virginia</b>
</td>
<td vertical-align="middle" style="padding:25px 10px">
<ul>
<li>Transfer sensitive data collected at another institution to an Ivy Central Storage share at UVA.</li>
<br>
<li>Transfer results from an analysis carried out on a remote supercomputer at a national lab or supercomputing center.</li>
</ul>
<span style="float:right;font-size:85%;margin-bottom:-8px;"><i class="fas fa-angle-double-down"></i> Expand</span>
</td>
</tr>
<tr id="accordion3" class="collapse">
<td>
<p align="center"><b>Sources</b></p>
Any institution that uses Globus, such as:
<ul>
<li>Other universities</li>
<li>Supercomputing facilities</li>
</ul>
</td>
<td>
<p align="center"><b>Destinations</b></p>
<ul>
<li><a href="/userinfo/storage/non-sensitive-data/#home/">/home</a></li>
<li><a href="/userinfo/storage/non-sensitive-data/#scratch/">/scratch</a></li>
<li><a href="/userinfo/storage/non-sensitive-data/#project/">/project</a></li>
<li><a href="/userinfo/storage/research-value/">Value Storage</a></li>
<li><a href="/userinfo/storage/sensitive-data/#ivy-central-storage/">Ivy Central Storage</a></li>
</ul>
</td>
<td>
<p align="center"><b>Data Transfer Methods</b></p>
<ul>
<li><a href="#globus">Globus Connect</a></li>
</ul>
</td>
</tr>
</tbody>
Data Transfer Methods
Globus Connect
Large-scale research data transfer
Transferring large amounts of research data is fast and simple with Globus Connect. Globus gives researchers unified access to their data through an easy-to-use web interface, and can be used to transfer data between your laptop and storage systems mounted on the Rivanna HPC system. Globus can also be used to transfer data from other universities or supercomputing facilities.
For transferring highly sensitive data such as HIPAA or CUI data to the Ivy secure computing environment, researchers _must_ use the secure Globus data transfer node (DTN).
Graphical SFTP Clients
Secure file transfer protocol (SFTP)
Programs such as MobaXterm, Filezilla, and Cyberduck provide a graphical user interface to transfer data between a local computer and a remote storage location that permits scp or sftp. These applications allow drag-and-drop file manipulation.
Command Line Tools
Transferring Files from a Terminal
Researchers who are comfortable with the command line can use a variety of command line tools to transfer their data between their laptops and storage systems. Programs such as `scp`, `sftp`, and `rsync` can be used to quickly transfer files.
Local Data Transfer with the Command Line
When using a Linux file system, users can invoke generic Linux commands to manage files and directories (mv, cp, mkdir), manage permissions (chmod) and navigate the file system (cd, ls, pwd). If you or your collaborators are unfamiliar with some of these commands, we encourage you to take time to review some of the material below:
