/tag/data-transfer
Globus Data Transfer
Globus is a simple, reliable, and fast way to access and move your research data between systems. Globus allows you to transfer data to and from systems such as:
Laptops & personal workstations Rivanna/Afton HPC clusters High-Security Research Standard Storage Lab / departmental storage Tape archives Cloud storage Off-campus resources (ACCESS, National Labs) Globus can help you share research data with colleagues and co-investigators, or to move data back and forth between a lab workstation and Rivanna/Afton or your personal computer.
Are your data stored at a different institution? At a supercomputing facility? All you need is your institution’s login credentials.
Scratch Recovery with Globus
Globus is a simple, reliable, and fast way to access and move your research data between systems. Researchers can transfer data from their old scratch directories to new scratch via a single web interface - no software download or installation is necessary.
Command Line Data Transfer
Standard Linux tools can efficiently transfer a small to moderate quantity of data to or from Rivanna/Afton.
scp scp uses the secure shell (SSH) protocol to transfer files between your local machine and a remote host, or between two remote hosts.
The following syntax enables copying from local to remote or vice versa. In both cases we are starting from the local system.
By default, scp works from the level of the directory in which it is invoked.
Copying from local to remote: scp source_file mst3k@hostaddress:target_file Copying from remote to local: scp mst3k@hostaddress:source_file target_file The following examples detail how to transfer data between your local computer and /project storage on Rivanna/Afton.
Cyberduck
Cyberduck is a transfer tool for Windows and Mac. It supports a large number of transfer targets and protocols. Only SFTP can be used with Rivanna/Afton. The free version will pop up donation requests.
Download Download Cyberduck
Connecting to the HPC System and File Transfer Launch Cyberduck. After launching Cyberduck, the user interface will open. To initiate a connection to UVA HPC, click the Open Connection button.
Enter Your Credentials. From the drop-down menu, select SFTP (SSH File Transfer Protocol). Then enter the appropriate information in the following fields:
Host: login.hpc.virginia.edu Username: your computing ID Password: your UVA HPC password Port: 22 When completed, click Connect.
Filezilla
Filezilla is a cross-platform data transfer tool. The free version supports FTP, FTPS, and SFTP. Only SFTP can be used with UVA HPC.
Download Download Filezilla
Connecting to the HPC System and File Transfer Launch FileZilla. After launching FileZilla, the user interface will open. In the left panel, you should see your local file system and files listed in the left side panels. You will enter your login credentials in the fields highlighted in the figure below.
Enter Your Credentials. Fill in the Host, Username, Password, and Port fields.
Host: login.hpc.virginia.edu Username: your computing ID Password: your Eservices password Port: 22 When completed, click Quickconnect.
Data Transfer
Efficient and reliable data transfer is a critical component of research computing. A variety of useful tools is available for rapid data transfer, whether you are transferring data from an external site or within different computing environments at UVA. Common Scenarios Transfer public or internal use data between local workstation/laptop and UVA storage Transfer sensitive or highly sensitive data to Ivy storage Transfer data between external institutions/supercomputing centers and UVA Transfer between UVA HPC and Cloud storage The data transfer method you choose heavily relies on the data sensitivity classification, where the data are currently located and to where you want to transfer the data.
Data Transfer
Data transfer Public & Moderately Sensitive Data Transfer Secure Copy (scp) scp uses secure shell (SSH) protocol to transfer files between your local machine and a remote host. scp can be used with the following syntax:
scp [source] [destination]
scp SourceFile mst3k@login.hpc.virginia.edu:/scratch/mst3k
scp SourceFile mst3k@login.hpc.virginia.edu:/project/Grouper_group_name
Detailed instructions and examples for using scp are listed here.
Secure File Transfer Protocol (sftp) sftp is a network protocol for secure file management. Instructions and examples for using sftp are located here.
Graphical File-Transfer Applications Filezilla and Cyberduck, and MobaXterm are examples of open source SFTP client software for file management through an interactive graphical user interface.
