/tag/hpc

  • Microservices

    Microservice architecture is an approach to designing and running applications. Such applications are typically run within containers, made popular in the last few years by Docker. Containers are portable, efficient, and disposable, and contain code and any dependencies in a single package. Containerized microservices typically run a single process, rather than an entire stack within the same computing environment. This allows portions of your application to be easily replaced or scaled as needed. Research Computing runs microservices in an orchestration environment named DCOS (Distributed Cloud Operating System), based on Apache Mesos and Apache Marathon. DCOS makes the deployment and management of many containers easy and scalable.
  • Rivanna Queues

    Several queues (or “partitions”) are availble to users for different types of jobs. One queue is restricted to single-node (serial or threaded) jobs; another for multinode parallel programs, and others are for access to specialty hardware such as large-memory nodes or nodes offering GPUs. Partition Max time per job Max nodes per job Max cores per job Max memory per core Max memory per node per job SU Charge Rate standard 7 days 1 40 9GB 375GB 1.00 parallel 3 days 45 900 9GB 120GB 1.00 largemem 4 days 1 16 64GB 975GB 1.
  • High Performance Computing

    Research Computing supports all UVA researchers who are interested in writing code to address their scientific inquiries. Whether these programming tasks are implemented interactively, in a series of scripts or as an open-source software package, services are available to provide guidance and enable collaborative development. RC has specific expertise in object oriented programming in Matlab, R, and Python. Examples of service areas include: Collaborating on package development Reviewing and debugging code Preparing scripts to automate or expedite tasks Developing web interfaces for interactive data exploration Advising on integration of existing software tools UVA has two local computational facilities available to researchers: Rivanna and Ivy.
  • Rivanna HPC Software

    Overview Research Computing at UVA offers a variety of standard software packages for all Rivanna users. We also install requested software based on the needs of the high-performance computing (HPC) community as a whole. Software used by a single group should be installed by that group’s members, ideally on leased storage controlled by the group. Departments with a set of widely-used software packages may install them to the lsp_apps space. The Research Computing group also provides limited assistance for individual installations. For help installing research software on your PC, please contact Research Software Support at res-consult@virginia.edu. Software Modules and Containers Software on Rivanna is accessed via environment modules or containers.
  • Frequently Asked Questions

    General Do you have a general computing question? Read our FAQ› Rivanna High Performance Computing Platform Read our FAQ › Ivy Secure Data Computing Platform Read our FAQ › Storage Research Data Storage & Transfer Read our FAQ ›
  • Rivanna FAQs

    General Usage Allocations Applications Job Management Storage Management Data Transfer Other Questions General Usage How do I gain access to Rivanna? A faculty or research staff member must first request an allocation on Rivanna. Full details can be found here. How do I log on to Rivanna? Use an SSH client from a campus-connected machine and connect to rivanna.hpc.virginia.edu. Instructions for using ssh and other login tools, as well as recommended clients for different operating systems, are here. You can also access Rivanna through our Web-based interface Open OnDemand or FastX. If you are off Grounds and connecting via SSH client or FastX you must use the UVA VPN.
  • Graphical SFTP/SCP Transfer Tools

    Several options are available to transfer data files between a local computer and Rivanna through user-friendly, graphical methods. Off Campus? Connecting to Rivanna from off Grounds via Secure Shell Access (SSH) or FastX requires a VPN connection. We recommend using the UVA More Secure Network if available. The UVA Anywhere VPN should only be used if the UVA More Secure Network is not available. Only Windows and Mac OSX operating systems are supported. Linux users should refer to these unsupported instructions to install and configure a VPN. Open OnDemand users do not need a VPN to access Rivanna. MobaXterm MobaXterm is a Windows application that combines an ssh client for logging in, a graphical secure-copy client for easy drag-and-drop file transfer, and an X11 server for displaying graphical applications.
  • Computing Environments at UVA

    Rivanna The primary vehicle for high-performance computing since 2014 has been the Rivanna cluster. Rivanna is a heterogenous system consisting of approximately 186 x 20-core nodes with 128GB of RAM each, 25 x 28-core nodes with 256 GB of RAM, and 152 x 40-core nodes with 384GB each. Five “big memory” nodes offer 1TB of RAM with 16 cores each. All nodes are supported by a high-performance FDR Infiniband network using Mellanox hardware and some have EDR Infiniband. A number of nodes supporting specialty hardware are included; there are 8 nodes with NVIDIA K80 GPGPUs, 4 nodes with NVIDIA V100 GPGPUs, 1 node with NVIDIA V100, and 2 nodes with NVIDIA RTX2080TI boards.
  • Computing Systems

    UVA Research Computing can help you find the right system for your computational workloads. From supercomputers to HIPAA secure systems to cloud-based deployments with advanced infrastructure, various systems are available to researchers. Are you submitting a grant proposal and need standard information about UVA research computing environments? Get it here. High Performance Computing - Rivanna A traditional high performance cluster with job scheduler, large file system, modules, and MPI processing. Get Started on Rivanna Secure Computing for Highly Sensitive Data - Ivy A multi-platform, HIPAA-compliant system for secure data that includes dedicated virtual machines (Linux and Windows), JupyterLab Notebooks, and Apache Spark.
  • Allocations

    function setCookie(key, value, expiry) { var expires = new Date(); expires.setTime(expires.getTime() + (expiry * 60 * 60 * 1000)); document.cookie = key + ‘=’ + value + ‘;expires=’ + expires.toUTCString() + ‘;path=/'; } function getCookie(key) { var keyValue = document.cookie.match('(^|;) ?’ + key + ‘=([^;]*)(;|$)'); return keyValue ? keyValue[2] : null; } function gen_random(length) { var text = “"; var possible = “ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789”; for (var i = 0; i var user_token = getCookie("__user_token”); Time on Rivanna is allocated as Service Units (SUs). One SU corresponds to one core-hour. Multiple SUs make up what is called an allocation (e.
  • Rivanna Storage

    There are a variety of options for storing large-scale research data at UVa. Public and moderately sensitive data storage systems can be accessed from the Rivanna high performance computing system. .tg {border-collapse:collapse;border-spacing:0;border-color:#ccc;} .tg td{font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:0px;overflow:hidden;word-break:normal;border-color:#ccc;color:#333;background-color:#fff;} .tg th{font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:0px;overflow:hidden;word-break:normal;border-color:#ccc;color:#333;background-color:#f0f0f0;} .tg .tg-hy9w{background-color:#eceeef;border-color:inherit;vertical-align:top} .tg .tg-dc35{background-color:#f9f9f9;border-color:inherit;vertical-align:top} .tg .tg-0qmj{font-weight:bold;background-color:#eceeef;border-color:inherit;vertical-align:top} Storage Directories Name Quota Price Data Protection Accessible from Best Practices /home 50GB Free 2-week snapshots Rivanna /home is best used as a working directory when using Rivanna interactively. SLURM jobs run against /home will be slower than those run against /scratch. The /home directory is a personal storage space that is not shareable with other users.
  • Workshops

    UVA Research Computing provides training opportunities covering a variety of data analysis, basic programming and computational topics. All of the classes listed below are taught by experts and are freely available to UVa faculty, staff and students. Upcoming Workshops Workshops for 2020 have been moved to online learning formats due to campus closures from COVID-19. DATE WORKSHOP INSTRUCTOR 09/03/2020, 09/10/2020, 09/17/2020, 09/24/2020 PhD Plus: Introduction to Programming in Python Katherine Holcomb 09/25/2020 Parallel Computing with Matlab Ed Hall 10/07/2020 Intro to Shiny Apps Christina Gancayco 10/09/2020 Data Science with Matlab Ed Hall 10/13/2020 Using Rivanna from the Command Line Gladys Andino 10/14/2020 Building Software Containers for Rivanna Ruoshi Sun 10/15/2020 Biopython Karsten Siller 10/21/2020 Customizing Shiny Apps Christina Gancayco 10/23/2020 Using the Julia Programming Language on Rivanna Ed Hall 10/24/2020, 11/01/2020 Tensorflow – Turning the Knobs (Overview and Q&A) Jacalyn Huband 11/12/2020 Automation of Image Processing with Fiji/ImageJ and OMERO Karsten Siller 11/17/2020 Using Bioinformatics Tools on Rivanna Gladys Andino Research Computing is partnering with the Research Library and the Health Sciences Library to deliver workshops covering a variety of research computing topics.
  • Rivanna

    Rivanna is the University of Virginia’s High-Performance Computing (HPC) system. As a centralized resource it has hundreds of pre-installed software packages available for computational research across many disciplines. Currently the Rivanna supercomputer has over 8,000 cores and 8PB of various storage. All UVA faculty, staff, and postdoctoral associates are eligible to use Rivanna, or students when part of faculty research. The sections below contain important information for new and existing Rivanna users. Please read each carefully. New users are invited to attend one of our free orientation sessions (“Introduction to the HPC System”) held throughout the year during office hours or by appointment.
  • FastX Web Portal

    Overview FastX is a commercial solution that enables users to start an X11 desktop environment on a remote system. It is available on the Rivanna frontends. Using it is equivalent to logging in at the console of the frontend. Using FastX for the Web We recommend that most users access FastX through its Web interface. To connect, point a browser to: https://rivanna-desktop.hpc.virginia.edu Off Campus? Connecting to Rivanna from off Grounds via Secure Shell Access (SSH) or FastX requires a VPN connection. We recommend using the UVA More Secure Network if available. The UVA Anywhere VPN should only be used if the UVA More Secure Network is not available.
  • Open OnDemand

    Overview Open OnDemand is a graphical user interface that allows access to Rivanna via a web browser. Within the Open OnDemand environment users have access to a file explorer; interactive applications like JupyterLab, RStudio Server & FastX Web; a command line interface; and a job composer and job monitor. Logging in to Rivanna Rivanna is accessible through the Open OnDemand web client at https://rivanna-portal.hpc.virginia.edu. Your login is your UVA computing ID and your password is your Netbadge password. Some services, such as FastX Web, require the Eservices password. If you do not know your Eservices password you must change it through ITS by changing your Netbadge password (see instructions).
  • Open OnDemand: File Explorer

    Open OnDemand provides an integrated file explorer to browse and manage small files. Rivanna has multiple locations to store your files with different limits and policies. Specifically, each user has a relatively small amount of permanent storage in his/her home directory and a large amount of temporary storage (/scratch) where large data sets can be staged for job processing. Researchers can also lease storage that is accessible on Rivanna. Contact Research Computing or visit the storage website for more information. The file explorer provides these basic functions: Renaming of files Viewing of text and small image files Editing text files Downloading & uploading small files To see the storage locations that you have access to from within Open OnDemand, click on the Files menu.
  • Open OnDemand: Job Composer

    Open OnDemand allows you to submit SLURM jobs to the cluster without using shell commands. The job composer simplifies the process of: Creating a script Submitting a job Downloading results Submitting Jobs We will describe creating a job from a template provided by the system. Open the Job Composer tab from the Open OnDemand Dashboard. Go to the New Job tab and from the dropdown, select From Template. You can choose the default template or you can select from the list. Click on Create New Job. You will need to edit the file that pops up, so click the light blue Open Editor button at the bottom.
  • Pulse Laser Irradiation and Surface Morphology

    Dr. Zhigilei and his team are using Rivanna to perform large-scale atomistic simulations aimed at revealing fundamental processes responsible for the modification of surface morphology and microstructure of metal targets treated by short pulse laser irradiation. The simulations are performed with a highly-optimized parallel computer code capable of reproducing collective dynamics in systems consisting of up to billions of atoms. As a result, the simulations naturally account for the complexity of the material response to the rapid laser energy deposition and provide clear visual representations, or “atomic movies,” of laser-induced dynamic processes. The mechanistic insights revealed in the simulations have an immediate impact on the development of the theoretical understanding of laser-induced processes and assist in optimization of laser processing parameters in current applications based on laser surface modification and nanoparticle generation in laser ablation.
  • Fluid Dynamics and Reef Health

    Professor Reidenbach and his team are using Rivanna to run computational fluid dynamics simulations of wave and tide driven flows over coral reefs in order to determine how storms, nutrient inputs, and sediments impact reef health. This is an image of dye fluxing from the surface of the Hawaiian coral Porites compressa utilizing a technique known as planar laser induced fluorescence (PLIF). Reefs such as this one have been severely impacted by human alteration, both locally through additional inputs of sediments and nutrients, and globally through increased sea surface temperatures caused by climate change. Reidenbach is hopeful that his computational models will allow scientists to better predict the future health of reefs based on human activity and improve global reef restoration efforts.
  • Pricing

    Below is a schedule of prices for Research Computing resources. Rivanna Allocations Type SU Limits Cost SU Lifetime Standard 100,000 per application; renewable (400K SUs max per fiscal year) Free 12 months Deans’ Allocations None Free 12 months by default, negotiable Purchased None $0.015 (<1M SUs); $0.01 (=1M SUs) Forever Instructional 25,000 Free 2 weeks after last teaching session ** Non-UVA personnel are charged at a rate of $0.07/SU About Allocations Storage Name Security Cost Project Standard $60 TB/year Value Standard $45 TB/year ZFS Standard $30 TB/year Ivy Central Storage High $45 TB/year Ivy NAS Storage High $60 TB/year Storage Details Request Storage
  • Economic Market Behavior

    While conducting research for a highly-technical study of market behavior, Dr. Ciliberto realized that he needed to parallelize an integration over a sample distribution. RC staff member Ed Hall successfully parallelized Ciliberto’s Matlab code and taught him how to do production runs on the University’s high-performance clusters. “The second stage estimator was computationally intensive,” Ciliberto recalls. “We needed to compute the distribution of the residuals and unobservables for multiple parameter values and at many different points of the distribution, which requires parallelizing the computation. Ed Hall’s expertise in this area was crucial. In fact, without Ed’s contribution, this project could not have been completed.
  • Tracking Bug Movements

    Ed Hall worked with the Brodie Lab in the Biology department, to set up a workflow to analyze videos of bug tracking experiments on the Rivanna Linux cluster. They wanted to use the community Matlab software (idTracker) for beetle movement tracking. Their two goals were to shorten the software runtime and to automate the process. There was a large backlog of videos to go through. Ed installed the idTracker software on Rivanna and modified the code to parallelize the bug tracking process. He wrote and documented shell scripts to automate their workflow on the cluster. PI: Edmund Brodie, PhD (Department of Biology)
  • Logging In

    Rivanna is accessible through a web portal, secure shell terminals, or a remote desktop environment. For of all of these access points, your login is your UVA computing ID and your password is your Eservices password. If you do not know your Eservices password you must change it through ITS. Off Campus? Connecting to Rivanna from off Grounds via Secure Shell Access (SSH) or FastX requires a VPN connection. We recommend using the UVA More Secure Network if available. The UVA Anywhere VPN should only be used if the UVA More Secure Network is not available. Only Windows and Mac OSX operating systems are supported.
  • MobaXterm

    MobaXterm is the recommended login tool for Windows users. It bundles a tabbed ssh client, a graphical drag-and-drop sftp client, and an X11 window server for Windows, all in one easy-to-use package. Some other tools included are a simple text editor with syntax coloring and several useful Unix utlities such as cd, ls, grep, and others, so that you can run a lightweight Linux environment on your local machine as well as use it to log in to a remote system. Download To download MobaXterm, click the link below. Select the “Home” version, “Installer” edition, Download MobaXterm Run the installer as directed.
  • SLURM Job Manager

    Overview Rivanna is a multi-user, managed environment. It is divided into frontends, which are directly accessible by users, and compute nodes, which must be accessed through the resource manager. We use the Simple Linux Utility for Resource Management (SLURM), an open-source tool that performs cluster management and job scheduling for Linux clusters. Jobs are submitted to the resource manager, which queues them until the system is ready to run them. SLURM selects which jobs to run, when to run them, and how to place them on the compute node, according to a predetermined site policy meant to balance competing user needs and to maximize efficient use of cluster resources.
  • Active Galactic Nuclei

    Some galaxies have an extremely energetic central region known as an Active Galactic Nucleus. These regions are among the brightest objects in the universe, often outshining all of the stars in their home galaxy combined. In at least some cases, the power source at the center of these extraordinary nuclei is actually a black hole; as gases are drawn toward the black hole, they spiral around it, generating gravitational energy that is converted into heat and electromagnetic waves. A simulation created by Prof. John Hawley (CLAS) with collaborators from Johns Hopkins University reveals this process at an unprecedented level of detail.
  • Quantifying Cerebral Cortex Regions

    A powerful new technique for quantifying regions of the cerebral cortex was developed by Nick Tustison and James Stone at the University of Virginia along with collaborators from the University of Pennsylvania. It was evaluated using large data sets comprised of magnetic resonance imaging (MRI) of the human brain processed on a high-performance computing cluster at the University of Virginia. By making this technique available as open-source software, other neuroscientists are now able to investigate various hypotheses concerning the relationship between brain structure and development. Tustison’s and Stone’s software has been widely disseminated and is being actively incorporated into a variety of clinical research studies, including a collaborative effort between the Department of Defense and Department of Veterans Affairs, exploring the long term effects of traumatic brain injury (TBI) among military service members.
  • Rivanna HPC Software

    Overview Research Computing at UVA offers a variety of standard software packages for all Rivanna users. We also install requested software based on the needs of the HPC community as a whole. Software used by a single group should be installed by that group’s members, ideally on leased storage controlled by the group. Departments with a set of widely-used software packages may install them to the lsp_apps space. The Research Computing group also provides limited assistance for individual installations. For help installing research software on your PC, please contact Research Software Support at res-consult@virginia.edu. Software Modules and Containers Software on Rivanna is provided via environment modules or as containers.
  • Rivanna Software List

    function searchFunction() { var input, filter, table, tr, td, i, txtValue; input = document.getElementById(“searchInput”); filter = input.value.toUpperCase(); table = document.getElementById(“moduleTable”); tr = table.getElementsByTagName(“tr”); for (i = 0; i -1) { tr[i].style.display = “"; } else { tr[i].style.display = “none”; } } } } Module Category Description R lang R is a free software environment for statistical computing and graphics. abinit chem ABINIT is a package whose main program allows one to find the total energy, charge density and electronic structure of systems made of electrons and nuclei (molecules and periodic solids) within Density Functional Theory (DFT), using pseudopotentials and a planewave or wavelet basis.
  • Software Modules

    The lmod modules system on Rivanna enables users to easily set their environments for selected software and to choose versions if appropriate. The lmod system is hierarchical; not every module is available in every environment. We provide a core environment which contains most of the software installed by Research Computing staff, but software that requires a compiler or MPI is not in that environment and a compiler must first be loaded. <button class="btn btn-primary"“>View All Modules Basic Commands List all available software in the core environment: module avail Use “module spider” to find all possible modules. module spider module spider hdf5 If a version is specified to spider, it will indicate how to load that version.
  • BART Web

    BART (Binding Analysis for Regulation of Transcription) Web Working with researchers in the Zang Lab in the Center for Public Health Genomics (CPHG), RC helped launch BARTweb, an interactive web-based tool for users to analyze their Genelist or ChIP-seq datasets. BARTweb is a containerized Flask front-end (written in Python) that ingests files and submits them to a more robust Python-based genomics pipeline running on Rivanna, UVA’s high performance computing cluster (HPC). This architecture – of a public web application that uses a supercomputer to process data – is a new model for UVA, and one that eases the learning curve for researchers who may not have access to an HPC system or the expertise to run a BART pipeline in the command-line.
  • Anaconda on Rivanna

    Overview Built to complement the rich, open source Python community, the Anaconda platform provides an enterprise-ready data analytics platform that empowers companies to adopt a modern open data science analytics architecture. Rivanna has Python 2 and 3 available as part of the Anaconda distribution. Anaconda comes installed with many packages best suited for scientific computing, data processing, and data analysis, while making deployment very simple. Its package manager conda installs and updates python packages and dependencies, keeping different package versions isolated on a project-by-project basis. Anaconda is available as open source under the New BSD license. It also ships with pip, the common python package manager.
  • What is Research Computing?

    UVA Research Computing (RC) is a new program that aims to support computational biomedical research by providing advanced cyberinfrastructure and expertise in data analysis at scale. Our mission is to foster a culture of computational thinking and promote interdisciplinary collaboration in various data-driven research domains. We offer services related to high performance computing, cloud architecture, scientific programming and big data solutions. We also aim to promote computationally intensive research at UVA through collaborative efforts such as UVA’s own CADRE (Computation And Data Resource Exchange) and XSEDE (Extreme Science and Engineering Discovery Environment). One of our driving philosophies is that researchers already have medical and scientific expertise, and should not have to become computing experts on top of that.
  • User Guides

    High Performance Computing Standard and high security HPC to run your code, generally written in R, Python or shell scripts. Get Started › Secure Computing Secure virtual machines and interactive notebooks for processing HIPAA and other highly sensitive data. Get Started › Storage Need large, or extremely large storage offsite or on grounds? Can you count in GB, TB, or PB? Learn more about storage options and pricing. Get Started › Cloud Have an idea you’d like to test? Need an environment provisioned in short-order? We can help you build in the AWS cloud.
  • Mission

    Research Computing is a support team at the University of Virginia whose mission is to empower researchers to achieve more through the use of cutting-edge computational resources. We strive to create innovative solutions for researchers who need help solving complex optimization, parallelization, workflow, and data analysis issues. Our strategy is to build and maintain the University’s best computing platforms while educating the next generation of researchers on the power and utility of advanced computing.
  • SSH on Rivanna

    SSH is the secure shell. It is the primary application used to access Rivanna from the command line. Connecting to a Remote Host For Windows, MobaXterm is our recommended ssh client; this package also provides an SFTP client and an X11 server in one bundle. Mac OSX and Linux users access the cluster from a terminal through OpenSSH, which are preinstalled on these operating systems. Open a terminal (on OSX, the Terminal application) and type ssh -Y mst3k@rivanna.hpc.virginia.edu where mst3k should be replaced by your user ID. You will generally need to use this format unless you set up your user account on your Mac or Linux system with your UVA ID.