Persson Group Handbook
  • Persson Group Handbook
  • (Materials Project Handbook)
  • Getting Started
    • Before you Arrive
    • After you Arrive
    • Coming Onsite
    • Getting a Computer
    • Workspace Setup
    • Getting Settled
  • Group Policies
    • Meetings
    • Leave Time
    • Purchasing
    • Travel
    • Writing/Submitting Your Paper
    • Group Agreements
  • Computing
    • Setting up your computer
    • Software
    • High Performance Computing (HPC)
      • NERSC (Perlmutter)
        • Conda Environments
        • Frequently encountered blockers
      • BRC (Savio)
      • LRC (Lawrencium)
      • NREL (Kestrel)
    • Simulation Codes
      • VASP
      • Q-Chem
      • Gaussian
      • OpenMM
      • LAMMPS
      • GROMACS
    • Fireworks & Atomate
      • Setup
      • Fireworks: Advanced Usage
    • Machine Learning (ML)
  • Group Administration
    • Group Meeting Schedule
    • Group Jobs (Fall 2024)
    • Physical Resources (printers, etc.)
    • Undergraduate Research Mentoring
    • Contributing to Handbook
    • Group Roster
  • Group resources
    • Tutorials and Helpful Links
    • Group Member Expertise
    • Professional development
    • Health and wellbeing
  • Graduate Students
    • UAW 4811 Contracts
    • Research Credits (298/299)
    • Course Recommendations
    • How to Qual (MSE)
    • How to Graduate
Powered by GitBook
On this page
  • Overview
  • Setup Guides (see subpages)
  • Back-Up Data Frequently
  • Mongo DB
  • NERSC Cori, BRC Savio, LRC Lawrencium
  • Additional resources
Export as PDF
  1. Computing

High Performance Computing (HPC)

This section and its subpages describe how to set up and run calculations on various computing resources that our group has access to, as well as how to backup your data.

PreviousSoftwareNextNERSC (Perlmutter)

Last updated 8 months ago

Overview

Our group’s main computing resources are:

  • (the LBNL supercomputing center, one of the biggest in the world)

  • / Berkeley Research Computing (BRC)

  • / Berkeley Lab Research Computing (LRC)

  • (the NREL supercomputing center)

  • Argonne Leadership Computing Facility (sometimes)

  • Oak Ridge Leadership Computing Facility (sometimes)

At any time, if you feel you are computing-limited, please contact Kristin so she can work with you on finding solutions.

Setup Guides (see subpages)

Back-Up Data Frequently

Mongo DB

You should back-up your Mongo DB data frequently. The Mongo DB NERSC offers is not backed-up automatically. It's important to run regular backups during the course of your research. For Mongo DB you can:

  • use the “mongodump” command line tool, tutorials available online — it’s a one line command

module load mongodb; mongodump --host=<host e.g. mongo01.nersc.gov> --port=27017 --username=<username> --password="<password>" --db="<db_name>" --authenticationDatabase="<db_name (same as --db flag)>" --out="<path to directory>"

NERSC Cori, BRC Savio, LRC Lawrencium

You should back-up your "scratch" directory data frequently.

Each cluster has different methods of deleting old data on your scratch directory:

You can use one of the following popular tools for backing up raw data in our group:

Both of these tools can help backup your raw calculation files to your Google Drive, or Box Drive, or even external hard drives.

Additional resources

Other Persson group members and the NERSC website are both excellent resources for getting additional help. If that fails, you can reach out to the NERSC Operations staff:

  • 1-800-666-3772 (or 1-510-486-8600)

  • Computer Operations = menu option 1 (24/7)

  • Account Support = menu option 2, accounts@nersc.gov

  • HPC Consulting = menu option 3, or consult@nersc.gov

Special thanks to the original authors of this page: Kara Fong, Eric Sivonxay, and John Dagdelen

Get a free education license for , right click your database, and click export or,

Savio deletes data older than

Lawrencium does

Cori has a

For NERSC users, the data archive system is an excellent resource for putting your data long-term. There are command line tools for you to directly compress your large directory and send to HPSS in your scratch nodes. Alternatively, you could also use Globus (listed below) to transfer your data, but note that you have to compress your folders first rather than moving thousands of small documents over. This could be an alternatvie to moving data to /project directory under matgen nodes, because the latter would use up lots of inodes and NERSC will complain. The only downside is that others couldn't access your easily in HPSS.

NERSC
Savio
Lawrencium
Peregrine
NERSC (Perlmutter)
BRC (Savio)
LRC (Lawrencium)
Studio 3T
6 months
not back-up the scratch directory but backs up other types of directories
purging policy on the scratch directory
HPSS
Rclone
Globus