High Performance Computing (HPC)
This section and its subpages describe how to set up and run calculations on various computing resources that our group has access to, as well as how to backup your data.
Last updated
This section and its subpages describe how to set up and run calculations on various computing resources that our group has access to, as well as how to backup your data.
Last updated
Our group’s main computing resources are:
(the LBNL supercomputing center, one of the biggest in the world)
/ Berkeley Research Computing (BRC)
/ Berkeley Lab Research Computing (LRC)
(the NREL supercomputing center)
Argonne Leadership Computing Facility (sometimes)
Oak Ridge Leadership Computing Facility (sometimes)
At any time, if you feel you are computing-limited, please contact Kristin so she can work with you on finding solutions.
You should back-up your Mongo DB data frequently. The Mongo DB NERSC offers is not backed-up automatically. It's important to run regular backups during the course of your research. For Mongo DB you can:
use the “mongodump” command line tool, tutorials available online — it’s a one line command
module load mongodb; mongodump --host=<host e.g. mongo01.nersc.gov> --port=27017 --username=<username> --password="<password>" --db="<db_name>" --authenticationDatabase="<db_name (same as --db flag)>" --out="<path to directory>"
You should back-up your "scratch" directory data frequently.
Each cluster has different methods of deleting old data on your scratch directory:
You can use one of the following popular tools for backing up raw data in our group:
Both of these tools can help backup your raw calculation files to your Google Drive, or Box Drive, or even external hard drives.
Other Persson group members and the NERSC website are both excellent resources for getting additional help. If that fails, you can reach out to the NERSC Operations staff:
1-800-666-3772 (or 1-510-486-8600)
Computer Operations = menu option 1 (24/7)
Account Support = menu option 2, accounts@nersc.gov
HPC Consulting = menu option 3, or consult@nersc.gov
Special thanks to the original authors of this page: Kara Fong, Eric Sivonxay, and John Dagdelen
Get a free education license for , right click your database, and click export or,
Savio deletes data older than
Lawrencium does
Cori has a
For NERSC users, the data archive system is an excellent resource for putting your data long-term. There are command line tools for you to directly compress your large directory and send to HPSS in your scratch nodes. Alternatively, you could also use Globus (listed below) to transfer your data, but note that you have to compress your folders first rather than moving thousands of small documents over. This could be an alternatvie to moving data to /project directory under matgen nodes, because the latter would use up lots of inodes and NERSC will complain. The only downside is that others couldn't access your easily in HPSS.