High Performance Computing (HPC)
This section and its subpages describe how to set up and run calculations on various computing resources that our group has access to, as well as how to backup your data.
Our group’s main computing resources are:
- Argonne Leadership Computing Facility (sometimes)
- Oak Ridge Leadership Computing Facility (sometimes)
At any time, if you feel you are computing-limited, please contact Kristin so she can work with you on finding solutions.
You should back-up your Mongo DB data frequently. The Mongo DB NERSC offers is not backed-up automatically. It's important to run regular backups during the course of your research. For Mongo DB you can:
- use the “mongodump” command line tool, tutorials available online — it’s a one line command
module load mongodb; mongodump --host=<host e.g. mongo01.nersc.gov> --port=27017 --username=<username> --password="<password>" --db="<db_name>" --authenticationDatabase="<db_name (same as --db flag)>" --out="<path to directory>"You should back-up your "scratch" directory data frequently.
Each cluster has different methods of deleting old data on your scratch directory:
For NERSC users, the data archive system HPSS is an excellent resource for putting your data long-term. There are command line tools for you to directly compress your large directory and send to HPSS in your scratch nodes. Alternatively, you could also use Globus (listed below) to transfer your data, but note that you have to compress your folders first rather than moving thousands of small documents over. This could be an alternatvie to moving data to /project directory under matgen nodes, because the latter would use up lots of
inodes and NERSC will complain. The only downside is that others couldn't access your easily in HPSS. You can use one of the following popular tools for backing up raw data in our group:
Both of these tools can help backup your raw calculation files to your Google Drive, or Box Drive, or even external hard drives.
Other Persson group members and the NERSC website are both excellent resources for getting additional help. If that fails, you can reach out to the NERSC Operations staff:
- 1-800-666-3772 (or 1-510-486-8600)
- Computer Operations = menu option 1 (24/7)
- Account Support = menu option 2, [email protected]
- HPC Consulting = menu option 3, or [email protected]
Special thanks to the original authors of this page: Kara Fong, Eric Sivonxay, and John Dagdelen
Last modified 4mo ago