Persson Group Handbook
  • Persson Group Handbook
  • (Materials Project Handbook)
  • Getting Started
    • Before you Arrive
    • After you Arrive
    • Coming Onsite
    • Getting a Computer
    • Workspace Setup
    • Getting Settled
  • Group Policies
    • Meetings
    • Leave Time
    • Purchasing
    • Travel
    • Writing/Submitting Your Paper
    • Group Agreements
  • Computing
    • Setting up your computer
    • Software
    • High Performance Computing (HPC)
      • NERSC (Perlmutter)
        • Conda Environments
        • Frequently encountered blockers
      • BRC (Savio)
      • LRC (Lawrencium)
      • NREL (Kestrel)
    • Simulation Codes
      • VASP
      • Q-Chem
      • Gaussian
      • OpenMM
      • LAMMPS
      • GROMACS
    • Fireworks & Atomate
      • Setup
      • Fireworks: Advanced Usage
    • Machine Learning (ML)
  • Group Administration
    • Group Meeting Schedule
    • Group Jobs (Fall 2024)
    • Physical Resources (printers, etc.)
    • Undergraduate Research Mentoring
    • Contributing to Handbook
    • Group Roster
  • Group resources
    • Tutorials and Helpful Links
    • Group Member Expertise
    • Professional development
    • Health and wellbeing
  • Graduate Students
    • UAW 4811 Contracts
    • Research Credits (298/299)
    • Course Recommendations
    • How to Qual (MSE)
    • How to Graduate
Powered by GitBook
On this page
  • Setting up NREL account
  • Logging on
  • Conda Environment Setup
Export as PDF
  1. Computing
  2. High Performance Computing (HPC)

NREL (Kestrel)

This page describes how to get setup at the National Renewable Energy Laboratory (NREL) on Kestrel.

PreviousLRC (Lawrencium)NextSimulation Codes

Last updated 8 months ago

If you encounter an error with NREL, there is most likely documentation on .

Setting up NREL account

Visit website to get started on making a user account. You will need to setup a MFA before proceeding.

Send an email to HPC-Help@nrel.gov to get added on the correct projects. Ask Kristin or the NREL coordinators of the group to see which projects you belong to.

Logging on

For the purposes of this example, assume your NREL account username is: johndoe

Kestrel login:

ssh johndoe@kestrel.nrel.gov

Once in, the command line will prompt you to type in your account password+OTP.

General NREL HPC login:

ssh johndoe@hpcsh.nrel.gov

Use this for loggin into the NREL HPC systems. From there, you can connect to any other NREL HPC system like Swift, Eagle, etc.

Conda Environment Setup

I personally would recommend setting up miniconda instead of using their default conda modules. If you would like to use their default conda module, execute module load conda and proceed to the conda install steps.

Miniconda

cd ~/ #ensures you start off in your home dir

mkdir -p ~/miniconda3
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.sh
bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3
rm ~/miniconda3/miniconda.sh

~/miniconda3/bin/conda init bash
~/miniconda3/bin/conda init zsh

source ~/.bashrc

You should now see a (base) or (miniconda) on the lefthand side of your command prompt.

Setting up your conda environment

Warning: This is different for every user. Please check with your mentor before proceeding.

The following conda setup is designed for VASP users who will be using atomate2 with fireworks.

conda create -n cms python=3.9 pandas seaborn numpy scipy matplotlib
conda activate cms 

pip install git+https://github.com/materialsproject/atomate2.git #replace with your desired way to downloading atomate2
pip install fireworks
pip install pydantic==2.4.2 #to avoid some pydantic bugs with fireworks...

In addition, also setup these two additional files:

atomate2.yaml

VASP_CMD: srun -n 104 -c 1 vasp_std 
VASP_GAMMA_CMD: srun -n 104 -c 1 vasp_gam
VASP_CMD: srun -n 104 -c 1 --cpu-bind=cores --gpu-bind=single:1 -G 4 vasp_std
VASP_GAMMA_CMD: srun -n 104 -c 1 --cpu-bind=cores --gpu-bind=single:1 -G 4 vasp_std
VASP_CMD: srun -n 104 -c 2 vasp_std 
VASP_GAMMA_CMD: srun -n 104 -c 2 vasp_gam

jobflow.yaml

Please use trajectory store version if you need to run AIMD/MD type calculations with atomate2

JOB_STORE:
  docs_store:
    type: MongoStore
    database: johndoe_general
    host: mongodb03.nersc.gov
    port: 27017
    username: johndoe_general_admin
    password: ***PASSWORD***
    collection_name: kestrel_outputs
  additional_stores:
    data:
      type: GridFSStore
      database: johndoe_general
      host: mongodb03.nersc.gov
      port: 27017
      username: johndoe_general_admin
      password: ***PASSWORD***
      collection_name: kestrel_outputs_blobs
JOB_STORE:
  docs_store:
    type: MongoStore
    database: johndoe_general
    host: mongodb03.nersc.gov
    port: 27017
    username: johndoe_general_admin
    password: ***PASSWORD***
    collection_name: kestrel_outputs
  additional_stores:
    data:
      type: GridFSStore
      database: johndoe_general
      host: mongodb03.nersc.gov
      port: 27017
      username: johndoe_general_admin
      password: ***PASSWORD***
      collection_name: kestrel_outputs_blobs
    trajectory:
      type: GridFSStore
      database: johndoe_general_admin
      host: mongodb03.nersc.gov
      port: 27017
      username: johndoe_general_admin
      password: s***PASSWORD***
      collection_name: kestrel_trajectory_blobs

You can find these install instructions on

Please refer to for setting up the fireworks yaml files and the fw_config directory.

https://nrel.github.io/HPC/Documentation/Systems/Kestrel/
https://www.nrel.gov/hpc/user-accounts.html
https://docs.anaconda.com/miniconda/#quick-command-line-install
https://materialsproject.gitbook.io/persson-group-handbook/computing/atomate-setup/setup#set-up-an-environment