# NREL (Kestrel)

If you encounter an error with NREL, there is most likely documentation on <https://nrel.github.io/HPC/Documentation/Systems/Kestrel/>.

## Setting up NREL account

Visit website <https://www.nrel.gov/hpc/user-accounts.html> to get started on making a user account. You will need to setup a MFA before proceeding.

Send an email to <HPC-Help@nrel.gov> to get added on the correct projects. Ask Kristin or the NREL coordinators of the group to see which projects you belong to.&#x20;

## Logging on

For the purposes of this example, assume your NREL account username is: `johndoe`

#### Kestrel login:

```
ssh johndoe@kestrel.nrel.gov
```

Once in, the command line will prompt you to type in your account password+OTP.&#x20;

#### General NREL HPC login:&#x20;

```
ssh johndoe@hpcsh.nrel.gov
```

Use this for loggin into the NREL HPC systems. From there, you can connect to any other NREL HPC system like Swift, Eagle, etc.&#x20;

## Conda Environment Setup&#x20;

I personally would recommend setting up miniconda instead of using their default conda modules. If you would like to use their default conda module, execute `module load conda` and proceed to the conda install steps.&#x20;

#### Miniconda

You can find these install instructions on <https://docs.anaconda.com/miniconda/#quick-command-line-install>

```
cd ~/ #ensures you start off in your home dir

mkdir -p ~/miniconda3
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.sh
bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3
rm ~/miniconda3/miniconda.sh

~/miniconda3/bin/conda init bash
~/miniconda3/bin/conda init zsh

source ~/.bashrc
```

You should now see a `(base)` or `(miniconda)` on the lefthand side of your command prompt.&#x20;

#### Setting up your conda environment

Warning: This is different for every user. Please check with your mentor before proceeding.&#x20;

The following conda setup is designed for VASP users who will be using atomate2 with fireworks.&#x20;

```
conda create -n cms python=3.9 pandas seaborn numpy scipy matplotlib
conda activate cms 

pip install git+https://github.com/materialsproject/atomate2.git #replace with your desired way to downloading atomate2
pip install fireworks
pip install pydantic==2.4.2 #to avoid some pydantic bugs with fireworks...
```

Please refer to <https://materialsproject.gitbook.io/persson-group-handbook/computing/atomate-setup/setup#set-up-an-environment> for setting up the fireworks yaml files and the fw\_config directory.&#x20;

In addition, also setup these two additional files:&#x20;

#### atomate2.yaml

{% tabs %}
{% tab title="Kestre CPU" %}

```
VASP_CMD: srun -n 104 -c 1 vasp_std 
VASP_GAMMA_CMD: srun -n 104 -c 1 vasp_gam
```

{% endtab %}

{% tab title="Kestrel GPU" %}

```
VASP_CMD: srun -n 104 -c 1 --cpu-bind=cores --gpu-bind=single:1 -G 4 vasp_std
VASP_GAMMA_CMD: srun -n 104 -c 1 --cpu-bind=cores --gpu-bind=single:1 -G 4 vasp_std
```

{% endtab %}

{% tab title="Kestrel CPU 2 Nodes" %}

```
VASP_CMD: srun -n 104 -c 2 vasp_std 
VASP_GAMMA_CMD: srun -n 104 -c 2 vasp_gam
```

{% endtab %}
{% endtabs %}

#### jobflow\.yaml

Please use trajectory store version if you need to run AIMD/MD type calculations with atomate2

{% tabs %}
{% tab title="General" %}

```
JOB_STORE:
  docs_store:
    type: MongoStore
    database: johndoe_general
    host: mongodb03.nersc.gov
    port: 27017
    username: johndoe_general_admin
    password: ***PASSWORD***
    collection_name: kestrel_outputs
  additional_stores:
    data:
      type: GridFSStore
      database: johndoe_general
      host: mongodb03.nersc.gov
      port: 27017
      username: johndoe_general_admin
      password: ***PASSWORD***
      collection_name: kestrel_outputs_blobs
```

{% endtab %}

{% tab title="With trajectory store" %}

```
JOB_STORE:
  docs_store:
    type: MongoStore
    database: johndoe_general
    host: mongodb03.nersc.gov
    port: 27017
    username: johndoe_general_admin
    password: ***PASSWORD***
    collection_name: kestrel_outputs
  additional_stores:
    data:
      type: GridFSStore
      database: johndoe_general
      host: mongodb03.nersc.gov
      port: 27017
      username: johndoe_general_admin
      password: ***PASSWORD***
      collection_name: kestrel_outputs_blobs
    trajectory:
      type: GridFSStore
      database: johndoe_general_admin
      host: mongodb03.nersc.gov
      port: 27017
      username: johndoe_general_admin
      password: s***PASSWORD***
      collection_name: kestrel_trajectory_blobs
```

{% endtab %}
{% endtabs %}


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://materialsproject.gitbook.io/persson-group-handbook/computing/high-performance-computing/nrel-kestrel.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.