arrow-left

All pages
gitbookPowered by GitBook
1 of 7

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Gaussian

Gaussian is available on Savio. Anyone wanting to use Gaussian on Savio must sign . Print out two copies of the form and sign both (i.e. they should not be signed electronically). Then provide both physical copies to Alice. If you are remote, an e-mailed copy will suffice for the time being.

Exhibit Aarrow-up-right

Q-Chem

More information to come

Talk to Sam Blau, Ryan, or Evan if you have Q-Chem questions!

hashtag
Lawrencium

module load /clusterfs/mp/modules/qchem/5.4.2

Cori

Coming soon!

Perlmutter

Coming soon!

LAMMPS

Talk to Rishabh, Jingyang, or Ryan if you have LAMMPS questions!

More Information to come

hashtag
Creating a LAMMPS data file with relevant force field parameters

  1. Create the molecule in Avogadro or wherever you want to

  2. Then use antechamber to get the partial charges. Antechamber is part of AmberTools. Installation instructions are available in their

  3. The relevant command:- antechamber -fi pdb -fo prepi -i <infile.pdb> -o output.prepi -c bcc -j 4 -at gaff -nc -1 The -nc flag is the net charge in the molecule. It will error out if not invoked for ions.

    Sometimes it might be advisable to take partial charges and LJ pair coefficients directly from papers. This is especially important if you are trying to benchmark your results against previous publications. For instance, in case of TFSI-, antechamber partial charges are significantly different than partial charges available in previous publications -

    In short,

  4. Open the .pdb file in VMD. Use Extensions -> TK console to open the TCL scripting menu

  5. Then use this sequence of commands to get the bonds, angles, dihedrals and most importantly, the box size topo retype bonds topo guessangles topo guessdihedrals set sel [atomselect top all]

    set center [measure center $sel weight none]

    $sel moveby [vecscale -1.0 $center]

    set minmax [measure minmax $sel -withradii]

  6. Once all these TCL commands execute without any errors, we can get the lammps data file using: - topo writelammpsdata <lammps.data> full [Note: we can change the atom type to something else like atomic depending on the system, but mostly for molecular systems, full will be the way to go]

  7. This is just a skeletal data file with the relevant bonds, angles, and dihedrals. No parameters are currently in this datafile. Sadly, the grunt work starts here. On paper, we can use the moltemplate package to get all the parameters, but it has never worked reliably (at least for me) . The work around is to visit the relevant force field parameter page on the moltemplate GitHub. For instance, for GAFF go to, and extract all the parameters (LJ, bond, angle, dihedral) from there. Remember to benchmark your LJ parameters.

  8. Here is what a finished file will look like

hashtag
Lawrencium

module load ms/lammps/3Mar20-mpi

hashtag
Compiling LAMMPs

author: Bryant Li

The GPU parallelized versions that works with KOKKOS is still a work in process. Most of the time, there is a default module load lammps on each super computer. This is specifically designed for those we needs to compile LAMMPs with an external package. Please refer to for what LAMMPs can accommodate.

hashtag
Kestrel

Below is an example with PyACE:

Useful links:

Note: if doesn't work/not fast enough, try to speed it up with this:

After executing the last line in the command prompt, you will have an lmp binary in the lammps/build. directory. You can set up a submit script like below:

hashtag
Perlmutter

Below is an example with pair_allegro. WARNING!!!! ONLY WORKS FOR CPU.

Prereqs:

Conda env setup:

Make lammps:

Module list at the time:

GPU Setup (THIS HASN'T WORKED FOR pair_allegro; due to a PyTorch error. You may have better luck if you use a different external code).

module list:

Prereqs:

Conda env setup:

Make lammps:

antechamber
is a good starting point, but it is always advisable to tread carefully with partial charges in MD. Also remember, that these charges are not scaled. Why is charge scaling important? Go here – ‘
Kirby, B. J.; Jungwirth, P. Charge Scaling Manifesto: A Way of Reconciling the Inherently Macroscopic and Microscopic Natures of Molecular Simulations. J. Phys. Chem. Lett. 2019, 10 (23), 7531–7536. https://doi.org/10.1021/acs.jpclett.9b02652.
’

set box [vecscale 1.1 [vecsub [lindex $minmax 1] [lindex $minmax 0]]]

pbc set $box

manualarrow-up-right
https://pubs.acs.org/doi/full/10.1021/jp0686893arrow-up-right
https://github.com/jewettaij/moltemplate/blob/master/moltemplate/force_fields/gaff.ltarrow-up-right
https://docs.lammps.org/Build_extras.htmlarrow-up-right
https://docs.lammps.org/Build_extras.html#ml-pace https://github.com/ICAMS/lammps-user-pace/ https://pacemaker.readthedocs.io/en/latest/pacemaker/quickstart/#lammpsarrow-up-right
https://docs.nersc.gov/development/programming-models/kokkos/ arrow-up-right
LAMMPS Description 
8 atoms
7 bonds
12 angles
9 dihedrals
0 impropers

4 atom types
3 bond types
4 angle types
1 dihedral types
0 improper types

-3.443375 3.376625  xlo xhi
-3.591275 3.749025  ylo yhi
-2.930725 3.049975  zlo zhi

Masses

1 12.010700 # C
2 18.998404 # F
3 15.999400 # O
4 32.064999 # S

Atoms # full

1 1 3 -0.63000 -1.613375 -0.108625 -1.138875 # O UNL
2 1 4 1.02000 -0.839375 0.519375 -0.059875 # S UNL
3 1 3 -0.63000 -1.476375 0.389375 1.258125 # O UNL
4 1 1 0.35000 0.748625 -0.462625 0.053125 # C UNL
5 1 2 -0.16000 1.596625 -0.000625 1.027125 # F UNL
6 1 2 -0.16000 1.473625 -0.444625 -1.110875 # F UNL
7 1 2 -0.16000 0.539625 -1.787625 0.340125 # F UNL
8 1 3 -0.63000 -0.429375 1.895375 -0.368875 # O UNL

Bonds

1 3 1 2
2 3 2 8
3 2 2 4
4 3 2 3
5 1 4 6
6 1 4 7
7 1 4 5

Angles

1 4 1 2 8
2 1 1 2 4
3 4 1 2 3
4 1 4 2 8
5 4 3 2 8
6 1 3 2 4
7 3 2 4 6
8 3 2 4 7
9 3 2 4 5
10 2 6 4 7
11 2 5 4 6
12 2 5 4 7

Dihedrals

1 1 1 2 4 6
2 1 1 2 4 7
3 1 1 2 4 5
4 1 8 2 4 6
5 1 8 2 4 7
6 1 8 2 4 5
7 1 3 2 4 6
8 1 3 2 4 7
9 1 3 2 4 5

Pair Coeffs #lj/charmm/coul/long

1 0.0660 3.50000000000 #C Kelkar Maginn 
2 0.0530 2.95000000000 #F Kelkar Maginn
3 0.2100 2.95992190114 #O Kelkar Maginn
4 0.2500 3.56359487256 #S Kelkar Maginn

Bond Coeffs #harmonic

1 356.9 1.3497 # SOURCE1_SOURCE5 2188 0.0139, this is c3-f
2 233.5 1.8075 # SOURCE1_SOURCE5 644 0.0143, this is c3-s6
3 512.7 1.4533 # SOURCE1_SOURCE5 2456 0.0105, this is o-s6

Angle Coeffs #harmonic

1 65.4 108.61   # SOURCE3_SOURCE5 1062 1.0758, this is c3-s6-o
2 70.9 107.36   # SOURCE2_SOURCE5 1178 0.5429, this is f-c3-f
3 63.1 109.68   # SOURCE4_SOURCE5 57 0.4273, this is f-c3-s6
4 73.6 120.05   # SOURCE4_SOURCE5 971 1.8153, this is o-s6-o

Dihedral Coeffs #fourier

1 1 0.14445 3 0.0 #F-C-S-O used same parameters as X-c3-s6-X
#substitute this line for whichever type of lammps is configured to your code
git clone https://github.com/ICAMS/lammps-user-pace.git


cd lammps
mkdir build
cd build

cmake -DCMAKE_BUILD_TYPE=Release -D BUILD_MPI=ON -DPKG_ML-PACE=ON ../cmake

make
#!/bin/bash -l
#SBATCH --nodes=1
#SBATCH --time=24:00:00
#SBATCH --partition=standard
#SBATCH --account=bmrkp
#SBATCH --job-name=lmp_launcher
#SBATCH --output=lmp_launcher-%j.out
#SBATCH --signal=SIGINT@60
#SBATCH --mail-type=all
#SBATCH --mail-user=<<YOUR_EMAIL@lbl.gov>> #remove this line if you want

exe="/PATH_TO_LAMMPSBUILD/lammps/build/lmp"
input="-in in.lammps"

export OMP_PROC_BIND=spread
export OMP_PLACES=threads

command="srun -n 96 --cpu-bind=cores $exe $input" 

echo $command

$command
cd repo
git clone -b stable_29Sep2021_update2 --depth 1 git@github.com:lammps/lammps
git clone git@github.com:mir-group/pair_allegro
cd pair_allegro
./patch_lammps.sh ../lammps/
conda create -n pair_allegro python=3.9 numpy matplotlib pandas plotly seaborn
conda activate pair_allegro 
pip install torch==1.12.1+cpu torchvision==0.13.1+cpu torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cpu
pip install mkl-include (apparently already within it...?)
cd lammps
mkdir build
cd build

cmake ../cmake -DCMAKE_PREFIX_PATH=`python -c 'import torch;print(torch.utils.cmake_prefix_path)'` -DMKL_INCLUDE_DIR=`python -c "import sysconfig;from pathlib import Path;print(Path(sysconfig.get_paths()[\"include\"]).parent)"`

make -j16
Currently Loaded Modules:
  1) craype-x86-milan                        6) cray-dsmml/0.2.2        11) xalt/2.10.2              16) gpu/1.0
  2) libfabric/1.15.2.0                      7) cray-libsci/22.11.1.2   12) Nsight-Compute/2022.1.1  17) evp-patch
  3) craype-network-ofi                      8) craype/2.7.19           13) Nsight-Systems/2022.2.1  18) python/3.9-anaconda-2021.11
  4) xpmem/2.5.2-2.4_3.20__gd0f7936.shasta   9) perftools-base/22.09.0  14) cudatoolkit/11.7         19) gcc/11.2.0
  5) PrgEnv-gnu/8.3.3                       10) cpe/22.11               15) craype-accel-nvidia80    20) cray-mpich/8.1.22
Currently Loaded Modules:
  1) craype-x86-milan                        6) cray-dsmml/0.2.2        11) xalt/2.10.2              16) gpu/1.0
  2) libfabric/1.15.2.0                      7) cray-libsci/22.11.1.2   12) Nsight-Compute/2022.1.1  17) evp-patch
  3) craype-network-ofi                      8) craype/2.7.19           13) Nsight-Systems/2022.2.1  18) python/3.9-anaconda-2021.11
  4) xpmem/2.5.2-2.4_3.20__gd0f7936.shasta   9) perftools-base/22.09.0  14) cudatoolkit/11.7         19) gcc/11.2.0
  5) PrgEnv-gnu/8.3.3                       10) cpe/22.11               15) craype-accel-nvidia80    20) cray-mpich/8.1.22
cd pair_allegro_lammps
git clone -b stable_29Sep2021_update2 --depth 1 git@github.com:lammps/lammps
git clone git@github.com:mir-group/pair_allegro
cd pair_allegro
./patch_lammps.sh ../lammps/
conda create -n pair_allegro_gpu python=3.9 numpy matplotlib pandas plotly seaborn
conda activate pair_allegro_gpu 
pip install torch==1.12.1+cu116 torchvision==0.13.1+cu116 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu116
pip install mkl-include (apparently already installed)
cd lammps
mkdir build
cd build

module load cudnn/8.7.0


cmake ../cmake -DCMAKE_PREFIX_PATH=`python -c 'import torch;print(torch.utils.cmake_prefix_path)'` -DMKL_INCLUDE_DIR=`python -c "import sysconfig;from pathlib import Path;print(Path(sysconfig.get_paths()[\"include\"]).parent)" -DCUDA_TOOLKIT_ROOT_DIR=/opt/nvidia/hpc_sdk/Linux_x86_64/22.5/cuda/11.7

make -j16

VASP

We generally run 2 versions of VASP - 5.4.4 and 6. VASP 6 is used primarily on nodes operating on the Knights Landing (KNL) CPU architecture, while VASP 5.4.4 is generally used on all other CPUs.

hashtag
VASP on NERSC

NERSC maintains compiled, and optimized versions of the vasp binaries. To get access to these binaries, you must be added to the group VASP license. If/when you determine that your research requires the use of VASP, please contact Kristin and request to be added to the VASP License. We currently have two sets of licenses i) UC Berkeley Affiliated (Purchased through VASP Software GmbH) and ii) LBNL Affiliated (Purchased through Materials Design). Once you have been added to one of the licenses, fill out the VASP License Confirmation Requestarrow-up-right to get access to the binaries on NERSC machines.

More information can be found in the

The potcar files can be found in the following directory.

*Note*: Users with only jcesr access will need to request access to matgen to access these POTCARs. Contact the NERSC Resource Manager and request to be added to the 'matgen' UNIX group.

hashtag
VASP on Berkeley Research Computing (savio)

When running vasp on savio, we have access to our own compilations. To access the vasp binaries on savio, add the following line to your .bashrc file.

POTCARs are located at:

For issues with these compilations, please contact Eric Sivonxay (esivonxay@lbl.gov)

hashtag
Lawrencium

When running vasp on Lawrencium, we have access to our own compilations. To access the vasp binaries on Lawrencium, add the following line to your .bashrc file.

Pre-release versions of the VASP modules are located at export /clusterfs/mp/temp_modfiles

POTCARs are located at:

For issues with these compilations, please contact Eric Sivonxay (esivonxay@lbl.gov)

Special thanks to the original authors of this page: Eric Sivonxay

Simulation Codes

NERSC documentationarrow-up-right
/project/projectdirs/matgen/POTCARs
export MODULEPATH=${MODULEPATH}:/global/home/groups/co_lsdi/sl7/modfiles
/clusterfs/cloudcuckoo/POTCARs
export MODULEPATH=${MODULEPATH}:/global/home/groups-sw/lr_mp/modules/vasp
/clusterfs/mp/software/POTCARs

OpenMM

hashtag
DEPRECATION WARNING

Use of pymatgen.io.openmm is no longer recommended. It has been superceded by workflows in atomate2. Once the tutorials are complete, someone ought to update this page :) Points of contact for MD questions: 1. Orion (leaving August 2024) 2. Nessa 3. Rohith

hashtag
OpenMM

Our group has a robust infrastructure for setting up and running molecular MD simulations. The core of this infrastructure is in the Pymatgen . Once you have installed that package and its dependencies, you should be able to use the following code. For a quick install of the dependencies, execute these commands:

First, you should import all of the functions and objects that we'll use below.

hashtag
Generators

A pymatgen InputSet contains all of the files needed to set up and run a simulation. Writing those files by hand would be incredibly tedious, so we use a Generator to create the InputSet. The Generator is instantiated with simulation properties as input, like temperature and force field. Once it is created, it can generated many InputSets with different sets of molecules.

hashtag
Input Set

An InputSet is created by calling generator.get_input_set() and providing the molecules (as smiles) and the density of the solution. When this happens, a ton of code is run behind the scenes that sets up the simulation.

We might not want to run our simulation immediately! For example, we might want to upload it to a supercomputer. We can easily do this by saving all of the files with input_set.write_input.

It is equally easy to load an InputSet from a directory.

hashtag
Simulations

An InputSet can create a dynamic OpenMM Simulation object. This simulation object can be run and modified using the OpenMM software package.

Once we have a simulation, we will first need to minimize the energy, otherwise the simulation box will explode. Once that's done, we can start the simulation!

Congratulations! You ran your first OpenMM simulation!

hashtag
Reporters

By default, an OpenMM Simulation will not output any information. If we want data, we will need to add OpenMM Reporters to write out the trajectory and simulation state. In the next cell, I attach the most useful reporters, StateData, which writes state information, DCDReporter, which saves the trajectory, and PDBReporter, which saves the topology.

200 is an extremely short time interval, in reality, you would want to use something like 1,000 for the StateDataReporter and 10,000 for the DCD reporter. Next we can run the simulation and see that it ran. We are using a short time interval here because this is just a demonstration.

hashtag
Running a real simulation

A real molecular dynamics simulation is slightly more complex than the toy examples we've covered here. We first need to equilibrate the pressure of the simulation, which will change the volume until the simulation contents are at atmospheric pressure. Then we need to anneal the simulation by heating it to 400K to make sure we find a true equilibrium. Fortunately, the Pymatgen code has very convenient functions for doing both of these things. A real simulation would look something like below (though in reality we would run it for many more timesteps).

OpenMM IO add-onarrow-up-right
¶arrow-up-right
>>> conda install -c conda-forge parmed openmm openff-toolkit packmol ambertools=21 compilers openbabel openmmforcefields rdkit
from pymatgen.io.openmm.generators import OpenMMSolutionGen
from pymatgen.io.openmm.sets import OpenMMSet
from pymatgen.io.openmm.simulations import equilibrate_pressure, anneal

from openmm.app import StateDataReporter, DCDReporter, PDBReporter
openmm_generator = OpenMMSolutionGen(
    force_field="Sage",
    temperature=298,
    step_size=0.001,
)
molecules = {"O": 200, "CCO": 1}

input_set = openmm_generator.get_input_set(molecules, density=1) # get that input set!
input_set.write_input(f"input_set_files") # write that input!
input_set = OpenMMSet.from_directory("input_set_files")
simulation = input_set.get_simulation()
simulation.minimizeEnergy() # see what happens if you comment out this line with a fresh simulation :)
simulation.step(1000)
simulation = input_set.get_simulation() # first we will refresh our simulation

!mkdir output # first lets make a directory to save our files

# we actually only use the PDB reporter once, we will need the PDB later when we use MDAnalysis
pdb_reporter = PDBReporter("output/topology.pdb", 1)
pdb_reporter.report(simulation, simulation.context.getState(getPositions=True))

# the StateDataReporter and DCDReporter will write out every n_timesteps
n_timesteps = 200
simulation.reporters.append(StateDataReporter("output/state.txt", n_timesteps, step=True, potentialEnergy=True, temperature=True, volume=True, density=True))
simulation.reporters.append(DCDReporter(f"output/trajectory.dcd", n_timesteps))
simulation.minimizeEnergy()
equilibrate_pressure(simulation, 10000)
anneal(simulation, 400, [2000, 2000, 2000])
simulation.step(10000)

GROMACS

More Information to come