DP-GEN
This article introduces how to run a typical DP-GEN job on Bohrium.
Introduction
DP-GEN (Deep Potential Generator) is committed to solving the following two problems:
- How to sufficiently sample the potential energy surface and effectively cover the sample space required for training reliable models?
- How to efficiently screen samples and select conformations with high training value for first-principles calculations?
The workflow is shown as follows: the "explore-label-train" loop iterates until the potential function accuracy meets the expected requirements.
To facilitate data generation and monitor the accuracy of the potential function, DP-GEN also integrates modules such as init_bulk, init_surf, autotest, and simplify.
For more DP-GEN introductions, please refer to https://github.com/deepmodeling/dpgen
How to run DP-GEN jobs on Bohrium
This case takes about 60 min to run
Step 1, prepare input data
First, you need to start the management node, and select the image dpgen:0.10.6 for this case. Connect to the management node through the Web Shell, and execute the following statement to ensure that DP-GEN and its dependencies are correctly installed:
dpgen -h
If everything is normal, the DP-GEN version and information about some dependent libraries will be printed, as shown in the following example:
Version: 0.10.6
Date: Nov-01-2022
Path: /opt/miniconda/lib/python3.7/site-packages/dpgen
Dependency
------------
numpy 1.21.6 /opt/miniconda/lib/python3.7/site-packages/numpy
dpdata 0.2.10 /opt/miniconda/lib/python3.7/site-packages/dpdata
pymatgen unknown version or path
monty 2022.9.9 /opt/miniconda/lib/python3.7/site-packages/monty
ase 3.22.1 /opt/miniconda/lib/python3.7/site-packages/ase
paramiko 2.11.0 /opt/miniconda/lib/python3.7/site-packages/paramiko
custodian 2022.5.26 /opt/miniconda/lib/python3.7/site-packages/custodian
Reference
------------
Please cite:
Yuzhi Zhang, Haidi Wang, Weijie Chen, Jinzhe Zeng, Linfeng Zhang, Han Wang, and Weinan E,
DP-GEN: A concurrent learning platform for the generation of reliable deep learning
based potential energy models, Computer Physics Communications, 2020, 107206.
------------
Description
------------
usage: dpgen [-h]
{init_surf,init_bulk,auto_gen_param,init_reaction,run,run/report,collect,simplify,autotest,db}
...
dpgen is a convenient script that uses DeepGenerator to prepare initial data,
drive DeepMDkit and analyze results. This script works based on several sub-
commands with their own options. To see the options for the sub-commands, type
"dpgen sub-command -h".
positional arguments:
{init_surf,init_bulk,auto_gen_param,init_reaction,run,run/report,collect,simplify,autotest,db}
init_surf Generating initial data for surface systems.
init_bulk Generating initial data for bulk systems.
auto_gen_param auto gen param.json
init_reaction Generating initial data for reactive systems.
run Main process of Deep Potential Generator.
run/report Report the systems and the thermodynamic conditions of
the labeled frames.
collect Collect data.
simplify Simplify data.
autotest Auto-test for Deep Potential.
db Collecting data from DP-GEN.
optional arguments:
-h, --help show this help message and exit
Please ensure that the information is correct before preparing the input files. Using methane as an example, DP-GEN input files are stored in the Bohrium_DP-GEN_run_example folder. After entering the data disk with the cd /data command, execute the following commands in sequence to download and unzip the input files:
wget https://bohrium-example.oss-cn-zhangjiakou.aliyuncs.com/Bohrium_DP-GEN_run_example.zip
unzip Bohrium_DP-GEN_run_example.zip
cd Bohrium_DP-GEN_run_example
Use the tree . -L 2 command to view the input files, which mainly include the following content:
.
|-- CH4.POSCAR
|-- CH4.POSCAR.01x01x01
| |-- 00.place_ele
| |-- 01.scale_pert
| |-- 02.md
| `-- param.json
|-- INCAR_methane
|-- INCAR_methane.md
|-- INCAR_methane.rlx
|-- POTCAR_C
|-- POTCAR_H
|-- init.json
|-- machine.json
`-- run_param.json
Explanation of files required for DP-GEN operation: File/Folder Name|Description :-:|-| CH4.POSCAR.01x01x01 folder|VASP structure file machine.json|Machine configuration file; including computing resources, logs, etc. INCAR_methane|VASP calculation parameter file; single point energy, used in this tutorial run process POTCAR_C, POTCAR_H|VASP pseudopotential files; set according to the number of system elements run_param.json|Calculation parameter file
Step 2, Prepare Configuration Files
The folder already contains the configuration file machine.json. In the Files Management page, you can double-click the machine.json file in the left file tree to edit and save it online, or you can edit it in the command line window:
vi machine.json
Enter i to enter edit mode, complete the modification, press esc to exit edit mode and enter : to enter the last line command mode, then enter wq to save and exit. The content of the configuration file is as follows:
Note:
Replace the
0000after"project_id"with your own project ID, which can be viewed on the "Project Management" page. There are three locations to be replaced.Fill in your Bohrium account email and password for
emailsandpassword. There are three locations to be modified.Adjust the
image_addressaccording to your needs. In this example, the training part usesregistry.dp.tech/dptech/deepmd-kit:2.1.5-cuda11.6, the exploration part usesregistry.dp.tech/dptech/deepmd-kit:2.1.5-cuda11.6, and the labeling part uses the VASP image (if you need to use the VASP image, please provide your VASP authorization certificate to the Bohrium email).
{
"api_version": "1.0",
"deepmd_version": "2.1.5",
"train" :[
{
"command": "dp",
"machine": {
"batch_type": "Bohrium",
"context_type": "BohriumContext",
"local_root" : "./",
"remote_profile":{
"email": "",
"password": "",
"program_id": 0000,
"keep_backup":true,
"input_data":{
"log_file": "00*/train.log",
"grouped":true,
"job_name": "dpgen_train_job",
"disk_size": 100,
"scass_type":"c8_m32_1 * NVIDIA V100",
"checkpoint_files":["00*/checkpoint","00*/model.ckpt*"],
"checkpoint_time":30,
"platform": "ali",
"job_type": "container",
"image_address":"registry.dp.tech/dptech/deepmd-kit:2.1.5-cuda11.6",
"on_demand":0
}
}
},
"resources": {
"number_node": 1,
"cpu_per_node": 8,
"gpu_per_node": 1,
"queue_name": "V100_8_32",
"group_size": 1,
"custom_flags": [],
"strategy": {"if_cuda_multi_devices": true},
"para_deg": 3,
"source_list": []
}
}],
"model_devi":[
{
"command": "lmp -i input.lammps -v restart 0",
"machine": {
"batch_type": "Bohrium",
"context_type": "BohriumContext",
"local_root" : "./",
"remote_profile":{
"email": "",
"password": "",
"program_id":0000,
"keep_backup":true,
"input_data":{
"log_file": "*/model_devi.log",
"grouped":true,
"job_name": "dpgen_model_devi_job",
"disk_size": 200,
"scass_type":"c8_m32_1 * NVIDIA V100",
"platform": "ali",
"job_type": "container",
"image_address":"registry.dp.tech/dptech/deepmd-kit:2.1.5-cuda11.6",
"on_demand":0
}
}
},
"resources": {
"number_node": 1,
"cpu_per_node": 8,
"gpu_per_node": 1,
"queue_name": "V100_8_32",
"group_size": 250,
"source_list": []
}
}],
"fp":[
{
"command": "ulimit -s unlimited; mpirun -np 32 vasp_std",
"machine": {
"batch_type": "Bohrium",
"context_type": "BohriumContext",
"local_root" : "./",
"remote_profile":{
"email": "",
"password": "",
"program_id":0000,
"input_data":{
"api_version":2,
"log_file": "task*/fp.log",
"grouped":true,
"job_name": "dpgen_fp_job",
"disk_size": 100,
"scass_type":"c32_m64_cpu",
"platform": "ali",
"job_type":"container",
"image_address":"You need to provide Authorization certificate to bohrium@dp.tech Email Get VASP Image",
"on_demand":0
}
}
},
"resources": {
"number_node": 4,
"cpu_per_node": 8,
"gpu_per_node": 0,
"queue_name": "CPU",
"group_size": 10,
"source_list": ["/opt/intel/oneapi/setvars.sh"]
}
}]
}
Step 3, Submit the Job
nohup dpgen run run_param.json machine.json 1>log 2>err&
Note: The DP-GEN process needs to wait for the job to complete before submitting further rounds of jobs, so please do not close the management node.
Check the Job
You can learn how to view the job status on the Bohrium platform in the Monitor Task Document.
Download the Results
You can learn how to download the job results on the Bohrium platform in the Result Download Document.
More DP-GEN Parameter Introduction
If you want to use a custom input.lammps template in the model_devi step, such as implementing plumed functions, please refer to the document DP-GEN model_devi Step LAMMPS Custom Template
For more detailed settings of DP-GEN parameters, please refer to the DP-GEN Source Code Repository and DP-GEN Official Documentation.