Request for Proposals for Biomolecular Simulation Time on Anton

Introduction

The Pittsburgh Supercomputing Center (PSC, www.psc.edu) is soliciting proposals for computer time on Anton 3 [1], a special-purpose supercomputer for molecular dynamics (MD) simulation designed by D. E. Shaw Research (DESRES). This 64-node third-generation Anton  machine hosted at PSC,  replaces and greatly enhances the capabilities of the previous generation Anton machine [2]. It is made available without cost by DESRES for non-commercial research use by universities and other not-for-profit institutions.

To qualify for an allocation on Anton, the principal investigator (PI) must be a faculty or staff member at a U.S. academic or non-profit research institution and must have the appropriate level of authority and responsibility to direct the project supported by the allocation. International collaborations are allowed, and co-PIs do not need to be associated with U.S. academic or non-profit research institutions. Graduate students and postdoctoral researchers may not be PIs, but qualified advisors may apply for allocations on their behalf. Researchers with no previous experience with Anton are encouraged to apply.

Each investigator can serve as a PI for only a single application for computer time on Anton in a given allocation round. If there is a co-PI on the proposal, the PI must clearly demonstrate that the work described in the proposal will be conducted primarily in the PI’s lab and will be directly supervised by the PI. In addition, if a PI or co-PI is on another Anton proposal (as co-PI or PI, respectively), investigators involved in both proposals must also clearly demonstrate that the research efforts described in the two proposals are distinct and independent of each other, and thus do not constitute an effective doubling of the allowed allocation for a given research effort (see #2 in the Structure of Proposal section).

PIs who previously had an allocation on Anton 2 are required to provide a progress report of their past work on the Anton machine at PSC (see #7 in the Structure of Proposal section). Applications by a returning PI that are unaccompanied by a progress report will not be considered, and the  new application should not be identical in text to the previous application.

Trajectories generated on Anton and all their associated data will be kept private until publication, or up to one year after the end of an Anton allocation, whichever comes first. After this embargo period, the trajectories and data will become part of a unique collection of hundreds of long-timescale trajectories that will be made widely available to the research and education communities. To this end, investigators must agree to share their trajectories and data no later than one year after the end of the allocation period in which the data were generated. Investigators are responsible for ensuring that they comply with the data sharing policies of the agencies funding their research.

The remainder of this Request for Proposals (RFP) describes the intended focus of solicited projects, outlines simulation requirements, offers proposal preparation instructions, and explains the proposal review process.

Intended Focus of Proposed Projects

Anton enables scientists to perform MD simulations of biomolecular systems approximately two orders of magnitude faster than other available platforms (see the Estimating Requested Simulation Resources section). To maximize the benefit of Anton to the scientific community, proposed projects should focus exclusively on questions that will be greatly advanced by the unique capabilities of Anton. In their description of the scientific objectives, investigators must clearly explain why achieving their objectives requires access to Anton, and could not be efficiently achieved on other high performance computing systems available at their institutions or through national supercomputing resources made available by the NSF, DOE, or other programs. Generally, this justification will include the need for longer MD trajectories than are feasible on conventional systems. Proposals that do not clearly justify the need for Anton over other available computing resources will receive a lower ranking in the review process.

Simulation Requirements

Anton uses specialized hardware to perform molecular dynamics simulations orders of magnitude faster than general-purpose hardware running traditional MD software [1,2]. Importantly, Anton does not run Desmond, AMBER, NAMD, GROMACS, or any other MD simulation software package, although it uses a Desmond structure (DMS) file as an initial input and its trajectory output is compatible with Desmond’s DTR files. Each simulation must be built, using tools available at PSC, specifically for Anton.

Anton is designed primarily to accelerate classical MD simulations of biomolecular systems with periodic boundary conditions and explicit solvent. To make best use of the available computational resources, consideration will be given only to applicants whose projects satisfy the criteria outlined below. Investigators who have questions regarding the suitability of their proposed simulations are encouraged to contact anton-support@psc.edu to discuss their planned project before submitting their proposal.

1. Simulations must be standard MD runs in the constant NVE, constant NVT (isothermal), or constant NPT (isothermal, isobaric) ensembles.  Constant NVT and NPT simulations must use the Multigrator framework [3] with either Nose-Hoover or Langevin thermostats and isotropic or semi-isotropic MTK barostats. Simulation conditions may include the specification of a uniform constant applied electric field. Position restraints, on a per atom basis, are allowed. Enhanced sampling is also available in four forms as follows: (i) simulated tempering, including adaptive weighting, (ii) application of restraints between the centers of mass of groups of atoms, (iii) application of conformational restraints, each based on the calculation of RMSD (root mean squared deviation) with respect to atomic positions of a given reference structure, and (iv) tempered binding, as used in References [4] and [5].  For restraints in both (ii) and (iii), equilibria and spring constants can be varied during a simulation according to a schedule or adaptively to implement a form of umbrella sampling. Applicants with systems that have dozens of restraints and/or restraints involving thousands of atoms should contact anton-support@psc.edu before submitting a proposal.
2. The simulation cell must have only right angles (i.e., it must be a cubic or orthorhombic box), and must have a minimum of 50 Angstroms on each side. Applicants with systems shaped such that one dimension of the simulation cell is much larger than the others should contact anton-support@psc.edu before submitting a proposal.
3. Proposed simulations must use recent variants of the following standard, nonpolarizable biomolecular force fields: CHARMM (e.g., CHARMM22, CHARMM27 – including CMAP corrections, and CHARMM36), AMBER (e.g., AMBER99, AMBER99SB, AMBER03, and AMBER19SB). Modified versions of the CHARMM and AMBER force fields, based on published research by DESRES, are also acceptable (and available through the simulation setup tools). Water should be modeled with the SPC, TIP3P, or TIP4P models, or their variants. For more information on force fields available on Anton, see here.
4. Chemical systems proposed for simulation must contain between 100,000 and 8,500,000 atoms (including solvent atoms), though systems between 500,000 and 6,000,000 atoms are recommended for maximum efficiency. Proposals with systems significantly smaller than 500K atoms should have additional justification explaining why the study could not be done with a larger system that runs more efficiently on Anton 3. Chemical systems proposed for simulation must consist of some combination of protein, DNA, RNA, lipids, water, and standard ions. Investigators who wish to use custom parameters or molecules that are not included in the standard distribution of the supported force fields (outlined in 3 above) should contact anton-support@psc.edu to discuss the suitability of their simulations before submitting their proposal.
5. Anton is capable of producing long, continuous MD trajectories. To maximize the benefit of this resource to the research community, researchers should not propose to run simulation trajectories that would finish in less than one hour on Anton (e.g., less than ~6,000 ns for a 100,000 atom system, and less than ~125 ns for an 8,500,000 atom system; see Estimating Requested Simulation Resources).

Biomolecular systems should be well equilibrated prior to running on Anton. PSC will provide instructions to convert files to Anton format for equilibrated systems generated with the following programs: Desmond, Amber, CHARMM, Gromacs, or NAMD. However, for the easiest conversion, we strongly suggest using Desmond to equilibrate, especially if your system has custom molecules. Researchers can request allocations of compute time on the Bridges-2 system at PSC if they would like to use this resource for preparation and equilibration of systems they propose to run on Anton (https://wp-dev.psc.edu/resources/bridges-2/).

Estimating Requested Simulation Resources

Applicants may refer to the table below, with benchmarks for a number of systems of various sizes, to estimate the amount of machine time required for their project. For example, simulating the 1,066,628 STMV system for 100 microseconds would require 100 microseconds/42.3 microseconds per day = about 2.4 machine-days. The actual achievable simulation times, however, may vary even for different molecular systems of similar size. The rates in the table tend to represent an upper bound.

No more than a total of 9.1 machine-days will be allocated to any one principal investigator. It is anticipated that 10 to 20 allocations will be made at or near 9.1 machine-days, and 30 to 40 allocations will be made at or near 4.5 machine-days. Applicants are encouraged, but not required, to target their requested resources at one of these levels, and they must provide justification for the number of machine-days requested (see #6 in Structure of Proposal section). Note that simulations on Anton run on all 64 nodes.

For more details regarding Anton 3, please see the References.

Structure of Proposal

Proposals should be two to six pages in length, not including references. PIs who previously received an allocation on Anton at PSC must also provide a separate progress report (2 pages maximum) demonstrating their successful use of Anton to produce high-impact scientific results. This report will be an important factor in assessing the project feasibility for proposals from returning investigators (see Project Feasibility and Team Qualifications and Progress Report sections below). In addition, applicants may submit up to two additional supporting documents (e.g., published papers) in PDF format on the submission page; however, all information required to assess the merits of the proposal must be contained within the text of the proposal and (for returning investigators) the  progress report. Reviewers will not be required to refer to supporting documents or any other external documents when reviewing proposals.

The main proposal document must have all the sections listed below. Proposals that do not clearly address all the points listed in each section will receive a lower ranking:

1. Summary of the project, including descriptive title of proposed research (400 words maximum for summary & title).
2. Name, address, email, and telephone number of the Principal Investigator and co-PIs. If a PI or co-PI is on another Anton proposal (as co-PI or PI, respectively), describe here how the two proposals are distinct and independent of each other, and thus do not constitute an effective doubling of the allowed allocation for a given research effort. Failure to provide such an explanation may result in a lower ranking.
3. Background information (1 page maximum): Investigators should include sufficient background information on the research field to allow reviewers to judge the scientific merit of the proposed research.
4. Scientific Objectives to be accomplished on Anton (2 pages maximum): Investigators should clearly explain, in detail, why Anton is necessary for the planned project and why the project could not be efficiently completed on conventional systems (see the Intended Focus of Proposed Projects section for more details). Applicants should also clearly explain the scientific impact of their proposed project. Proposals with systems significantly smaller than 500K atoms should have additional justification explaining why the study could not be done with a larger system that runs more efficiently on Anton 3.
5. Project Feasibility and Team Qualifications (2 pages maximum): Applicants must clearly and explicitly address all five points outlined in the above Simulation Requirements section of this RFP, providing all necessary details regarding their proposed simulations to ensure technical feasibility. Please state clearly whether the proposed system has already been built and equilibrated. If the system has not yet been run in production, please provide evidence that the proposed system can be successfully built and simulated. In addition, applicants must describe the expertise and experience of their research team members (including prior experience running MD simulations) to ensure that the proposed simulations can be successfully completed on Anton. If there is a co-PI on a proposal, provide information here demonstrating that the work described in the proposal will be conducted primarily in the PI’s lab and will be directly supervised by the PI. It is particularly important for applicants who had previous Anton allocations to report successful outcomes from those awards, but these outcomes should be described in a separate progress report (see Progress Report below). 
6. Requested Resources (1 page maximum): Investigators must clearly state and provide justification for the number of Anton machine-days requested for their project subject to the limits given in Estimating Requested Simulation Resources. The justification should provide strong scientific arguments as to why the length and number of proposed simulation runs will be both sufficient and necessary to achieve the stated scientific objectives. Please refer to the Estimating Requested Simulation Resources section for benchmarks to facilitate estimation of requested resources. No more than a total of 9.1 machine-days can be requested per PI.
7. Progress Report (required only for groups with previous allocations, in a separate document , 2 pages maximum): The Progress Report should describe how the PI’s current or prior allocation was used and summarize the findings or results. These results can be discussed in the context of publications, presentations, and other communications resulting from this work, but all information required to assess progress should be clearly communicated in the text of the progress report itself and should not depend on reviewers reading supporting documents or any other external documents. The Progress Report should also explain any significant digressions from the originally proposed resource usage plan for the prior period. Applications by a returning PI that are unaccompanied by a progress report will not be considered.If a PI’s allocation was significantly underutilized, the Progress Report should briefly describe the reasons for the underutilization and any mitigation by the PI to avoid the situation with the current allocation. PIs that demonstrate a pattern of low utilization across multiple allocation periods, without strong justification for this underutilization, will receive a lower ranking.

Important Dates and Proposal Review Process

More information about Anton 3  can be found at https://wp-dev.psc.edu/resources/anton-3.  Proposals should be submitted electronically in PDF format at https://allocations.psc.edu. You will need to have a PSC account to access the submission portal. Go to https://allocations.psc.edu and create a PSC account if you don’t already have one. Please do this at least 2 days prior to the submission deadline so you can get access to the submission system in time to submit your proposal.

The deadline for applications is 11:59 pm, EDT, Monday, October 14, 2024.

Proposals will initially be assessed for technical feasibility by PSC staff (see the Simulation Requirements section) and will then be reviewed by a peer committee to be convened by the National Academies of Sciences, Engineering, and Medicine. Resource allocations by PSC will follow the recommendations of the National Academies of Sciences, Engineering, and Medicine. Resources are expected to be available starting in March 2025.

Proposals will be ranked based on the scientific merit of the proposed research, the strength of the justification for the requested resources, and evidence of the ability of the research team to successfully execute the proposed studies. In addition, proposals by PIs that previously received an allocation on Anton 2 at PSC will need to demonstrate that the scientific outcomes from the previous award(s) justify another award. At least 25% of the total allocated time will be reserved for PIs that have not previously had an allocation on an Anton system at PSC. Since only a limited number of allocations will be available on Anton, a successful proposal will clearly state how access to Anton will facilitate breakthrough science.

Anton Webinars: To help with the proposal submission, we will offer two webinars on September 23, 2024:

• 2:00-3:00 PM (EDT) Anton 3 Capabilities and Enhanced Sampling Techniques   Register now
• 3:00-4:00 PM (EDT) How to Write a Successful Anton 3 Proposal    Register now

For general questions regarding this RFP, or to discuss feasibility and technical aspects of projects, please contact anton-support@psc.edu.

References

1. Shaw DE, Adams PJ, Azaria A, Bank JA, Batson B, Bell A, Bergdorf M, et al. Anton 3: twenty microseconds of molecular dynamics simulation before lunch. SC ’21: The International Conference for High Performance Computing, Networking, Storage and Analysis. ACM; 2021. doi:10.1145/3458817.3487397
2. Shaw DE, Grossman JP, Bank JA, Batson B, Butts JA, Chao JC,  Deneroff MM, et al. Anton 2: Raising the Bar for Performance and Programmability in a Special-Purpose Molecular Dynamics Supercomputer. IEEE; 2014. doi:10.1109/SC.2014.9
3. Lippert RA, Predescu C, Ierardi DJ, Mackenzie KM, Eastwood MP, Dror RO, Shaw DE. Accurate and efficient integration for molecular dynamics simulations at constant temperature and pressure. J Chem Phys. 2013;139: 164106. doi:10.1063/1.4825247
4. Pan AC, Jacobson D, Yatsenko K, Sritharan D, Weinreich TM, Shaw DE. Atomic-level characterization of protein–protein association. Proceedings of the National Academy of Sciences. 2019;116: 4244–4249. doi:10.1073/pnas.1815431116
5. Wang Q, Pechersky Y, Sagawa S, Pan AC, Shaw DE. Structural mechanism for Bruton’s tyrosine kinase activation at the cell membrane. Proceedings of the National Academy of Sciences. 2019;116: 9390–9399. doi:10.1073/pnas.1819301116
6. Desmond/GPU Benchmark Systems. Available: http://www.deshawresearch.com/downloads/download_trajectory_benchmark2021.cgi