Proud of being Poles - read the new issue of the Coopernicus Quarterly!
Knowledge article main photo
Role of Supercomputers in Computational Chemistry: Molecular Level Seen Through the prism of light of In Silico Experiments

C:\Users\anetka\Desktop\Coopernicus\TOC.png

In our daily lives, we do not often encounter the term “supercomputer” directly. However, these computational systems help us with many not always obvious, behind-the-scenes activities. From checking the weather forecast on a smartphone app, designing new car models, fighting against diseases to obtain new drugs, supercomputers are involved. The meaning of this term (and the associated phrase “high-performance computing,” or HPC) has evolved over the years, but it has always referred to computers that far exceed home personal units in the case of computational power.

How do we measure the computational capabilities of HPC–class computers? The primary unit is the number of floating-point operations per second, abbreviated as “flops”. In simple terms, this is the number of multiplications or additions of real (non-integer) values. It  was not until the 1980s that processors could easily perform such calculations, as they began to be equipped with floating-point arithmetic units. Since 1993, the TOP500 [1] list has been maintained, ranking the world’s 500 fastest supercomputers. In 1993, the fastest scientific supercomputer in Los Alamos (USA) reached 60 gigaflops, whereas the current record holder, Frontier from Oak Ridge National Laboratory (USA), surpassed the staggering barrier of 1018 flops, with a performance of 1.2 exaflops (1.2×1018 operations per second), nearly twice as fast as the second-place machine. Interestingly, the first to break the exaflop barrier and hold the speed record is not a single supercomputer but a distributed network of volunteers contributing their (often home) computers to the Folding@Home project, currently reaching 20 exaflops [2]. However, this is only possible due to the specific nature of the research problem, and usually, the use of a single supercomputer is unavoidable.

The essential features that make a machine a potential supercomputer have changed over the years. High-speed Fast connections between processors (nowadays, between individual computing nodes) have always played an important role. For example, the mentioned record-holder, Frontier, uses dedicated network cards with a bandwidth of 100 GB/s [3], compared to the home internet speed of a few dozen MB/s or even the formal bandwidth of a network card in our computer (usually 1 GB/s). What about the speed of the processor itself? Initially, there was a race for the highest clock speed, from the initial few megahertz in the 1980s to 2–3 GHz at the beginning of this century, which remains the case today. So where does the increased computational power come from? The current times have brought the development of multi-core processors, where the same clock speed now supports up to 64 computing units in one package. This development path seems to have slowed  down recently, but graphics cards (GPUs) favored by gamers have entered the game. Quickly calculating the colors of thousands of pixels to create a smooth image in a game requires many specialized cores working in parallel, similar to how matrix multiplication or fast Fourier transform works, fundamental numerical methods of contemporary science. In the early 2000s, standards were developed for programming graphics cores to perform such computational tasks. Most of the computing power of today’s supercomputers comes from GPUs. A good example is Europe’s fastest supercomputer, LUMI, located in Kajaani, Finland [4]. This machine achieves 380 petaflops and has several interesting features. It is powered ecologically by hydroelectric energy, and the heat generated during operation is used for district heating in Kajaani, covering one-fifth of the city’s demand. The power requirements of the current TOP500 list leaders (LUMI – 7 MW, Frontier – 22 MW) are comparable to the energy consumption of a medium–sized city. The LUMI computer is also notable for being built through the collaboration of ten European countries, including Poland, and is available to Polish scientists [5]. The current TOP500 list (as of November 2023) includes four supercomputers from Poland: three from the Academic Computer Centre Cyfronet in Krakow and one from the Poznań Supercomputing and Networking Center, with performances reaching several petaflops.

In concluding this overview of the architecture of contemporary supercomputers, it is important to note the high costs associated with building and maintaining these installations. The top ranks of the TOP500 list are regularly dominated by the world’s strongest economic and scientific countries (USA, Japan, China, and the European HPC consortium). But what about private companies? Anyone who uses modern search engines has undoubtedly wondered at the immense computational power required to return search results for any query almost instantaneously, within a few seconds at most. However, you will not find machines from Google or Amazon on the TOP500 list. Why is that? Scientific HPC and database searching are two different areas with entirely different specifications and functionalities. Comparing them is like trying to determine which athlete is better—a world champion sprinter or a weightlifter? These are completely different sports disciplines, although efficient searching of large databases also uses GPU acceleration [6]. So, let us stay with the “scientific” aspect of HPC and consider how the supercomputer specifically aids primarily chemists who use computational chemistry tools in their research.

The beginning of the 20th century brought the formulation of early quantum theory [7], which became the starting point for modern quantum mechanics. In 1926, Erwin Schrödinger published a scientific paper in which he presented the “Schrödinger equation” [8]. This is one of the fundamental equations of non-relativistic quantum mechanics, which can be solved exactly for only a few cases, one of which is the hydrogen atom [9].

To understand the physicochemical properties of more complex entities such as isolated molecules, proteins or nucleic acids, we can solve the Schrödinger equation approximately and describe the system on the basis of quantum mechanics or molecular mechanics, which is based on classical physics laws [10]. These methods can be generally divided into static methods like the Hartree–Fock method, semi–empirical methods, post-Hartree–Fock methods (e.g., Møller–Plesset perturbation theory – MPn), Coupled Cluster Methods (CC), and Density Functional Theory (DFT), as well as dynamic methods such as classical molecular dynamics, ab initio molecular dynamics, and hybrid methods like quantum mechanics/molecular mechanics (QM/MM) [11 – 20]. The choice of appropriate theoretical tools depends on our research goal and the available computational resources (computational clusters/supercomputers). The applicability of these methods is closely related to advances in high-performance computing (HPC). In the early 20th century, without computers, the methods of theoretical chemistry mentioned above had to wait for times when available computational power would allow their application in in silico experiments. The methods of theoretical chemistry and physics are now widely used in various scientific fields. One important application is Computer-Aided Drug Design (CADD). Using CADD, we can employ various computational chemistry tools to propose promising candidates for further biological assays [21]. Thus, we can use static and dynamic models to characterize the physicochemical properties of molecules or complexes, such as protein-ligand interactions. Chemoinformatics tools like Molinspiration [22] and SwissADME [23] are also used to predict the physicochemical properties and bioactivity of molecules. Another aspect is modeling of catalytic reaction [24] pathways and studying the interactions of molecules on metal surfaces [25]. These are just a few examples of computational chemistry applications. Thanks to theoretical chemistry and physics methods, and the development of HPC, we can describe and understand many processes at the molecular level. This approach can save time and financial resources, and it aligns with the principles of green chemistry, which is crucial for environmental protection. Simulations can be performed for the simplest models, such as in the gas phase, but also considering the solvent (with well–known continuum and discrete solvation models, as well as hybrid models) [26, 27] and in the crystalline phase. We can also simulate phase transitions, and many other phenomena important for rational molecule design and intermolecular interactions. Another significant advancement is the integration of artificial intelligence (AI) in various aspects of our lives, including chemistry. AI can assist chemists by proposing synthesis pathways for new compounds based on available literature [28]. AI’s contribution to computational methods is noteworthy, especially with the development of highly specialized graphics accelerators, which have significantly increased the efficiency of matrix operations that form the basis of artificial neural networks (ANNs). A fundamental publication in this field demonstrated that using an Nvidia GeForce GTX 280 graphics accelerator could speed up computations by up to 70 times compared to a dual-core CPU clocked at 3.16 GHz [29]. Since then, not only specialized GPU accelerators but also processors dedicated to neural networks, like Google’s Tensor Processing Units, have emerged. Some promising applications of ANNs in computational chemistry and molecular physics include force fields parametrization and prediction of physicochemical  features based on geometric descriptors and the chemical constitution of studied systems. The main advantages of these potentials for molecular interactions are (i) relatively low computational complexity and (ii) arbitrarily small errors in obtained interaction energies (and consequently forces acting on atoms) compared to reference often Coupled Cluster Methods. The characteristics of the molecular system description obtained in this way are a direct result of the nature of ANNs and the minimization of the value of the selected loss function [30–32]. In recent years, a new neural network architecture (Transformer model [33]) has emerged, which has led to an increased interest in AI methods, particularly Large Language Models (LLMs) like ChatGPT [34] and Claude [35]. Given the rapid development of ANN architectures, graphical chips, and easier access to large datasets, it is difficult to predict what the future will bring us. 

To conclude this article,  an example of supercomputers’ application in research, consider modeling the binding site of a carbonic anhydrase IX mimic. In order to characterize the binding site, models containing one of the inhibitors of carbonic anhydrase IX [36], acetazolamide (AZM), were constructed, and amino acids interacting with the ligand in the protein were selected. It was noted that the presence of a chalcogen bond (a “relative” of the well-known hydrogen bonds, part of the large family of non-covalent interactions) plays an important role in stabilizing the ligand’s conformation.

C:\Users\anetka\Desktop\Coopernicus\protein_AZM_prezentacja.png

Figure 1. Carbonic anhydrase protein mimic IX with AZM ligand highlighted. The figure was prepared using the VMD program [37] based on PDB data: code 3DC3 [38].

The simulations were performed using static models based on DFT theory [17, 18] and post-Hartree-Fock methods (MP2 and CC) [13–16], as well as dynamic models. The Car-Parrinello Molecular Dynamics (CPMD) [39] scheme was applied, as well as the Path Integrals Molecular Dynamics (PIMD) method, which allows quantum effects (e.g., proton tunneling) in the behavior of atomic nuclei to be considered [40, 41], and the Metadynamics method [42] to explore the free energy surface of the studied systems and determine the depth of energy minima. Not only geometric and energetic parameters were analyzed (using Energy Decomposition Analysis (EDA) [43] and Symmetry-Adapted Perturbation Theory (SAPT) [44]), but also the electronic structure of the studied complexes. For this purpose, the Atoms In Molecules (AIM) theory [45] and the Non-Covalent Interactions index (NCI) [46] were used. The study also analyzed electrostatic potential maps, and the Basis Set Superposition Error (BSSE) [47] was considered in the interaction energy. The application of such diverse methodologies and computational approaches allowed for a very detailed description of the nature of interactions and the evolution of the studied systems over time. It was noted that the inhibitor of Carbonic Anhydrase IX, AZM, exhibits significant conformational freedom as an isolated molecule, whereas in the protein, its lability decreases, and the conformation is stabilized by a network of intermolecular hydrogen bonds, electrostatic interactions, and chalcogen bonding (as mentioned above). A methodological challenge was the application of ab initio molecular dynamics methods to study interactions weaker than conventional hydrogen bonds. It was possible with a careful selection of methodological details. The presented studies, especially the molecular dynamics schemes, require significant CPU time and would not be feasible without the use of the latest supercomputers. They exemplify the operation of two laws concerning the acceleration of computations through parallelization – Amdahl’s law and Gustafson’s law [48]. The first limits the potential acceleration of computations through parallelization, as there is always some (sometimes quite large) part of the code that cannot be sped up this way, limiting the practical gain from increasing the number of processors. The second law has a positive aspect: within a pre-determined time frame for research, parallelization allows us to solve increasingly complex problems. In the early 20th century, we could study protein dynamics on the scale of hundreds of thousands of atoms and hundreds of nanoseconds. Today, it is possible to simulate many millions of atoms for microseconds. What new opportunities and challenges will the future bring? We will find out together with our readers, observing and co–creating it.

Bibliography

[1] https://top500.org

[2] https://stats.foldingathome.org/os

[3] https://www.olcf.ornl.gov/frontier/

[4] https://www.lumi-supercomputer.eu/lumi_supercomputer/

[5] https://guide.plgrid.pl/grants/lumi/

[6] https://research.google/pubs/searching-for-fast-models-on-datacenter-accelerators/

[7] ter Haar D. The Old Quantum Theory. Pergamon Press, 1967. ISBN 0-08-012101-2.

[8] Schrödinger E. Quantisierung als Eigenwertproblem I. Ann. Phys. 1926, 79, 361–376.

[9] Kołos, W. Chemia kwantowa, Wydawnictwo Naukowe PWN, 1978.

[10] Harvey, J. Chemia obliczeniowa, Wydawnictwo Naukowe PWN SA, 2019

[11] Piela, L. Idee chemii kwantowej, Wydawnictwo Naukowe PWN, 2003.

[12] Kołos, W; Sadlej, J. Atom i cząsteczka, Wydawnictwo Naukowo-Techniczne, 1998

[13] Møller, C.; Plesset, M.S. Note on an Approximation Treatment for Many-Electron Systems. Phys. Rev. 1934, 46, 618-622.

[14] Coester, F. Bound states of a many-particle system. Nucl. Phys. 1958, 7, 421-424.

[15] Čižek, J. Calculation of wavefunction components in ursell-type expansion using quantum field theoretical methods. Chem. Phys. 1996, 45, 4256.

[16] Čižek, J.; Paldus, J. Correlation problems in atomic and molecular systems III. Rederivation of the coupled-pair many-electron theory using the traditional quantum chemical methodst. Inter. J. Quantum. Chem. 1971, 5, 539.

[17] Hohenberg, P.; Kohn, W. Inhomogeneous electron gas. Phys. Rev. 1964, 136, B864-B871.

[18] Kohn, W.; Sham, L.J. Self-consistent equations including exchange and correlation effects. Phys. Rev. 1965, 140, A1133-A1138.

[19] Santamaria, R. Molecular dynamics. Springer, 2023.

[20] Warshel A., Levitt M. Theoretical studies of enzymic reactions: Dielectric, electrostatic and steric stabilization of the carbonium ion in the reaction of lysozyme. J. Mol. Biol. 1976, 103, 227-249.

[21] Bukhsh Singh, D. Ed. Computer-Aided Drug Design. Springer Nature Singapore, 2020.

[22] https://www.molinspiration.com/

[23] Daina, A.; Michielin, O.;Zoete, V. SwissADME: a free web tool to evaluate pharmacokinetics, druglikeness and medicinal chemistry friendliness of small molecules. Sci. Rep. 2017, 7, 42717.

[24] Salciccioli, M.; Stamatakis, M.; Caratzoulas, S.; Vlachos, D.G. A review of multiscale modeling of metal-catalyzed reactions: Mechanism development for complexity and emergent behavior. Chem. Eng. Sci. 2011, 66, 4319-4355.

[25] Golibrzuch, K.;Bartels, N. Auerbach, D.J.; Wodtke, A.M. The Dynamics of Molecular Interactions and Chemical Reactions at Metal Surfaces: Testing the Foundations of Theory. Annu. Rev. Phys. Chem. 2015, 66, 399-425.

[26] Kongsted, J.; Mennucci, B. How to Model Solvent Effects on Molecular Properties Using Quantum Chemistry? Insights from Polarizable Discrete or Continuum Solvation Models. J. Phys. Chem. A 2007, 111, 39, 9890-9900.

[27] Pliego, Jr., J.R.; Riveros, J.M. Hybrid discrete-continuum solvation methods. WIREs Comput. Mol. Sci. 2020,10, e1440.

[28] https://www.elsevier.com/solutions/reaxys/predictive-retrosynthesis/

[29] Raina, R.; Madhavan, A.; Ng. A.Y. Large-scale deep unsupervised learning using graphics processors. In Proceedings of the 26th Annual International Conference on Machine Learning (ICML ’09). Association for Computing Machinery, New York, NY, USA, 2009, 873–880.

[30] Käser, S.; Vazquez-Salazar, L.I.; Meuwly, M.; Töpfer, K. Neural network potentials for chemistry: concepts, applications and prospects. Digit. Discov. 2023, 2, 28-58.

[31] Kulichenko, M.; Smith, J.S.; Nebgen, B.; Li, Y.W.; Fedik, N.; Boldyrev, A.I.; Lubbers, N.; Barros, K.; Tretiak, S. The Rise of Neural Networks for Materials and Chemical Dynamics. J. Phys. Chem. Lett. 2021 12 (26), 6227-6243.

[32] Gasteiger, J.; Zupan, J. Neural Networks in Chemistry. Angew. Chem. Int. Ed. Engl. 1993, 32, 503-527.

[33] Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS’17). Curran Associates Inc., Red Hook, NY, USA, 2017, 6000–6010.

[34] https://chatgpt.com

[35] https://claude.ai

[36] Wojtkowiak, K., Michalczyk, M., Zierkiewicz, W.; Jezierska, A.; Panek, J.J. Int. J. Mol. Sci. 2022, 23, 13701.

[37] Humphrey, W.; Dalke, A.; Schulten, K. VMD: Visual molecular dynamics. J. Mol. Graph. 1996, 14, 33-38.

[38] Genis, C.; Sippel, K.H.; Case, N.; Cao, W.; Avvaru, B.S.; Tartaglia, L.J.; Govindasamy, L.; Tu, C.; Agbandje-McKenna, M.; Silverman, D.N.; Rosser, C.J.; McKenna, R. Design of a carbonic anhydrase IX active-site mimic to screen inhibitors for possible anticancer properties. Biochemistry 2009, 48, 1322-1331.

[39] Car, R.; Parrinello, M. Unified Approach for Molecular Dynamics and Density-Functional Theory. Phys. Rev. Lett. 1985, 55, 2471–2474.

[40] Tuckerman, M.E.; Marx, D.; Klein, M.L.; Parrinello, M. Efficient and general algorithms for path integral Car-Parrinello molecular dynamics. J. Chem. Phys. 1996, 104, 5579–5588.

[41] Feynman, R.P. Space-Time Approach to Non-Relativistic Quantum Mechanics. Rev. Mod. Phys. 1948, 20, 367–387.

[42] Laio, A.; Parrinello, M. Escaping free-energy minima. Proc. Natl. Acad. Sci. USA 2002, 99, 12562–12566.

[43] Kitaura, K.; Morokuma, K. A new energy decomposition scheme for molecular interactions within the Hartree-Fock approximation. Int. J. Quantum Chem. 1976, 10, 325–340.

[44] Jeziorski, B.; Moszynski, R.; Szalewicz, K. Perturbation Theory Approach to Intermolecular Potential Energy Surfaces of van der Waals Complexes. Chem. Rev. 1994, 94, 1887–1930.

[45] Bader, R. Atoms in Molecules: A Quantum Theory; International Series of Monographs on Chemistry; Clarendon Press: Oxford, UK, 1994.

[46] Johnson, E.R.; Keinan, S.; Mori-Sánchez, P.; Contreras-García, J.; Cohen, A.J.; Yang, W. Revealing Noncovalent Interactions. J. Am. Chem. Soc. 2010, 132, 6498–6506.

[47] Boys, S.; Bernardi, F. The calculation of small molecular interactions by the differences of separate total energies. Some procedures with reduced errors. Mol. Phys. 1970, 19, 553–566.

[48] Gustafson, J. Reevaluating Amdahl’s Law. Commun. ACM, 1988, 31, 532–533.

Kamil Wojtkowiak
Karol Kułacz
Jarosław J. Panek
Aneta Jezierska
Leave a comment