Why Is AlphaFold 2 So Powerful?

AlphaFold 2 is an engineering feat. It is not simply an incremental improvement over AlphaFold 1, but rather a complete overhaul of the structural prediction approach. The new method includes many innovative components that, when combined, allow it to achieve atomic-level accuracy for the first time ever in protein structural modeling. This is a significant milestone in the field.

Next, I will discuss some of the most significant new components of AlphaFold 2 in comparison to AlphaFold 1 (in my opinion).

1. End-to-end learning.      In AlphaFold 1, the deep learning model is used to learn how to calculate the inter-residue distance matrix. This distance matrix is then used as input to a differentiable model, which derives the protein geometry through gradient descent optimization. However, this portion of the solution, from the distance matrix to the final geometry model, is not "learned" in the traditional sense of machine learning. In contrast, AlphaFold 2 incorporates a more comprehensive "learning" process, in which the entire solution, from the input features to the final geometry model, is learned through deep learning. As a general rule, machine learning models tend to perform better when more of the solution can be learned through the training process.

2. Replacing CNN with a Transformer-like architecture.      AlphaFold 1 used a convolutional neural network (CNN) architecture to derive the distance matrix from the input features. In AlphaFold 2, this part of the model has been replaced with a Transformer-like architecture called Evoformer. However, it should be noted that Evoformer is not exactly a Transformer, as it handles a matrix rather than a vector. Nonetheless, the basic idea is similar. It is becoming increasingly common for Transformers to replace CNNs in deep learning because they tend to perform better. However, it takes a significant amount of engineering expertise and training data to successfully build and train a Transformer-like architecture.

3. A RNN-like decoder (Structural Module).      In AlphaFold 2, the Evoformer serves as an encoder, and is accompanied by a decoder called the Structural Module. Upon closer examination, the Structural Module appears to be similar to a recurrent neural network (RNN). It is important to note that there are many components within both the Evoformer and the Structural Module that were likely carefully crafted, tested, and refined, each likely contributing to an incremental improvement in the model's overall performance. It took a great deal of insight, engineering talent, and effort to envision, implement, and test these components within the complex architecture, but I will not delve into the details here.

4. Iterations between Evoformer and Structural Module ("Recycling").      The coupling of the Evoformer and Structural Module in AlphaFold 2 is a clever idea that mimics the step-by-step folding of a protein sequence in vitro. This is achieved naturally through the encoding and decoding process. It is also worth noting that both the Evoformer and Structural Module contain multiple blocks (48 blocks in the Evoformer and 8 blocks in the Structural Module). The "recycling" between these two modules can be thought of as representing major conformational changes in the protein as it folds, while the changes within each individual block may be seen as representing relatively minor and local adjustments following a major conformational change.

5. Creative data augmentation.      One of the challenges DeepMind faced in developing AlphaFold 2 was the need for a large amount of data to train the more sophisticated network architecture. They realized that the existing protein structures in the Protein Data Bank (PDB) were not sufficient for this purpose. To address this issue, they implemented a data augmentation strategy inspired by the "noisy student" self-distillation approach. This involved using the trained network to generate predicted structures from Uniclust30, creating a new dataset. A higher quality portion of this new dataset was then used to create an augmentation set, which was used to further improve the training of the network.

These are just a few of the major new components that make AlphaFold 2 so impressive. There are many other features that I have not covered here to avoid using too much technical jargon, and possibly many more that I am not aware of. It is safe to say that AlphaFold 2 is an incredible feat of deep learning engineering that has led to one of the most significant achievements in structural modeling to date.

Read next: Five Ways of Using AlphaFold 2

-- About the author

Need assistance in your AlphaFold or structural bioinformatics project? We may be able to help. Take a look at the intro to our bioinformatician team, see some of the advantages of using our team's help here, and check out our FAQ page!

Send us an inquiry, chat with us online (during our business hours 9-5 Mon-Fri U.S. Central Time), or reach us in other ways!