How to Take Advantage of the New Disruptive AI Technology Called Transformers

Transformer neural networks are shaking up Artificial Intelligence

July 15, 2021

Starting in 2017, Transformers have facilitated impressive progress in the field of deep learning. Many of us consider Transformers to be the most important development in recent years and with the greatest potential in the area. For this reason, I believe that it is worthwhile for us to be watchful of their progress.

The new normal that changes the way we do NLP

Transformers were introduced in the seminal paper “Attention is all you need” by Vaswani et al. The gist of this paper is to introduce a mechanism called “neural attention”, which has quickly become one of the most influential ideas in deep learning applied to the NLP domain.

The Transformer model architecture at the seminal paper “Attention is all you need” by Vaswani et al.

It can be applied to other domains like computer vision

But also, the same attention mechanisms that make Transformers so effective for language models can be used in other domains, and nowadays…