Transformers: The New Gem of “Deep Learning”
In recent years, Transformers have experienced rapid progress in Natural Language Processing
We spend most of our time communicating in natural language, that is, in unstructured text, whether written or spoken. For this reason, research work has been going on for many years to ensure that machines understand these unstructured texts and extract relevant information from them. It is what is known as Natural Language Processing (NLP), one of the many areas of knowledge in the field of Artificial Intelligence.
Until recently, it was a true challenge for any of the multiple approaches used in NLP to understand the nuances of natural language. But in 2017, the research paper ‘Attention is all you need’ appeared, proposing a new neural network architecture called Transformer. It was definitely a turning point that completely changed the field of NLP.
Without realizing it, we often interact with Transformers. For example, already in 2019 Google introduced Transformers to its search engine to improve results. An example that Google itself explains in this post is when someone typed “2019 brazil traveler to usa need a visa” into their search engine (Figure 1). As humans, it’s easy to understand that this is someone who wants to go from Brazil to the United States…
