Natural Language Processing

The transformer is explained in the paper Attention is All You Need by Google Brain in 2017. This paper came with evolution in the field of Natural Language Processing. Many State Of The Art models in NLP built on the top of transformers.

Transformers is one of the topics which people found most complicated and not able to understand.

In this blog, I will give the general intuition on the transformer and I hope by the end of this blog, you will able to get intuition on how the transformer actually works.

Attention is All You Need

A Transformer is basically a type of machine…

In this blog, we will go through a gentle introduction to Inception architecture by Google in 2014 and versions of the Inception network.

The Problems

Before the Inception network came research tries how to make deeper models because the most straightforward way of improving the performance of deep neural networks is by increasing their size but some of the problems researchers faces are :

  • The problem of vanishing gradient arises as the model gets deeper because of which deeper layers not able to train properly and gradient update in deeper layers is not effective. Also, very deep networks are prone to overfitting.

BERT is a state-of-the-art model by Google that came in 2019. In this blog, I will go step by step to finetune the BERT model for movie reviews classification(i.e positive or negative ). Here, I will be using the Pytorch framework for the coding perspective.

BERT is built on top of the transformer (explained in paper Attention is all you Need). If you want to know how the transformer work, you can check out my blog on the Transformer- Attention is all you Need

Before digging into the code let’s first understand how the model works.

Input text sentence would first be tokenized into words, then the special tokens ( [CLS], [SEP]…

Luv Bansal

Student of IIT(BHU) and passionate about Machine Learning and Deep Learning with a keen interest in neural networks, Computer vision, NLP, and GANS

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store