Given a word sequence, we recognize that some words within it are more closely related with one another than others. This gives rise to the concept of self-attention in which a given word attends to other words in the sequence. Essentially, attention is about representing context by giving weights …