Transformer(Attention is all you need)

Autoregressive LM(GPT) vs Autoencoding LM(BERT) Autoregressive LM: Causal Language Model Autoencoding LM: Masked Language Model Transformer Architecture Tokenizing vs Embedding vs Encoding Tokenizing: process which converts text to token idx Embedding: process which converts Tokenized Words to Vectors Encoding: process which converts embedded Vectors to Sentence Matrix Positional Encoding Positi..

2022.11.17