Layer Normalization
Normalizing across features within each token.
Layer normalization rescales the activations of each individual example across its features, stabilizing training without depending on batch statistics. It is the normalization of choice inside Transformers.