Rotary Position Embedding
A positional scheme that rotates query/key vectors by position.
RoPE encodes position by rotating the query and key vectors in attention, which captures relative position naturally and extrapolates better to longer sequences. It is now the standard positional method in most modern LLMs.