論文『Titans: Learning to Memorize at Test Time』を読む。(Transformer2)。

Titans: Learning to Memorize at Test Time https://arxiv.org/pdf/2501.00663 概要 Over more than a decade there has been an extensive research effort of how effectively utilize recurrent models and attentions. While recurrent models aim to compress the data into a fixed-size memory (called hidden sta…