SSamDav

joined 1 year ago

Retentive Network: A Successor to Transformer for Large Language Models in c/[email protected]

[–] [email protected] 1 points 1 year ago

Would love to now how it compares with hyenna on the LRA.

Extending Context Window of Large Language Models via Positional Interpolation in c/[email protected]

[–] [email protected] 1 points 1 year ago (1 children)

One cool thing about this work is that there was a concurrent discussion in twitter about the proposed method. From different authors.