Entertainment at it's peak. The news is by your side.

Efficient Transformers: A Survey


[Submitted on 14 Sep 2020]

Safe PDF

Summary: Transformer model architectures maintain garnered suitable hobby lately attributable to
their effectiveness right thru a form of domains cherish language, imaginative and prescient and
reinforcement studying. Within the world of natural language processing for
instance, Transformers maintain turn out to be an essential staple in the up to the moment deep
studying stack. Only in the near past, a dizzying choice of emph{“X-primitive”} fashions maintain
been proposed – Reformer, Linformer, Performer, Longformer, to name about a –
which enhance upon the distinctive Transformer structure, loads of which procure
enhancements around computational and memory emph{effectivity}. With the aim of
serving to the avid researcher navigate this flurry, this paper characterizes a
gorgeous and considerate series of most modern effectivity-flavored “X-primitive” fashions,
providing an organized and comprehensive overview of present work and fashions
right thru more than one domains.

Submission history

From: Yi Tay [view email]

Mon, 14 Sep 2020 20: 38: 14 UTC (431 KB)

Read More

Leave A Reply

Your email address will not be published.