yzhangcs (Yu Zhang)

upvoted a paper 2 days ago

Qwen2-Audio Technical Report

Paper • 2407.10759 • Published Jul 15 • 55

upvoted a paper about 2 months ago

Gated Linear Attention Transformers with Hardware-Efficient Training

Paper • 2312.06635 • Published Dec 11, 2023 • 6

upvoted a collection about 2 months ago

GSA

Collection

3 items • Updated 5 days ago • 2

upvoted a paper about 2 months ago

Gated Slot Attention for Efficient Linear-Time Sequence Modeling

Paper • 2409.07146 • Published Sep 11 • 19

upvoted a paper 2 months ago

Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler

Paper • 2408.13359 • Published Aug 23 • 21

upvoted 2 articles 3 months ago

Article

Cosmopedia: how to create large-scale synthetic data for pre-training Large Language Models

Mar 20

• 65

Article

SmolLM - blazingly fast and remarkably powerful

Jul 16

• 258

upvoted a paper 3 months ago

Fast Matrix Multiplications for Lookup Table-Quantized LLMs

Paper • 2407.10960 • Published Jul 15 • 11

upvoted 4 papers 5 months ago

Parallelizing Linear Transformers with the Delta Rule over Sequence Length

Paper • 2406.06484 • Published Jun 10 • 3

Scalable MatMul-free Language Modeling

Paper • 2406.02528 • Published Jun 4 • 10

Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

Paper • 2405.21060 • Published May 31 • 63

Block Transformer: Global-to-Local Language Modeling for Fast Inference

Paper • 2406.02657 • Published Jun 4 • 36

upvoted 2 papers 6 months ago

Simple linear attention language models balance the recall-throughput tradeoff

Paper • 2402.18668 • Published Feb 28 • 18

Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models

Paper • 2401.04658 • Published Jan 9 • 24

upvoted an article 6 months ago

Article

Introduction to State Space Models (SSM)

By

•

Jul 19

• 87

upvoted 5 papers 7 months ago

Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length

Paper • 2404.08801 • Published Apr 12 • 63

Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention

Paper • 2404.07143 • Published Apr 10 • 103

Yu Zhang

AI & ML interests

Organizations

yzhangcs's activity

Qwen2-Audio Technical Report

Gated Linear Attention Transformers with Hardware-Efficient Training

GSA

Gated Slot Attention for Efficient Linear-Time Sequence Modeling

Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler

Cosmopedia: how to create large-scale synthetic data for pre-training Large Language Models

SmolLM - blazingly fast and remarkably powerful

Fast Matrix Multiplications for Lookup Table-Quantized LLMs

Parallelizing Linear Transformers with the Delta Rule over Sequence Length

Scalable MatMul-free Language Modeling

Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

Block Transformer: Global-to-Local Language Modeling for Fast Inference

Simple linear attention language models balance the recall-throughput tradeoff

Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models

Introduction to State Space Models (SSM)

HGRN2: Gated Linear RNNs with State Expansion

Pre-training Small Base LMs with Fewer Tokens

TransformerFAM: Feedback attention is working memory

Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length

Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention