Pruned MoEs (Mixtral-8x7B-Instruct-v0.1) - a Na0s Collection

Na0s 's Collections

Differential transformer

Depth pruned and fine tuned Llama-3.1-8B

Medical Whisper

Pruned MoEs (Mixtral-8x7B-Instruct-v0.1)

Pruned MoEs (Mixtral-8x7B-Instruct-v0.1)

updated 3 days ago

Pruned experts from Mixtral-8x7B-Instruct-v0.1 with respect to the paper "A Provably Effective Method for Pruning Experts in Fine-tuned Sparse MoEs"

Na0s/Mixtral-8x7B-v0.1-instruct-pruned-random-1-experts

Text Generation • Updated Oct 6 • 11
Na0s/Mixtral-8x7B-v0.1-instruct-pruned-random-2-experts

Text Generation • Updated Oct 6 • 10
Na0s/Mixtral-8x7B-v0.1-instruct-pruned-random-3-experts

Text Generation • Updated Oct 6 • 10
Na0s/Mixtral-8x7B-v0.1-instruct-pruned-random-4-experts

Text Generation • Updated Oct 6 • 16
Na0s/Mixtral-8x7B-v0.1-instruct-l2-norm-post-Gates-SFT-pruned-4-experts

Text Generation • Updated Oct 6 • 13
Na0s/Mixtral-8x7B-v0.1-instruct-l2-norm-post-Gates-SFT-pruned-3-experts

Text Generation • Updated Oct 6 • 11
Na0s/Mixtral-8x7B-v0.1-instruct-l2-norm-post-Gates-SFT-pruned-2-experts

Text Generation • Updated Oct 5 • 12
Na0s/Mixtral-8x7B-v0.1-instruct-l2-norm-post-Gates-SFT-pruned-1-experts

Text Generation • Updated Oct 6 • 7
Na0s/Mixtral-8x7B-Instruct-v0.1-exhaustive-LoRA

Text Generation • Updated 3 days ago • 13
Na0s/Mixtral-8x7B-Instruct-v0.1-LoRA-on-Gates

Text Generation • Updated Sep 26 • 13