instruction-pretrain's picture

instruction-pretrain

instruction-pretrain

·

https://huggingface.co/papers/2406.14491

DaixuanC45443

AI & ML interests

Synthetic Instructions for Pre-Training

Organizations

None yet

instruction-pretrain's activity

New activity in instruction-pretrain/ft-instruction-synthesizer-collection 21 days ago

Query about the size of dataset

#4 opened 21 days ago by

New activity in instruction-pretrain/finance-Llama3-8B about 1 month ago

How large is the corpus size used for pretraining the finance LLaMA?

#2 opened about 1 month ago by

New activity in instruction-pretrain/general-instruction-augmented-corpora 2 months ago

Kept getting stuck in the failed download of part-1/shard/00004.txt from the general-instruction-augmented-corpora

#12 opened 2 months ago by

commented 2 papers 2 months ago

Instruction Pre-Training: Language Models are Supervised Multitask Learners

Paper • 2406.14491 • Published Jun 20 • 85 •

Instruction Pre-Training: Language Models are Supervised Multitask Learners

Paper • 2406.14491 • Published Jun 20 • 85 •

New activity in AdaptLLM/medicine-tasks 2 months ago

Update README.md

#2 opened 2 months ago by

instruction-pretrain

New activity in AdaptLLM/law-tasks 2 months ago

Update README.md

#2 opened 2 months ago by

instruction-pretrain

New activity in AdaptLLM/finance-tasks 2 months ago

Update README.md

#2 opened 2 months ago by

instruction-pretrain

New activity in instruction-pretrain/finance-Llama3-8B 2 months ago

Update README.md

#1 opened 2 months ago by

New activity in instruction-pretrain/instruction-synthesizer 2 months ago

Can you share the training code for the instruction synthesizer？

#2 opened 4 months ago by

commented 2 papers 3 months ago

Instruction Pre-Training: Language Models are Supervised Multitask Learners

Paper • 2406.14491 • Published Jun 20 • 85 •

Instruction Pre-Training: Language Models are Supervised Multitask Learners

Paper • 2406.14491 • Published Jun 20 • 85 •

New activity in instruction-pretrain/instruction-synthesizer 3 months ago

Update README.md

#3 opened 3 months ago by

New activity in AdaptLLM/law-chat 3 months ago

Update README.md

#2 opened 3 months ago by

instruction-pretrain

New activity in AdaptLLM/finance-chat 3 months ago

Update README.md

#4 opened 3 months ago by

instruction-pretrain

New activity in AdaptLLM/medicine-chat 3 months ago

Update README.md

#5 opened 3 months ago by

instruction-pretrain

New activity in instruction-pretrain/ft-instruction-synthesizer-collection 4 months ago

License

#3 opened 4 months ago by

commented a paper 4 months ago

Instruction Pre-Training: Language Models are Supervised Multitask Learners

Paper • 2406.14491 • Published Jun 20 • 85 •

New activity in instruction-pretrain/InstructLM-1.3B 4 months ago

Thanks for the models and data, I have some questions.

#2 opened 5 months ago by

New activity in instruction-pretrain/ft-instruction-synthesizer-collection 4 months ago

Can you please also share the synthetic pretraining dataset generated with the synthesizer?

#2 opened 5 months ago by