16k and 32k models?
#1
by
this-username-already-exists3
- opened
Hello and thank you for releasing this and other models related to your very interesting paper. I am interested in further comparing your models on some long context tasks and prompting experiments, but the 16k and 32k transformer models are not available. In your paper you mention that you extend the transformer context using NTK, but also that all the models are trained for an additional 50B tokens. Does the additional training include the transformer models? If so, would it be possible to release them?