appvoid commited on
Commit
ff99345
1 Parent(s): adead96

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -5
README.md CHANGED
@@ -6,10 +6,10 @@ datasets:
6
  - appvoid/no-prompt-15k
7
  pipeline_tag: text-generation
8
  ---
9
- ![palmer](https://huggingface.co/appvoid/palmer-001/resolve/main/palmer.jpeg)
10
- # palmer
11
- ### a better base model
12
- palmer is a series of ~1b parameters language models fine-tuned to be used as base models instead of using custom prompts for tasks. This means that it can be further fine-tuned on more data with custom prompts as usual or be used for downstream tasks as any base model you can get. The model has the best of both worlds: some "bias" to act as an assistant, but also the abillity to predict the next-word from its internet knowledge base. It's a 1.3b llama 2 model so you can use it with your favorite tools/frameworks.
13
 
14
  ### evaluation
15
  |Model| ARC_C| HellaSwag| PIQA| Winogrande|
@@ -20,7 +20,7 @@ palmer is a series of ~1b parameters language models fine-tuned to be used as ba
20
  |no-prompt-1.3b| 0.3157| **0.6022**| 0.7334| 0.5864|
21
  |falcon-rw-1b-instruct-openorca (sota) | **0.3362**| 0.5997| **0.7394**| **0.6148**|
22
 
23
- This model was trained on less than 25% of the dataset yet achieves competitive performance to current sota on open llm leaderboard. Wait for what it's coming!
24
 
25
  ### training
26
  Training took ~5 P100 gpu hours. It was trained on 15,000 gpt-4 shuffled samples. palmer was fine-tuned using lower learning rates ensuring it keeps as much general knowledge as possible.
 
6
  - appvoid/no-prompt-15k
7
  pipeline_tag: text-generation
8
  ---
9
+ ![palmer](https://huggingface.co/appvoid/no-prompt-1.3b/resolve/main/_ccd1a5dd-2ddc-4d5a-8163-fd6d1b39f5f4.jpeg?download=true)
10
+ # no-prompt
11
+ ### a sheared-llama-1.3b fine-tuning
12
+ This model uses an 1.3 billion parameters model as base to be further fine-tuned on the same data as palmer. It works pretty good and even surpasses sota model on `hellaswag`.
13
 
14
  ### evaluation
15
  |Model| ARC_C| HellaSwag| PIQA| Winogrande|
 
20
  |no-prompt-1.3b| 0.3157| **0.6022**| 0.7334| 0.5864|
21
  |falcon-rw-1b-instruct-openorca (sota) | **0.3362**| 0.5997| **0.7394**| **0.6148**|
22
 
23
+ This model was trained on less than 25% of the dataset yet achieves competitive performance to current sota on open llm leaderboard.
24
 
25
  ### training
26
  Training took ~5 P100 gpu hours. It was trained on 15,000 gpt-4 shuffled samples. palmer was fine-tuned using lower learning rates ensuring it keeps as much general knowledge as possible.