End of training

README.md

@@ -2,9 +2,16 @@
 license: other
 base_model: deepseek-ai/deepseek-llm-7b-chat
 tags:
+- alignment-handbook
 - trl
 - dpo
 - generated_from_trainer
+- trl
+- dpo
+- generated_from_trainer
+datasets:
+- self-generate/ds_chat_original_cn_mining_oj_iter0-binarized
+- self-generate/ds_chat_original_cn_rl_oj_iter0-binarized
 model-index:
 - name: ds_chat_sppo_hard_new_iter0_2024-09-17-14.35
   results: []
@@ -16,7 +23,7 @@ should probably proofread and complete it, then remove this comment. -->
 [<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://ml.byteintl.net/experiment/tracking/detail?Id=project_20240915_20321b8f&selectedTrial=run_20240918_0b17b41a)
 # ds_chat_sppo_hard_new_iter0_2024-09-17-14.35
 
-This model is a fine-tuned version of [deepseek-ai/deepseek-llm-7b-chat](https://huggingface.co/deepseek-ai/deepseek-llm-7b-chat) on the None dataset.
+This model is a fine-tuned version of [deepseek-ai/deepseek-llm-7b-chat](https://huggingface.co/deepseek-ai/deepseek-llm-7b-chat) on the self-generate/ds_chat_original_cn_mining_oj_iter0-binarized and the self-generate/ds_chat_original_cn_rl_oj_iter0-binarized datasets.
 It achieves the following results on the evaluation set:
 - Loss: 0.4365
 - Rewards/chosen: 0.0466

config.json

@@ -24,6 +24,6 @@
   "tie_word_embeddings": false,
   "torch_dtype": "bfloat16",
   "transformers_version": "4.42.0",
-  "use_cache": false,
+  "use_cache": true,
   "vocab_size": 102400
 }