gouthamsk/mistral_c_coder

README.md

@@ -1,40 +1,50 @@
+---
+license: apache-2.0
+library_name: peft
+tags:
+- trl
+- sft
+- generated_from_trainer
+base_model: mistralai/Mistral-7B-Instruct-v0.2
+datasets:
+- generator
+model-index:
+- name: mistral_embedded_c
+  results: []
+---
+
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment. -->
-![This is an alt text.](http://biboxlabs.com/wp-content/uploads/2021/07/bibox-labs-logo.png 'Logo.')
 
-For more Info vist [Biboxlabs](https://biboxlabs.com/) or [Red Nerd](https://www.therednerds.com/)
 # mistral_embedded_c
 
 This model is a fine-tuned version of [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) on the generator dataset.
-It has been fine-tuned using LoRa (Long Range) technology, enhancing its capabilities for applications requiring long-range communication.
 
 ## Model description
-The `mistral_embedded_c` model is specifically designed for generating accurate embedded C code targeting microcontrollers such as esp32, pic18f, pic10f, and 8051 MCU. It is a fine-tuned variant of the Mistral-7B-Instruct-v0.2 model, customized to cater to the needs of embedded systems development.
 
+More information needed
 
 ## Intended uses & limitations
-The model can be utilized for prompting to generate precise embedded C code suitable for a variety of microcontrollers including esp32, pic18f, pic10f, and 8051 MCU. However, it should be noted that while the model is trained on a diverse dataset encompassing assembly and C code for these microcontrollers, its performance may vary depending on the complexity and specificity of the task.
 
+More information needed
 
 ## Training and evaluation data
 
-The model is trained of the mixed dataset which contains assembly and c code for esp32, pic18f, pic10f, and 8051 MCU.
-Training Data is specifically curated for specific purposes to evaluate the embedded c code. 
+More information needed
 
 ## Training procedure
-The training was done on an A100-40GB accelerator.
 
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 0.0002
-- train_batch_size: 4
+- learning_rate: 0.0003
+- train_batch_size: 6
 - eval_batch_size: 8
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: constant
 - lr_scheduler_warmup_steps: 0.03
-- training_steps: 300
+- training_steps: 250
 
 ### Training results
 

adapter_config.json

@@ -19,8 +19,8 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "v_proj",
-    "q_proj"
+    "q_proj",
+    "v_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a5b582d51a443321ddf95a24b85586415f9811970e889dc150e572b57f9807ab
+oid sha256:94c21bffc638995d4db56e042a6a362871c4eb542c0d2b348ec113e64c20a4e8
 size 109069176

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:74bfa0c4f571ce7c6291893b7d4a6e8285deb6607e3e98773d1dafefdc0a0757
+oid sha256:aa3964f56281ec4e7750b2236aea67547bd036116947cf99c73ba51f320f2718
 size 4920