SpectreLynx commited on
Commit
6c64927
1 Parent(s): 67fb966

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +27 -81
README.md CHANGED
@@ -1,8 +1,6 @@
1
  ---
2
  base_model:
3
  - Undi95/Llama-3-Unholy-8B
4
- - Locutusque/llama-3-neural-chat-v1-8b
5
- - ruslanmv/Medical-Llama3-8B-16bit
6
  library_name: transformers
7
  tags:
8
  - mergekit
@@ -18,7 +16,7 @@ datasets:
18
  - MaziyarPanahi/WizardLM_evol_instruct_V2_196k
19
  - ruslanmv/ai-medical-chatbot
20
  model-index:
21
- - name: Medichat-Llama3-8B
22
  results:
23
  - task:
24
  type: text-generation
@@ -34,10 +32,7 @@ model-index:
34
  - type: acc_norm
35
  value: 59.13
36
  name: normalized accuracy
37
- source:
38
- url: >-
39
- https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=sethuiyer/Medichat-Llama3-8B
40
- name: Open LLM Leaderboard
41
  - task:
42
  type: text-generation
43
  name: Text Generation
@@ -51,10 +46,7 @@ model-index:
51
  - type: acc_norm
52
  value: 82.9
53
  name: normalized accuracy
54
- source:
55
- url: >-
56
- https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=sethuiyer/Medichat-Llama3-8B
57
- name: Open LLM Leaderboard
58
  - task:
59
  type: text-generation
60
  name: Text Generation
@@ -69,10 +61,7 @@ model-index:
69
  - type: acc
70
  value: 60.35
71
  name: accuracy
72
- source:
73
- url: >-
74
- https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=sethuiyer/Medichat-Llama3-8B
75
- name: Open LLM Leaderboard
76
  - task:
77
  type: text-generation
78
  name: Text Generation
@@ -86,10 +75,7 @@ model-index:
86
  metrics:
87
  - type: mc2
88
  value: 49.65
89
- source:
90
- url: >-
91
- https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=sethuiyer/Medichat-Llama3-8B
92
- name: Open LLM Leaderboard
93
  - task:
94
  type: text-generation
95
  name: Text Generation
@@ -104,10 +90,7 @@ model-index:
104
  - type: acc
105
  value: 78.93
106
  name: accuracy
107
- source:
108
- url: >-
109
- https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=sethuiyer/Medichat-Llama3-8B
110
- name: Open LLM Leaderboard
111
  - task:
112
  type: text-generation
113
  name: Text Generation
@@ -122,64 +105,36 @@ model-index:
122
  - type: acc
123
  value: 60.35
124
  name: accuracy
125
- source:
126
- url: >-
127
- https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=sethuiyer/Medichat-Llama3-8B
128
- name: Open LLM Leaderboard
129
  language:
130
  - en
131
  ---
132
 
133
- ### Medichat-Llama3-8B
134
 
135
  Built upon the powerful LLaMa-3 architecture and fine-tuned on an extensive dataset of health information, this model leverages its vast medical knowledge to offer clear, comprehensive answers.
136
 
137
  This model is generally better for accurate and informative responses, particularly for users seeking in-depth medical advice.
138
 
139
 
140
- The following YAML configuration was used to produce this model:
141
-
142
- ```yaml
143
-
144
- models:
145
- - model: Undi95/Llama-3-Unholy-8B
146
- parameters:
147
- weight: [0.25, 0.35, 0.45, 0.35, 0.25]
148
- density: [0.1, 0.25, 0.5, 0.25, 0.1]
149
- - model: Locutusque/llama-3-neural-chat-v1-8b
150
- - model: ruslanmv/Medical-Llama3-8B-16bit
151
- parameters:
152
- weight: [0.55, 0.45, 0.35, 0.45, 0.55]
153
- density: [0.1, 0.25, 0.5, 0.25, 0.1]
154
- merge_method: dare_ties
155
- base_model: Locutusque/llama-3-neural-chat-v1-8b
156
- parameters:
157
- int8_mask: true
158
- dtype: bfloat16
159
-
160
- ```
161
-
162
- # Comparision Against Dr.Samantha 7B
163
-
164
- | Subject | Medichat-Llama3-8B Accuracy (%) | Dr. Samantha Accuracy (%) |
165
- |-------------------------|---------------------------------|---------------------------|
166
- | Clinical Knowledge | 71.70 | 52.83 |
167
- | Medical Genetics | 78.00 | 49.00 |
168
- | Human Aging | 70.40 | 58.29 |
169
- | Human Sexuality | 73.28 | 55.73 |
170
- | College Medicine | 62.43 | 38.73 |
171
- | Anatomy | 64.44 | 41.48 |
172
- | College Biology | 72.22 | 52.08 |
173
- | High School Biology | 77.10 | 53.23 |
174
- | Professional Medicine | 63.97 | 38.73 |
175
- | Nutrition | 73.86 | 50.33 |
176
- | Professional Psychology | 68.95 | 46.57 |
177
- | Virology | 54.22 | 41.57 |
178
- | High School Psychology | 83.67 | 66.60 |
179
- | **Average** | **70.33** | **48.85** |
180
-
181
-
182
- The current model demonstrates a substantial improvement over the previous [Dr. Samantha](sethuiyer/Dr_Samantha-7b) model in terms of subject-specific knowledge and accuracy.
183
 
184
  ### Usage:
185
  ```python
@@ -187,7 +142,7 @@ import torch
187
  from transformers import AutoTokenizer, AutoModelForCausalLM
188
 
189
  class MedicalAssistant:
190
- def __init__(self, model_name="sethuiyer/Medichat-Llama3-8B", device="cuda"):
191
  self.device = device
192
  self.tokenizer = AutoTokenizer.from_pretrained(model_name)
193
  self.model = AutoModelForCausalLM.from_pretrained(model_name).to(self.device)
@@ -224,12 +179,3 @@ if __name__ == "__main__":
224
  print(response)
225
 
226
  ```
227
-
228
- ## Quants
229
- Thanks to [Quant Factory](https://huggingface.co/QuantFactory), the quantized version of this model is available at [QuantFactory/Medichat-Llama3-8B-GGUF](https://huggingface.co/QuantFactory/Medichat-Llama3-8B-GGUF),
230
-
231
-
232
- ## Ollama
233
- This model is now also available on Ollama. You can use it by running the command ```ollama run monotykamary/medichat-llama3``` in your
234
- terminal. If you have limited computing resources, check out this [video](https://www.youtube.com/watch?v=Qa1h7ygwQq8) to learn how to run it on
235
- a Google Colab backend.
 
1
  ---
2
  base_model:
3
  - Undi95/Llama-3-Unholy-8B
 
 
4
  library_name: transformers
5
  tags:
6
  - mergekit
 
16
  - MaziyarPanahi/WizardLM_evol_instruct_V2_196k
17
  - ruslanmv/ai-medical-chatbot
18
  model-index:
19
+ - name: Ryeta-0
20
  results:
21
  - task:
22
  type: text-generation
 
32
  - type: acc_norm
33
  value: 59.13
34
  name: normalized accuracy
35
+
 
 
 
36
  - task:
37
  type: text-generation
38
  name: Text Generation
 
46
  - type: acc_norm
47
  value: 82.9
48
  name: normalized accuracy
49
+
 
 
 
50
  - task:
51
  type: text-generation
52
  name: Text Generation
 
61
  - type: acc
62
  value: 60.35
63
  name: accuracy
64
+
 
 
 
65
  - task:
66
  type: text-generation
67
  name: Text Generation
 
75
  metrics:
76
  - type: mc2
77
  value: 49.65
78
+
 
 
 
79
  - task:
80
  type: text-generation
81
  name: Text Generation
 
90
  - type: acc
91
  value: 78.93
92
  name: accuracy
93
+
 
 
 
94
  - task:
95
  type: text-generation
96
  name: Text Generation
 
105
  - type: acc
106
  value: 60.35
107
  name: accuracy
108
+
 
 
 
109
  language:
110
  - en
111
  ---
112
 
113
+ ### Ryeta-0
114
 
115
  Built upon the powerful LLaMa-3 architecture and fine-tuned on an extensive dataset of health information, this model leverages its vast medical knowledge to offer clear, comprehensive answers.
116
 
117
  This model is generally better for accurate and informative responses, particularly for users seeking in-depth medical advice.
118
 
119
 
120
+ # Benchmarks
121
+
122
+ | Subject | Medichat-Llama3-8B Accuracy (%) | |
123
+ |-------------------------|---------------------------------|
124
+ | Clinical Knowledge | 71.70 |
125
+ | Medical Genetics | 78.00 |
126
+ | Human Aging | 70.40 |
127
+ | Human Sexuality | 73.28 |
128
+ | College Medicine | 62.43 |
129
+ | Anatomy | 64.44 |
130
+ | College Biology | 72.22 |
131
+ | High School Biology | 77.10 |
132
+ | Professional Medicine | 63.97 |
133
+ | Nutrition | 73.86 |
134
+ | Professional Psychology | 68.95 |
135
+ | Virology | 54.22 |
136
+ | High School Psychology | 83.67 |
137
+ | **Average** | **70.33** |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
138
 
139
  ### Usage:
140
  ```python
 
142
  from transformers import AutoTokenizer, AutoModelForCausalLM
143
 
144
  class MedicalAssistant:
145
+ def __init__(self, model_name="SpectreLynx/Ryeta-0", device="cuda"):
146
  self.device = device
147
  self.tokenizer = AutoTokenizer.from_pretrained(model_name)
148
  self.model = AutoModelForCausalLM.from_pretrained(model_name).to(self.device)
 
179
  print(response)
180
 
181
  ```