Create README.md

README.md

@@ -0,0 +1,58 @@
+---
+base_model: daisd-ai/anydef-orpo-v2
+tags:
+- entity linking
+datasets:
+- arynkiewicz/anydef-kilt-tasks-v2
+model-index:
+- name: daisd-ai/anydef-v2-linear-W4A16
+  results: []
+license: apache-2.0
+inference: false
+---
+
+## Introduction
+
+This model is quantized version of linear merge of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) and [daisd-ai/anydef-orpo-v2](https://huggingface.co/daisd-ai/anydef-orpo-v2).
+
+## Merging
+
+Models were merged to improve quality of the final model ([idea](https://www.reddit.com/r/LocalLLaMA/comments/1fyx27y/im_pretty_happy_with_how_my_method_worked_out/)) and prevent huge losses during quantization. Merging was done using [mergekit](https://github.com/arcee-ai/mergekit) with following spec: 
+```yaml
+models:
+  - model: mistralai/Mistral-7B-v0.1
+    parameters:
+      weight: 0.3
+  - model: daisd-ai/anydef-orpo-v2
+    parameters:
+      weight: 0.7
+merge_method: linear
+dtype: bfloat16
+```
+
+## Quantization
+
+The quantization was applied using [LLM Compressor](https://github.com/vllm-project/llm-compressor) with 512 random examples from [anydef-kilt-tasks-v2](https://huggingface.co/datasets/daisd-ai/anydef-kilt-tasks-v2) dataset.
+We tested other number of examples, but did not see noticeable improvement with higher number of examples during quantization.
+
+## Inference
+
+For inference code you can check our [github](https://github.com/daisd-ai/universal-el).
+
+## Benchmarks results
+
+Precision (%):
+| Dataset   |  anydef-v2  | anydef-v2-quant (this)   |
+|------------|------------|------------|
+| RSS-500 | 66.89| 64.90|
+| ISTEX-1000| 85.82|  84.33|
+| Reuters-128| 64.88| 68.28|
+| TweekiGold| 75.93| 75.93|
+
+Retrieval rate (%):
+| Dataset   |  anydef-v2  | anydef-v2-quant (this)   |
+|------------|------------|------------|
+| RSS-500 | 84.11| 83.44|
+| ISTEX-1000| 97.76|   97.31|
+| Reuters-128| 83.33| 83.87|
+| TweekiGold| 91.67| 91.44|