i2b2 QueryBuilder - 34b
Model Description
This model will generate queries for your i2b2 query builder trained on this dataset for 10 epochs . For evaluation use.
- Do not use as a final research query builder.
- Results may be incorrect or mal-formatted.
- The onus of research accuracy is on the researcher, not the AI model.
Prompt Format
If you are using text-generation-webui, you can download the instruction template i2b2.yaml
Below is an instruction that describes a task.
### Instruction:
{input}
### Response:
```xml
Architecture
nmitchko/i2b2-querybuilder-codellama-34b is a large language model LoRa specifically fine-tuned for generating queries in the i2b2 query builder.
It is based on codellama-34b-hf at 34 billion parameters.
The primary goal of this model is to improve research accuracy with the i2b2 tool. It was trained using LoRA, specifically QLora Multi GPU, to reduce memory footprint.
See Training Parameters for more info This Lora supports 4-bit and 8-bit modes.
Requirements
bitsandbytes>=0.41.0
peft@main
transformers@main
Steps to load this model:
- Load base model (codellama-34b-hf) using transformers
- Apply LoRA using peft
# Sample Code Coming
Training Parameters
The model was trained for or 10 epochs on i2b2-query-data-1.0
i2b2-query-data-1.0 contains only tasks and outputs for i2b2 queries xsd schemas.
| Item | Amount | Units |
|---|---|---|
| LoRA Rank | 64 | ~ |
| LoRA Alpha | 16 | ~ |
| Learning Rate | 1e-4 | SI |
| Dropout | 5 | % |
Training procedure
The following bitsandbytes quantization config was used during training:
- quant_method: QuantizationMethod.BITS_AND_BYTES
- load_in_8bit: False
- load_in_4bit: True
- llm_int8_threshold: 6.0
- llm_int8_skip_modules: None
- llm_int8_enable_fp32_cpu_offload: False
- llm_int8_has_fp16_weight: False
- bnb_4bit_quant_type: nf4
- bnb_4bit_use_double_quant: True
- bnb_4bit_compute_dtype: bfloat16
Framework versions
- PEFT 0.6.0.dev0
- Downloads last month
- 7