README.md

---
base_model: google/gemma-2-2b-it
language:
- en
license: gemma
tags:
- text-generation-inference
- transformers
- unsloth
- gemma2
datasets:
- paraloq/json_data_extraction
library_name: peft
---

# Gemma-2 2B Instruct fine-tuned on JSON dataset

This model is a Gemma-2 2b model fine-tuned to paraloq/json_data_extraction.

The model has been fine-tuned to extract data from a text according to a json schema.
## Prompt

The prompt used during training is:
```py
"""Below is a text paired with input that provides further context. Write JSON output that matches the schema to extract information.

### Input:
{input}

### Schema:
{schema}

### Response:
"""
```

## Using the Model

You can use the model with the transformer library or with the wrapper from [unsloth] (https://unsloth.ai/blog/gemma2), which allows faster inference.

```py
import torch
from unsloth import FastLanguageModel

# Required to avoid cache size exceeded
torch._dynamo.config.accumulated_cache_size_limit = 2048

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = f"bastienp/Gemma-2-2B-it-JSON-data-extration",
    max_seq_length = 2048,
    dtype = torch.float16,
    load_in_4bit = False,
    token = HF_TOKEN_READ,
)
```

## Using the Quantized model (llama.cpp)

The model is supplied in GGFU format in 4bit and 8bit.

Example code with Llamacpp:
```py
from llama_cpp import Llama

llm = Llama.from_pretrained(
    "bastienp/Gemma-2-2B-it-JSON-data-extration",
    filename="*Q4_K_M.gguf", #*Q8_K_M.gguf for the 8 bit version
    verbose=False,
)
```

The base model used for fine-tuning is google/gemma-2-2b-it. This repository is **NOT** affiliated with Google.

Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms. 

- **Developed by:** bastienp
- **License:** gemma
- **Finetuned from model :** google/gemma-2-2b-it