metadata

base_model: google/gemma-2-2b-it
language:
  - en
license: gemma
tags:
  - text-generation-inference
  - transformers
  - unsloth
  - gemma2
datasets:
  - paraloq/json_data_extraction
pipeline_tag: text-generation

Gemma-2 2B Instruct fine-tuned on JSON dataset

This model is a Gemma-2 2b model fine-tuned to paraloq/json_data_extraction.

The model has been fine-tuned to extract data from a text according to a json schema.

Prompt

The prompt used during training is:

"""Below is a text paired with input that provides further context. Write JSON output that matches the schema to extract information.

### Input:
{input}

### Schema:
{schema}

### Response:
"""

Using the Model

You can use the model with the transformer library or with the wrapper from [unsloth] (https://unsloth.ai/blog/gemma2), which allows faster inference.

import torch
from unsloth import FastLanguageModel

# Required to avoid cache size exceeded
torch._dynamo.config.accumulated_cache_size_limit = 2048

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = f"bastienp/Gemma-2-2B-it-JSON-data-extration",
    max_seq_length = 2048,
    dtype = torch.float16,
    load_in_4bit = False,
    token = HF_TOKEN_READ,
)

Using the Quantized model (llama.cpp)

The model is supplied in GGFU format in 4bit and 8bit.

Example code with Llamacpp:

from llama_cpp import Llama

llm = Llama.from_pretrained(
    "bastienp/Gemma-2-2B-it-JSON-data-extration",
    filename="*Q4_K_M.gguf", #*Q8_K_M.gguf for the 8 bit version
    verbose=False,
)

Thanks to the google team that provided gemma-2, this model follows the gemma licence, please check it out if you want to use this repository.

Developed by: bastienp
License: gemma
Finetuned from model : google/gemma-2-2b-it