File size: 1,079 Bytes
fcf65fa
 
 
 
 
 
 
40d0174
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
---
language:
- hi
metrics:
- bleu
- rouge
---
# Model discription
Hindi Summarization model. It summarizes a hindi paragraph.

# Base model 
- mt5-small

# How to use

    from transformers import AutoTokenizer
    from transformers import AutoModelForSeq2SeqLM, Seq2SeqTrainingArguments, Seq2SeqTrainer

    checkpoint = "Jayveersinh-Raj/hindi-summarizer-small"
    tokenizer = AutoTokenizer.from_pretrained(checkpoint)
    model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint)

    # Input paragraph for summarization
    input_sentence = "<sum> your hindi paragraph"

    # Tokenize the input sentence
    input_ids = tokenizer.encode(input_sentence, return_tensors="pt").to("cuda")

    # Generate predictions
    with torch.no_grad():
       output_ids = model.generate(input_ids, max_new_tokens=200)

    # Decode the generated output
    output_sentence = tokenizer.decode(output_ids[0], skip_special_tokens=True)

    # Print the generated output
    print("Input:", input_sentence)
    print("Summarized:", output_sentence)

# Evaluation
- Rogue1: 0.38
- BLUE: 0.35