Delete README.md.md

README.md.md

@@ -1,123 +0,0 @@
-
-  # Fine-tuned Mistral Model for Multi-Document Summarization  
-  This model a fine-tuned model based on [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on 
-   [multi_x_science_sum](https://huggingface.co/datasets/multi_x_science_sum) dataset.
-
-  ## Model description
-
-Mistral-7B-multixscience-finetuned is finetuned on multi_x_science_sum
-dataset in order to extend the capabilities of the original 
-Mistral model in multi-document summarization tasks. 
-The fine-tuned model leverages the power of Mistral model fundation, 
-adapting it to synthesize and summarize information from 
-multiple documents efficiently.
-
-## Training and evaluation dataset
-
-Multi_x_science_sum is a large-scale multi-document 
-summarization dataset created from scientific articles. 
-Multi-XScience introduces a challenging multi-document 
-summarization task: writing the related-work section of a 
-paper based on its abstract and the articles it references.
-
-* [paper](https://arxiv.org/pdf/2010.14235.pdf)
-* [Source](https://huggingface.co/datasets/multi_x_science_sum)
-
-The training and evaluation datasets were uniquely generated 
-to facilitate the fine-tuning of the model for
-multi-document summarization, particularly focusing on 
-generating related work sections for scientific papers. 
-Using a custom-designed prompt-generation process, the dataset
-is created to simulate the task of synthesizing related work 
-sections based on a given paper's abstract and the abstracts 
-of its referenced papers.
-
-### Dataset Generation process
-
-The process involves generating prompts that instruct the 
-model to use the abstract of the current paper along with 
-the abstracts of cited papers to generate a new related work 
-section. This approach aims to mimic the real-world scenario
-where a researcher synthesizes information from multiple 
-sources to draft the related work section of a paper.
-
-* **Prompt Structure:** Each data point consists of an instructional prompt that includes:
-  
-  * The abstract of the current paper.
-  * Abstracts from cited papers, labeled with unique identifiers.
-  * An expected model response in the form of a generated related work section.
-
-### Prompt generation Code
-
-```
-def generate_related_work_prompt(data):
-    prompt = "[INST] <<SYS>>\n"
-    prompt += "Use the abstract of the current paper and the abstracts of the cited papers to generate new related work.\n"
-    prompt += "<</SYS>>\n\n"
-    prompt += "Input:\nCurrent Paper's Abstract:\n- {}\n\n".format(data['abstract'])
-    prompt += "Cited Papers' Abstracts:\n"
-    for cite_id, cite_abstract in zip(data['ref_abstract']['cite_N'], data['ref_abstract']['abstract']):
-        prompt += "- {}: {}\n".format(cite_id, cite_abstract)
-    prompt += "\n[/INST]\n\nGenerated Related Work:\n{}\n".format(data['related_work'])
-    return {"text": prompt}
-```
-The dataset generated through this process was used to train 
-and evaluate the finetuned model, ensuring that it learns to 
-accurately synthesize information from multiple sources into 
-cohesive summaries.
-
-## Training hyperparameters
-
-The following hyperparameters were used during training:
-```
-learning_rate: 2e-5
-train_batch_size: 4
-eval_batch_size: 4
-seed: 42
-optimizer: adamw_8bit
-num_epochs: 5
-```
-## Usage
-
-```
-import torch
-from transformers import AutoModelForCausalLM, AutoTokenizer
-from peft import PeftConfig, PeftModel
-
-base_model = "mistralai/Mistral-7B-v0.1"
-adapter = "OctaSpace/Mistral7B-fintuned"
-
-# Load tokenizer
-tokenizer = AutoTokenizer.from_pretrained(
-    base_model,
-    add_bos_token=True,
-    trust_remote_code=True,
-    padding_side='left'
-)
-
-# Create peft model using base_model and finetuned adapter
-config = PeftConfig.from_pretrained(adapter)
-model = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path,
-                                             load_in_4bit=True,
-                                             device_map='auto',
-                                             torch_dtype='auto')
-model = PeftModel.from_pretrained(model, adapter)
-
-device = "cuda" if torch.cuda.is_available() else "cpu"
-model.to(device)
-model.eval()
-
-# Prompt content:
-messages = [] # Put here your related work generation instruction
-
-input_ids = tokenizer.apply_chat_template(conversation=messages,
-                                          tokenize=True,
-                                          add_generation_prompt=True,
-                                          return_tensors='pt').to(device)
-summary_ids = model.generate(input_ids=input_ids, max_new_tokens=512, do_sample=True, pad_token_id=2)
-summaries = tokenizer.batch_decode(summary_ids.detach().cpu().numpy(), skip_special_tokens = True)
-
-# Model response: 
-print(summaries[0])
-
-```