Update README.md

README.md

@@ -15,192 +15,82 @@ The **Llama-3-instruction-constructionsafety-layertuning** model is a fine-tuned
 Llama-3-instruction-constructionsafety-layertuning model is contined pretrained model based on beomi/Llama-3-KoEn-8B-Instruction-preview.
 
 The training was conducted based on the QA datasets and RAW data of Constrution Safety Guidelines provided by the Korea Ocuupational Safety and Health Agency(KOSHA).
-The training was conducted using full parameter tuning, utilizing 2xA100GPU(80GB). Approximately 11,000 data were used for the training process.
-
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
 
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
+The training was conducted using full parameter tuning, utilizing 2xA100GPU(80GB). Approximately 11,000 data were used for the training process.
 
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+After fine-tuning the entire layers, layers 0, 30, and 31 were replaced with parameters from the base model. This was done as a precautionary measure to prevent errors resulting from training on raw data.
 
-[More Information Needed]
 
-### Out-of-Scope Use
+## Simple Use
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+```
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
 
-[More Information Needed]
+model_name = "juungwon/Llama-3-instruction-constructionsafety-layertuning"
+tuned_model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    token=access_token,
+    torch_dtype="auto",
+    device_map="auto",
+)
 
-## Bias, Risks, and Limitations
+tokenizer = AutoTokenizer.from_pretrained(model_name, token=access_token)
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+tokenizer.pad_token = tokenizer.eos_token
+pipe = pipeline("text-generation", model=tuned_model, tokenizer = tokenizer, torch_dtype=torch.bfloat16, device_map="auto")
 
-[More Information Needed]
+# We use the tokenizer's chat template to format each message - see https://huggingface.co/docs/transformers/main/en/chat_templating
+messages = [
+    {
+        "role": "system",
+        "content": "친절한 건설안전전문가로서 상대방의 요청에 최대한 '자세하고' 친절하게 답하자. 모든 대답은 한국어(Korean)으로 대답해줘.",
+    },
+    {"role": "user", "content": "흙막이 가시설 공사시 작업자 안전조치에 대해서 10가지 정도 알려주세요."},
+]
+prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+outputs = pipe(prompt, max_new_tokens=1024, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
+print(outputs[0]["generated_text"])
 
-### Recommendations
+```
+## Output
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+```
+<|begin_of_text|><|start_header_id|>system<|end_header_id|>
 
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+친절한 건설안전전문가로서 상대방의 요청에 최대한 '자세하고' 친절하게 답하자. 모든 대답은 한국어(Korean)으로 대답해줘.<|eot_id|><|start_header_id|>user<|end_header_id|>
 
-## How to Get Started with the Model
+흙막이 가시설 공사시 작업자 안전조치에 대해서 10가지 정도 알려주세요.<|eot_id|><|start_header_id|>assistant<|end_header_id|>
 
-Use the code below to get started with the model.
+1. 공사 중에는 현장사무실을 설치하여 공사관련 기록, 작업지휘, 작업자 대기, 안전교육, 안전점검 등의 업무를 수행할 수 있어야 한다. 2. 공사 전·후에는 주변의 지반침하, 지하수위, 지하 매설물, 지표면의 이완, 지반의 이완, 지표면의 균열, 노면의 이상 유무, 도로 시설물 등의 이상 유무를 확인하여야 한다. 3. 설계도서, 시방서, 안전보건규칙, 안전보건규칙 및 관련법규, 안전보건규칙과 관련된 지침, 산업안전보건기준에 관한 규칙을 검토하여 안전대책을 수립하여야 한다. 4. 흙막이 가시설 공사 시에는 작업자의 추락방지를 위하여 안전대, 안전모, 안전화 등 개인보호구를 착용하여야 한다. 5. 흙막이 가시설 공사 시에는 근골격계질환 예방을 위하여 적절한 휴식시간을 제공하여야 한다. 6. 흙막이 가시설 공사 시에는 작업자의 건강관리를 위하여 작업환경을 개선하고 적절한 휴식공간을 마련하여야 한다. 7. 흙막이 가시설 공사 시에는 작업자의 산업재해 예방을 위하여 안전교육, 안전시설, 안전장비를 마련하여야 한다. 8. 흙막이 가시설 공사 시에는 작업자의 안전을 위하여 안전작업계획을 수립하여야 한다. 9. 흙막이 가시설 공사 시에는 작업자의 안전을 위하여 토질, 지하수위, 토층, 매설물, 인접구조물, 지하수위, 지표면의 이상 유무, 도로 시설물 등의 이상 유무를 확인하여야 한다. 10. 흙막이 가시설 공사 시에는 작업자의 안전을 위하여 작업자 1인당 1개의 안전모, 안전화, 안전대 등 개인보호구를 착용하여야 한다.
 
-[More Information Needed]
+```
 
-## Training Details
 
 ### Training Data
-
+Training Data will be provided upon requests.
 <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
 
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
 
 **BibTeX:**
+@article{llama3cs-layertuning,
+  title={Llama-3-instruction-constructionsafety-layertuning},
+  author={L, Jungwon, A, Seungjun},
+  year={2024},
+  url={https://huggingface.co/juungwon/Llama-3-instruction-constructionsafety-layertuning}
+}
+
+@article{llama3koen,
+  title={Llama-3-KoEn},
+  author={L, Junbum},
+  year={2024},
+  url={https://huggingface.co/beomi/Llama-3-KoEn-8B}
+}
+
+@article{llama3modelcard,
+  title={Llama 3 Model Card},
+  author={AI@Meta},
+  year={2024},
+  url = {https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md}
+}
 
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]