0xroyce
/

Plutus-Meta-Llama-3.1-8B-Instruct-bnb-4bit

@@ -1,202 +1,104 @@
 ---
-base_model: unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit
-library_name: peft
 ---
-# Model Card for Model ID
-<!-- Provide a quick summary of what the model is/does. -->
 ## Model Details
-### Model Description
-<!-- Provide a longer summary of what this model is. -->
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-### Model Sources [optional]
-<!-- Provide the basic links for the model. -->
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-## Uses
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-### Direct Use
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-[More Information Needed]
-### Downstream Use [optional]
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-[More Information Needed]
-### Out-of-Scope Use
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-[More Information Needed]
-## Bias, Risks, and Limitations
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-[More Information Needed]
-### Recommendations
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-## How to Get Started with the Model
-Use the code below to get started with the model.
-[More Information Needed]
-## Training Details
-### Training Data
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-[More Information Needed]
-### Training Procedure
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-#### Preprocessing [optional]
-[More Information Needed]
-#### Training Hyperparameters
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-#### Speeds, Sizes, Times [optional]
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-[More Information Needed]
-## Evaluation
-<!-- This section describes the evaluation protocols and provides the results. -->
-### Testing Data, Factors & Metrics
-#### Testing Data
-<!-- This should link to a Dataset Card if possible. -->
-[More Information Needed]
-#### Factors
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-[More Information Needed]
-#### Metrics
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-[More Information Needed]
-### Results
-[More Information Needed]
-#### Summary
-## Model Examination [optional]
-<!-- Relevant interpretability work for the model goes here -->
-[More Information Needed]
-## Environmental Impact
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-## Technical Specifications [optional]
-### Model Architecture and Objective
-[More Information Needed]
-### Compute Infrastructure
-[More Information Needed]
-#### Hardware
-[More Information Needed]
-#### Software
-[More Information Needed]
-## Citation [optional]
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-**BibTeX:**
-[More Information Needed]
-**APA:**
-[More Information Needed]
-## Glossary [optional]
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-[More Information Needed]
-## More Information [optional]
-[More Information Needed]
-## Model Card Authors [optional]
-[More Information Needed]
-## Model Card Contact
-[More Information Needed]
-### Framework versions
-- PEFT 0.12.0

 ---
+language: en
+license: llama3.1
+tags:
+- llama
+- transformer
+- 8b
+- 4bit
+- instruction-tuning
+- conversational
+- llama3
+- meta
+pipeline_tag: text-generation
+inference: true
+model_creator: 0xroyce
+model_type: LLaMA
+datasets:
+- 0xroyce/Plutus
+base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
 ---
+# NOTE: MODEL IS BEING FINE-TUNED WITH DATEST LATEST
+# Plutus-Meta-Llama-3.1-8B-Instruct-bnb-4bit
+Plutus-Meta-Llama-3.1-8B-Instruct-bnb-4bit is a fine-tuned version of the LLaMA-3.1-8B model, specifically optimized for tasks related to finance, economics, trading, psychology, and social engineering. This model leverages the LLaMA architecture and employs 4-bit quantization to deliver high performance in resource-constrained environments while maintaining accuracy and relevance in natural language processing tasks.
+![Plutus Banner](https://iili.io/djQmWzu.webp)
 ## Model Details
+- **Model Type**: LLaMA
+- **Model Size**: 8 Billion Parameters
+- **Quantization**: 4-bit (bnb, bitsandbytes)
+- **Architecture**: Transformer-based
+- **Creator**: [0xroyce](https://huggingface.co/0xroyce)
+## Training
+Plutus-Meta-Llama-3.1-8B-Instruct-bnb-4bit was fine-tuned on the [**"Financial, Economic, and Psychological Analysis Texts"** dataset](https://huggingface.co/datasets/0xroyce/Plutus), which is a comprehensive collection of 85 influential books out of a planned 398. This dataset covers key areas such as:
+- **Finance and Investment**: Including stock market analysis, value investing, and exchange-traded funds (ETFs).
+- **Trading Strategies**: Focused on technical analysis, options trading, and algorithmic trading methods.
+- **Risk Management**: Featuring quantitative approaches to financial risk management and volatility analysis.
+- **Behavioral Finance and Psychology**: Exploring the psychological aspects of trading, persuasion, and psychological operations.
+- **Social Engineering and Security**: Highlighting manipulation techniques and cybersecurity threats.
+As the dataset contained only 21.36% of its planned content at the time of training, this version of the model is sometimes referred to as the '21% version.' This fine-tuning process enhances the model's ability to generate coherent and contextually relevant text in domains like financial analysis, economic theory, and trading strategies. The 4-bit quantization ensures that the model can be deployed in environments with limited computational resources without compromising performance.
+## Intended Use
+This model is well-suited for a variety of natural language processing tasks within the finance, economics, psychology, and cybersecurity domains, including but not limited to:
+- **Financial Analysis**: Extracting insights and performing sentiment analysis on financial texts.
+- **Economic Modeling**: Generating contextually relevant economic theories and market predictions.
+- **Behavioral Finance Research**: Analyzing and generating text related to trading psychology and investor behavior.
+- **Cybersecurity and Social Engineering**: Studying manipulation techniques and generating security-related content.
+## Performance
+While specific benchmark scores for Plutus-Meta-Llama-3.1-8B-Instruct-bnb-4bit are not provided, the model is designed to offer competitive performance within its parameter range, particularly for tasks involving financial, economic, and security-related data. The 4-bit quantization offers a balance between model size and computational efficiency, making it ideal for deployment in resource-limited settings.
+## Limitations
+Despite its strengths, the Plutus-Meta-Llama-3.1-8B-Instruct-bnb-4bit model has some limitations:
+- **Domain-Specific Biases**: The model may generate biased content depending on the input, especially within specialized financial, psychological, or cybersecurity domains.
+- **Inference Speed**: Although optimized with 4-bit quantization, real-time application latency may still be an issue depending on the deployment environment.
+- **Context Length**: The model has a limited context window, which can affect its ability to process long-form documents or complex multi-turn conversations effectively.
+## How to Use
+You can load and use the model with the following code:
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+tokenizer = AutoTokenizer.from_pretrained("0xroyce/Plutus-Meta-Llama-3.1-8B-Instruct-bnb-4bit")
+model = AutoModelForCausalLM.from_pretrained("0xroyce/Plutus-Meta-Llama-3.1-8B-Instruct-bnb-4bit")
+input_text = "Your text here"
+input_ids = tokenizer(input_text, return_tensors="pt").input_ids
+output = model.generate(input_ids, max_length=50)
+print(tokenizer.decode(output[0], skip_special_tokens=True))
+```
+## Ethical Considerations
+The Plutus-Meta-Llama-3.1-8B-Instruct-bnb-4bit model, like other large language models, can generate biased or potentially harmful content. Users are advised to implement content filtering and moderation when deploying this model in public-facing applications. Further fine-tuning is also encouraged to align the model with specific ethical guidelines or domain-specific requirements.
+## Citation
+If you use this model in your research or applications, please cite it as follows:
+```bibtex
+@misc{0xroyce2024plutus,
+  author = {0xroyce},
+  title = {Plutus-Meta-Llama-3.1-8B-Instruct-bnb-4bit},
+  year = {2024},
+  publisher = {Hugging Face},
+  howpublished = {\\url{https://huggingface.co/0xroyce/Plutus-Meta-Llama-3.1-8B-Instruct-bnb-4bit}},
+}
+```
+## Acknowledgements
+Special thanks to the open-source community and contributors who made this model possible.