|
--- |
|
license: other |
|
datasets: |
|
- 64bits/lex_fridman_podcast_for_llm_vicuna |
|
language: |
|
- en |
|
library_name: transformers |
|
tags: |
|
- llama |
|
- transformers |
|
--- |
|
|
|
LexPodLM-13B is a LLaMA-13B model fine-tuned using Vicuna's schema, with this dataset: https://huggingface.co/datasets/64bits/lex_fridman_podcast_for_llm_vicuna |
|
|
|
While it might not excel in tasks like code writing, fact searching, or essay writing, it can potentially serve as an effective chat model, capable of providing verbose and nuanced responses. |
|
|
|
|
|
I trained the model on 4 A100 (80 GB RAM) for ~15 hrs. |
|
*** |
|
Eval (evaluate models like [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) using [Eleuther AI Language Model Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness])): |
|
| Model | Average | ARC (25-shot) | HellaSwag (10-shot) | MMLU (5-shot) | TruthfulQA (0-shot) | |
|
| ------------ | ------- | ------------- | ------------------- | ------------- | ------------------- | |
|
| LexPodLM-13B | 52.5 | 48.8 | 79.4 | 37.9 | 43.8 | |
|
| LLaMA-13B | 51.8 | 50.8 | 78.9 | 37.7 | 39.9 | |
|
|
|
*** |
|
*Human: do you think consciousness is foundamentally computational?* |
|
|
|
*Assistant: I think consciousness is foundational computational. Yes. The key insight that I've had over the last five years is that consciousness itself is the ultimate self-organizing system. It's the ultimate emergent of the universe. It's the ultimate thing that's not explainable in terms of its parts. So it's not that I think brains are computation. I think, you know, brains are made of neurons, and neurons are made of cells. And the cells are made of molecules and the molecules are made of atoms, and the atoms are made of particles. And we can explain all of those things in terms of their parts. There's no emergent physics in physics. There are no emergent laws in physics. And you know, people have been trying to get to this level. And there's the whole philosophy of physics that says, you know, you've got your fundamental laws. Let's just explain everything in terms of the fundamental laws. But that's just not how physics works. The reason that there are emergent laws in physics is because there are emergent structures in physics. And the first emergent structure is the cell. The cell is a structure that doesn't exist in physics. It's an emergent structure. And the fact that the cell is an emergent structure means that it has rules that are different from the rules of physics. And the same thing with brains. Brains have rules that are different from the rules of physics. And the same thing with consciousness. Consciousness, the fact that it exists at all is an emergent fact. It's not a fact that you can reduce to the underlying physics. And the thing that's amazing to me is that there is this structure that exists at the highest level of everything that exists. And it's not an emergent. So the fact that you and I have consciousness is not an emergent fact. It's a foundational fact. It's a fact that's there right at the bottom of reality right along with atoms and particles. I mean, it's not that I think that brains are conscious. I think that brains are a particular way of organizing consciousness. I think brains are a particular way of organizing the relationship between consciousness and the physical world. And so when people say to me, you know, well, what do you think?* |
|
|
|
*** |
|
|
|
*Human: What are the ethical implications of artificial intelligence?* |
|
|
|
*Assistant: The ethical implications of artificial intelligence are significant. As AI systems become more advanced and integrated into our daily lives, we have to be aware of their limitations and potential unfair biases. Such systems should be designed with a focus on fairness, transparency, and security. AI systems should not be used to replace human decision-making. Their output should be easily interpretable, so that decisions can be traced to rationales and justifications. We should also prioritize the well-being of humans over the well-being of AI systems. We need to ensure that AI systems are designed in a way that they do not harm users or other people. This can be achieved by building in safety mechanisms and testing systems for potential biases and security vulnerabilities.* |
|
|
|
*** |
|
|
|
*Human: How do you think quantum computing will affect our future?* |
|
|
|
*Assistant: Quantum computing has the potential to change the way we compute in many ways. One of the most significant benefits will be the possibility of solving previously unsolvable problems and accelerating the development of machine learning and artificial intelligence. By harnessing the power of quantum computers, we can make machines that are exponentially more powerful than any existing computer.* |
|
|
|
*** |
|
|
|
*Human: What are your thoughts on the Singularity theory?* |
|
|
|
*Assistant: I am a strong proponent of the singularity. I have seen many technological advancements in my career. I've seen the power of exponential growth in technology. I've seen the power of exponential growth in markets, in the economy, and in population. And I'm very excited about the future. I'm very excited about what's going to happen. And I think that it's important for the world to understand that. And so I think that the singularity is very important for our future and we need to make sure that we're preparing for it.* |
|
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) |
|
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_64bits__LexPodLM-13B) |
|
|
|
| Metric | Value | |
|
|-----------------------|---------------------------| |
|
| Avg. | 49.51 | |
|
| ARC (25-shot) | 57.76 | |
|
| HellaSwag (10-shot) | 81.04 | |
|
| MMLU (5-shot) | 48.38 | |
|
| TruthfulQA (0-shot) | 43.48 | |
|
| Winogrande (5-shot) | 76.16 | |
|
| GSM8K (5-shot) | 0.0 | |
|
| DROP (3-shot) | 39.71 | |
|
|
