Update README.md
Browse files
README.md
CHANGED
@@ -33,14 +33,13 @@ The following table shows the performance degradation due to quantization:
|
|
33 |
|
34 |
|
35 |
## Use with llama.cpp
|
36 |
-
Install llama.cpp through brew (works on Mac and Linux)
|
37 |
|
|
|
38 |
```bash
|
39 |
brew install llama.cpp
|
40 |
```
|
41 |
|
42 |
-
Invoke the llama.cpp server
|
43 |
-
|
44 |
```bash
|
45 |
$ llama-server \
|
46 |
--hf-repo elyza/Llama-3-ELYZA-JP-8B-GGUF \
|
@@ -48,8 +47,7 @@ $ llama-server \
|
|
48 |
--port 8080
|
49 |
```
|
50 |
|
51 |
-
Call the API using curl
|
52 |
-
|
53 |
```bash
|
54 |
$ curl http://localhost:8080/v1/chat/completions \
|
55 |
-H "Content-Type: application/json" \
|
@@ -64,8 +62,7 @@ $ curl http://localhost:8080/v1/chat/completions \
|
|
64 |
}'
|
65 |
```
|
66 |
|
67 |
-
Call the API using Python
|
68 |
-
|
69 |
```python
|
70 |
import openai
|
71 |
|
@@ -85,7 +82,7 @@ completion = client.chat.completions.create(
|
|
85 |
|
86 |
## Use with Desktop App
|
87 |
|
88 |
-
There are various desktop applications that can handle GGUF models, but here we will introduce how to use the model in the no-code environment LM Studio.
|
89 |
|
90 |
- **Installation**: Download and install [LM Studio](https://lmstudio.ai/).
|
91 |
- **Downloading the Model**: Search for `elyza/Llama-3-ELYZA-JP-8B-GGUF` in the search bar on the home page 🏠, and download `Llama-3-ELYZA-JP-8B-q4_k_m.gguf`.
|
|
|
33 |
|
34 |
|
35 |
## Use with llama.cpp
|
|
|
36 |
|
37 |
+
Install llama.cpp through brew (works on Mac and Linux):
|
38 |
```bash
|
39 |
brew install llama.cpp
|
40 |
```
|
41 |
|
42 |
+
Invoke the llama.cpp server:
|
|
|
43 |
```bash
|
44 |
$ llama-server \
|
45 |
--hf-repo elyza/Llama-3-ELYZA-JP-8B-GGUF \
|
|
|
47 |
--port 8080
|
48 |
```
|
49 |
|
50 |
+
Call the API using curl:
|
|
|
51 |
```bash
|
52 |
$ curl http://localhost:8080/v1/chat/completions \
|
53 |
-H "Content-Type: application/json" \
|
|
|
62 |
}'
|
63 |
```
|
64 |
|
65 |
+
Call the API using Python:
|
|
|
66 |
```python
|
67 |
import openai
|
68 |
|
|
|
82 |
|
83 |
## Use with Desktop App
|
84 |
|
85 |
+
There are various desktop applications that can handle GGUF models, but here we will introduce how to use the model in the no-code environment [LM Studio](https://lmstudio.ai/).
|
86 |
|
87 |
- **Installation**: Download and install [LM Studio](https://lmstudio.ai/).
|
88 |
- **Downloading the Model**: Search for `elyza/Llama-3-ELYZA-JP-8B-GGUF` in the search bar on the home page 🏠, and download `Llama-3-ELYZA-JP-8B-q4_k_m.gguf`.
|