ToolBench-ToolLLaMA-2-7b-GGML

Runtime error

limcheekin commited on Aug 17, 2023

Commit

5041f48

•

1 Parent(s): 35f15e1

feat: updated model to q5_1 as q8_0 is too slow.

Files changed (3) hide show

Dockerfile CHANGED Viewed

@@ -15,7 +15,7 @@ RUN pip install -U pip setuptools wheel && \
 # Download model
 RUN mkdir model && \
-    curl -L https://huggingface.co/s3nh/ToolBench-ToolLLaMA-2-7b-GGML/resolve/main/ToolBench-ToolLLaMA-2-7b.ggmlv3.q8_0.bin -o model/ggmlv3-model.bin
 COPY ./start_server.sh ./
 COPY ./main.py ./

 # Download model
 RUN mkdir model && \
+    curl -L https://huggingface.co/s3nh/ToolBench-ToolLLaMA-2-7b-GGML/resolve/main/ToolBench-ToolLLaMA-2-7b.ggmlv3.q5_1.bin -o model/ggmlv3-model.bin
 COPY ./start_server.sh ./
 COPY ./main.py ./

README.md CHANGED Viewed

@@ -1,5 +1,5 @@
 ---
-title: ToolBench-ToolLLaMA-2-7b-GGML (q8_0)
 colorFrom: purple
 colorTo: blue
 sdk: docker
@@ -15,6 +15,6 @@ tags:
 pinned: false
 ---
-# ToolBench-ToolLLaMA-2-7b-GGML (q8_0)
 Please refer to the [index.html](index.html) for more information.

 ---
+title: ToolBench-ToolLLaMA-2-7b-GGML (q5_1)
 colorFrom: purple
 colorTo: blue
 sdk: docker
 pinned: false
 ---
+# ToolBench-ToolLLaMA-2-7b-GGML (q5_1)
 Please refer to the [index.html](index.html) for more information.

index.html CHANGED Viewed

@@ -1,10 +1,10 @@
 <!DOCTYPE html>
 <html>
   <head>
-    <title>ToolBench-ToolLLaMA-2-7b-GGML (q8_0)</title>
   </head>
   <body>
-    <h1>ToolBench-ToolLLaMA-2-7b-GGML (q8_0)</h1>
     <p>
       With the utilization of the
       <a href="https://github.com/abetlen/llama-cpp-python">llama-cpp-python</a>

 <!DOCTYPE html>
 <html>
   <head>
+    <title>ToolBench-ToolLLaMA-2-7b-GGML (q5_1)</title>
   </head>
   <body>
+    <h1>ToolBench-ToolLLaMA-2-7b-GGML (q5_1)</h1>
     <p>
       With the utilization of the
       <a href="https://github.com/abetlen/llama-cpp-python">llama-cpp-python</a>