Upload README.md

README.md

@@ -29,7 +29,7 @@
 
 ## Introduction 
 
-InternLM3 has open-sourced an 8 billion parameter instruction model, InternLM3-8B-Instruct, designed for general-purpose usage and advanced reasoning. This model has the following characteristics:
+InternLM3 has open-sourced an 8-billion parameter instruction model, InternLM3-8B-Instruct, designed for general-purpose usage and advanced reasoning. This model has the following characteristics:
 
 - **Enhanced performance at reduced cost**: 
 State-of-the-art performance on reasoning and knowledge-intensive tasks surpass models like Llama3.1-8B and Qwen2.5-7B. Remarkably, InternLM3 is trained on only 4 trillion high-quality tokens, saving more than 75% of the training cost compared to other LLMs of similar scale. 
@@ -60,16 +60,17 @@ We conducted a comprehensive evaluation of InternLM using the open-source evalua
 | Coding       | LiveCodeBench(2407-2409 Pass@1) | **17.8**              | 16.8                | 12.9                 | 21.8                      |
 |              | HumanEval(Pass@1)               | 82.3                  | **85.4**            | 72.0                 | 86.6                      |
 | Instrunction | IFEval(Prompt-Strict)           | **79.3**              | 71.7                | 75.2                 | 79.7                      |
-| LongContext  | RULER(4-128K Average)           | 87.9                  | 81.4                | **88.5**             | 90.7                      |
+| Long Context | RULER(4-128K Average)           | 87.9                  | 81.4                | **88.5**             | 90.7                      |
 | Chat         | AlpacaEval 2.0(LC WinRate)      | **51.1**              | 30.3                | 25.0                 | 50.7                      |
 |              | WildBench(Raw Score)            | **33.1**              | 23.3                | 1.5                  | 40.3                      |
 |              | MT-Bench-101(Score 1-10)        | **8.59**              | 8.49                | 8.37                 | 8.87                      |
+
 - The evaluation results were obtained from [OpenCompass](https://github.com/internLM/OpenCompass/) (some data marked with *, which means come from the original papers), and evaluation configuration can be found in the configuration files provided by [OpenCompass](https://github.com/internLM/OpenCompass/). 
 - The evaluation data may have numerical differences due to the version iteration of [OpenCompass](https://github.com/internLM/OpenCompass/), so please refer to the latest evaluation results of [OpenCompass](https://github.com/internLM/OpenCompass/).
 
 **Limitations:** Although we have made efforts to ensure the safety of the model during the training process and to encourage the model to generate text that complies with ethical and legal requirements, the model may still produce unexpected outputs due to its size and probabilistic generation paradigm. For example, the generated responses may contain biases, discrimination, or other harmful content. Please do not propagate such content. We are not responsible for any consequences resulting from the dissemination of harmful information.
 
-### Conersation Mode
+### Conversation Mode
 
 #### Transformers inference
 
@@ -121,7 +122,7 @@ You can run batch inference locally with the following python code:
 import lmdeploy
 model_dir = "internlm/internlm3-8b-instruct"
 pipe = lmdeploy.pipeline(model_dir)
-response = pipe(["Please tell me five scenic spots in Shanghai"])
+response = pipe("Please tell me five scenic spots in Shanghai")
 print(response)
 
 ```
@@ -151,7 +152,42 @@ Find more details in the [LMDeploy documentation](https://lmdeploy.readthedocs.i
 
 #### vLLM inference
 
-TODO
+We are still working on merging the PR(https://github.com/vllm-project/vllm/pull/12037) into vLLM. In the meantime, please use the following PR link to install it manually.
+
+```python
+git clone https://github.com/RunningLeon/vllm.git
+pip install -e .
+```
+
+inference code:
+
+```python
+from vllm import LLM, SamplingParams
+
+llm = LLM(model="internlm/internlm3-8b-instruct")
+sampling_params = SamplingParams(temperature=1, repetition_penalty=1.005, top_k=40, top_p=0.8)
+
+system_prompt = """You are an AI assistant whose name is InternLM (书生·浦语).
+- InternLM (书生·浦语) is a conversational language model that is developed by Shanghai AI Laboratory (上海人工智能实验室). It is designed to be helpful, honest, and harmless.
+- InternLM (书生·浦语) can understand and communicate fluently in the language chosen by the user such as English and 中文."""
+
+prompts = [
+    {
+        "role": "system",
+        "content": system_prompt,
+    },
+    {
+        "role": "user",
+        "content": "Please tell me five scenic spots in Shanghai"
+    },
+]
+outputs = llm.chat(prompts,
+                   sampling_params=sampling_params,
+                   use_tqdm=False)
+print(outputs)
+```
+
+
 
 
 
@@ -272,7 +308,36 @@ print(response)
 
 #### vLLM inference
 
-TODO
+We are still working on merging the PR(https://github.com/vllm-project/vllm/pull/12037) into vLLM. In the meantime, please use the following PR link to install it manually.
+```python
+git clone https://github.com/RunningLeon/vllm.git
+pip install -e .
+```
+
+inference code
+
+
+```python
+from vllm import LLM, SamplingParams
+
+llm = LLM(model="internlm/internlm3-8b-instruct")
+sampling_params = SamplingParams(temperature=1, repetition_penalty=1.005, top_k=40, top_p=0.8, max_tokens=8192)
+
+prompts = [
+    {
+        "role": "system",
+        "content": thinking_system_prompt,
+    },
+    {
+        "role": "user",
+        "content": "Given the function\(f(x)=\mathrm{e}^{x}-ax - a^{3}\),\n(1) When \(a = 1\), find the equation of the tangent line to the curve \(y = f(x)\) at the point \((1,f(1))\).\n(2) If \(f(x)\) has a local minimum and the minimum value is less than \(0\), determine the range of values for \(a\)."
+    },
+]
+outputs = llm.chat(prompts,
+                   sampling_params=sampling_params,
+                   use_tqdm=False)
+print(outputs)
+```
 
 
 
@@ -332,6 +397,7 @@ InternLM3是开源社区首个支持浏览器使用的通用对话模型。在
 | Chat         | AlpacaEval 2.0(LC WinRate)      | **51.1**              | 30.3                | 25.0                 | 50.7                      |
 |              | WildBench(Raw Score)            | **33.1**              | 23.3                | 1.5                  | 40.3                      |
 |              | MT-Bench-101(Score 1-10)        | **8.59**              | 8.49                | 8.37                 | 8.87                      |
+
 - 以上评测结果基于 [OpenCompass](https://github.com/internLM/OpenCompass/) 获得（部分数据标注`*`代表数据来自原始论文），具体测试细节可参见 [OpenCompass](https://github.com/internLM/OpenCompass/) 中提供的配置文件。
 - 评测数据会因 [OpenCompass](https://github.com/internLM/OpenCompass/) 的版本迭代而存在数值差异，请以 [OpenCompass](https://github.com/internLM/OpenCompass/) 最新版的评测结果为主。
 
@@ -407,7 +473,7 @@ lmdeploy serve api_server internlm/internlm3-8b-instruct --model-name internlm3-
 curl http://localhost:23333/v1/chat/completions \
     -H "Content-Type: application/json" \
     -d '{
-    "model": "internlm2_5-7b-chat",
+    "model": "internlm3-8b-instruct",
     "messages": [
     {"role": "user", "content": "介绍一下深度学习。"}
     ]
@@ -419,9 +485,42 @@ curl http://localhost:23333/v1/chat/completions \
 
 
 ##### vLLM 推理
-TODO
+我们还在推动PR(https://github.com/vllm-project/vllm/pull/12037) 合入vllm，现在请使用以下PR链接手动安装
+
+```python
+git clone https://github.com/RunningLeon/vllm.git
+pip install -e .
+```
+
+推理代码
 
-#### 深度推理模式
+```python
+from vllm import LLM, SamplingParams
+
+llm = LLM(model="internlm/internlm3-8b-instruct")
+sampling_params = SamplingParams(temperature=1, repetition_penalty=1.005, top_k=40, top_p=0.8)
+
+system_prompt = """You are an AI assistant whose name is InternLM (书生·浦语).
+- InternLM (书生·浦语) is a conversational language model that is developed by Shanghai AI Laboratory (上海人工智能实验室). It is designed to be helpful, honest, and harmless.
+- InternLM (书生·浦语) can understand and communicate fluently in the language chosen by the user such as English and 中文."""
+
+prompts = [
+    {
+        "role": "system",
+        "content": system_prompt,
+    },
+    {
+        "role": "user",
+        "content": "Please tell me five scenic spots in Shanghai"
+    },
+]
+outputs = llm.chat(prompts,
+                   sampling_params=sampling_params,
+                   use_tqdm=False)
+print(outputs)
+```
+
+#### 深度思考模式
 
 ##### 解谜 demo
 
@@ -432,7 +531,7 @@ TODO
 
 
 
-##### 深度推理 system prompt
+##### 深度思考 system prompt
 
 ```python
 thinking_system_prompt = """You are an expert mathematician with extensive experience in mathematical competitions. You approach problems through systematic thinking and rigorous reasoning. When solving problems, follow these thought processes:
@@ -541,7 +640,36 @@ print(response)
 
 ##### vLLM 推理
 
-TODO
+我们还在推动PR(https://github.com/vllm-project/vllm/pull/12037) 合入vllm，现在请使用以下PR链接手动安装
+
+```python
+git clone https://github.com/RunningLeon/vllm.git
+pip install -e .
+```
+
+推理代码
+
+```python
+from vllm import LLM, SamplingParams
+
+llm = LLM(model="internlm/internlm3-8b-instruct")
+sampling_params = SamplingParams(temperature=1, repetition_penalty=1.005, top_k=40, top_p=0.8, max_tokens=8192)
+
+prompts = [
+    {
+        "role": "system",
+        "content": thinking_system_prompt,
+    },
+    {
+        "role": "user",
+        "content": "Given the function\(f(x)=\mathrm{e}^{x}-ax - a^{3}\),\n(1) When \(a = 1\), find the equation of the tangent line to the curve \(y = f(x)\) at the point \((1,f(1))\).\n(2) If \(f(x)\) has a local minimum and the minimum value is less than \(0\), determine the range of values for \(a\)."
+    },
+]
+outputs = llm.chat(prompts,
+                   sampling_params=sampling_params,
+                   use_tqdm=False)
+print(outputs)
+```