Upload folder using huggingface_hub

README.md

@@ -53,12 +53,12 @@ InternVL 2.0 is a multimodal large language model series, featuring models of va
 
 - It is important to mention that the MMVet scores we report are evaluated using GPT-4-0613 as the judge model. Different versions of GPT-4 can lead to significant variations in the scores for this dataset. For instance, using GPT-4-Turbo would result in significantly lower scores.
 
-Limitations: Although we have made efforts to ensure the safety of the model during the training process and to encourage the model to generate text that complies with ethical and legal requirements, the model may still produce unexpected outputs due to its size and probabilistic generation paradigm. For example, the generated responses may contain biases, discrimination, or other harmful content. Please do not propagate such content. We are not responsible for any consequences resulting from the dissemination of harmful information.
-
 ### Video Benchmarks
 
 TBD
 
+Limitations: Although we have made efforts to ensure the safety of the model during the training process and to encourage the model to generate text that complies with ethical and legal requirements, the model may still produce unexpected outputs due to its size and probabilistic generation paradigm. For example, the generated responses may contain biases, discrimination, or other harmful content. Please do not propagate such content. We are not responsible for any consequences resulting from the dissemination of harmful information.
+
 ## Quick Start
 
 We provide an example code to run InternVL2-8B using `transformers`.
@@ -296,22 +296,101 @@ LMDeploy is a toolkit for compressing, deploying, and serving LLM, developed by
 pip install lmdeploy
 ```
 
-You can run batch inference locally with the following python code:
+LMDeploy abstracts the complex inference process of multi-modal Vision-Language Models (VLM) into an easy-to-use pipeline, similar to the Large Language Model (LLM) inference pipeline.
+
+#### A 'Hello, world' example
 
 ```python
+from lmdeploy import pipeline, TurbomindEngineConfig, ChatTemplateConfig
 from lmdeploy.vl import load_image
-from lmdeploy import ChatTemplateConfig, pipeline
 
 model = 'OpenGVLab/InternVL2-8B'
 system_prompt = '我是书生·万象，英文名是InternVL，是由上海人工智能实验室及多家合作单位联合开发的多模态基础模型。人工智能实验室致力于原始技术创新，开源开放，共享共创，推动科技进步和产业发展。'
 image = load_image('https://raw.githubusercontent.com/open-mmlab/mmdeploy/main/tests/data/tiger.jpeg')
 chat_template_config = ChatTemplateConfig('internlm2-chat')
 chat_template_config.meta_instruction = system_prompt
-pipe = pipeline(model, chat_template_config=chat_template_config)
+pipe = pipeline(model, chat_template_config=chat_template_config,
+                backend_config=TurbomindEngineConfig(session_len=8192))
 response = pipe(('describe this image', image))
+print(response.text)
+```
+
+If `ImportError` occurs while executing this case, please install the required dependency packages as prompted.
+
+#### Multi-images inference
+
+When dealing with multiple images, you can put them all in one list. Keep in mind that multiple images will lead to a higher number of input tokens, and as a result, the size of the context window typically needs to be increased.
+
+```python
+from lmdeploy import pipeline, TurbomindEngineConfig, ChatTemplateConfig
+from lmdeploy.vl import load_image
+from lmdeploy.vl.constants import IMAGE_TOKEN
+
+model = 'OpenGVLab/InternVL2-8B'
+system_prompt = '我是书生·万象，英文名是InternVL，是由上海人工智能实验室及多家合作单位联合开发的多模态基础模型。人工智能实验室致力于原始技术创新，开源开放，共享共创，推动科技进步和产业发展。'
+chat_template_config = ChatTemplateConfig('internlm2-chat')
+chat_template_config.meta_instruction = system_prompt
+pipe = pipeline(model, chat_template_config=chat_template_config,
+                backend_config=TurbomindEngineConfig(session_len=8192))
+
+image_urls=[
+    'https://raw.githubusercontent.com/open-mmlab/mmdeploy/main/demo/resources/human-pose.jpg',
+    'https://raw.githubusercontent.com/open-mmlab/mmdeploy/main/demo/resources/det.jpg'
+]
+
+images = [load_image(img_url) for img_url in image_urls]
+# Numbering images improves multi-image conversations
+response = pipe((f'Image-1: {IMAGE_TOKEN} Image-2: {IMAGE_TOKEN}\ndescribe these two images', images))
+print(response.text)
+```
+
+#### Batch prompts inference
+
+Conducting inference with batch prompts is quite straightforward; just place them within a list structure:
+
+```python
+from lmdeploy import pipeline, TurbomindEngineConfig, ChatTemplateConfig
+from lmdeploy.vl import load_image
+
+model = 'OpenGVLab/InternVL2-8B'
+system_prompt = '我是书生·万象，英文名是InternVL，是由上海人工智能实验室及多家合作单位联合开发的多模态基础模型。人工智能实验室致力于原始技术创新，开源开放，共享共创，推动科���进步和产业发展。'
+chat_template_config = ChatTemplateConfig('internlm2-chat')
+chat_template_config.meta_instruction = system_prompt
+pipe = pipeline(model, chat_template_config=chat_template_config,
+                backend_config=TurbomindEngineConfig(session_len=8192))
+
+image_urls=[
+    "https://raw.githubusercontent.com/open-mmlab/mmdeploy/main/demo/resources/human-pose.jpg",
+    "https://raw.githubusercontent.com/open-mmlab/mmdeploy/main/demo/resources/det.jpg"
+]
+prompts = [('describe this image', load_image(img_url)) for img_url in image_urls]
+response = pipe(prompts)
 print(response)
 ```
 
+#### Multi-turn conversation
+
+There are two ways to do the multi-turn conversations with the pipeline. One is to construct messages according to the format of OpenAI and use above introduced method, the other is to use the `pipeline.chat` interface.
+
+```python
+from lmdeploy import pipeline, TurbomindEngineConfig, ChatTemplateConfig, GenerationConfig
+from lmdeploy.vl import load_image
+
+model = 'OpenGVLab/InternVL2-8B'
+system_prompt = '我是书生·万象，英文名是InternVL，是由上海人工智能实验室及多家合作单位联合开发的多模态基础模型。人工智能实验室致力于原始技术创新，开源开放，共享共创，推动科技进步和产业发展。'
+chat_template_config = ChatTemplateConfig('internlm2-chat')
+chat_template_config.meta_instruction = system_prompt
+pipe = pipeline(model, chat_template_config=chat_template_config,
+                backend_config=TurbomindEngineConfig(session_len=8192))
+
+image = load_image('https://raw.githubusercontent.com/open-mmlab/mmdeploy/main/demo/resources/human-pose.jpg')
+gen_config = GenerationConfig(top_k=40, top_p=0.8, temperature=0.8)
+sess = pipe.chat(('describe this image', image), gen_config=gen_config)
+print(sess.response.text)
+sess = pipe.chat('What is the woman doing?', session=sess, gen_config=gen_config)
+print(sess.response.text)
+```
+
 ## License
 
 This project is released under the MIT license, while InternLM is licensed under the Apache-2.0 license.
@@ -379,8 +458,116 @@ InternVL 2.0 是一个多模态大语言模型系列，包含各种规模的模
 
 - 需要提到的是，我们报告的 MMVet 分数是使用 GPT-4-0613 作为评判模型评估的。不同版本的 GPT-4 会导致该数据集分数的显著变化。例如，使用 GPT-4-Turbo 会导致分数显著降低。
 
+### 视频相关评测
+
+TBD
+
 限制：尽管在训练过程中我们非常注重模型的安全性，尽力促使模型输出符合伦理和法律要求的文本，但受限于模型大小以及概率生成范式，模型可能会产生各种不符合预期的输出，例如回复内容包含偏见、歧视等有害内容，请勿传播这些内容。由于传播不良信息导致的任何后果，本项目不承担责任。
 
+## 部署
+
+### LMDeploy
+
+LMDeploy 是由 MMRazor 和 MMDeploy 团队开发的用于压缩、部署和服务大语言模型（LLM）的工具包。
+
+```sh
+pip install lmdeploy
+```
+
+LMDeploy 将多模态视觉-语言模型（VLM）的复杂推理过程抽象为一个易于使用的管道，类似于大语言模型（LLM）的推理管道。
+
+#### 一个“你好，世界”示例
+
+```python
+from lmdeploy import pipeline, TurbomindEngineConfig, ChatTemplateConfig
+from lmdeploy.vl import load_image
+
+model = 'OpenGVLab/InternVL2-8B'
+system_prompt = '我是书生·万象，英文名是InternVL，是由上海人工智能实验室及多家合作单位联合开发的多模态基础模型。人工智能实验室致力于原始技术创新，开源开放，共享共创，推动科技进步和产业发展。'
+image = load_image('https://raw.githubusercontent.com/open-mmlab/mmdeploy/main/tests/data/tiger.jpeg')
+chat_template_config = ChatTemplateConfig('internlm2-chat')
+chat_template_config.meta_instruction = system_prompt
+pipe = pipeline(model, chat_template_config=chat_template_config,
+                backend_config=TurbomindEngineConfig(session_len=8192))
+response = pipe(('describe this image', image))
+print(response.text)
+```
+
+如果在执行此示例时出现 `ImportError`，请按照提示安装所需的依赖包。
+
+#### 多图像推理
+
+在处理多张图像时，可以将它们全部放入一个列表中。请注意，多张图像会导致输入 token 数量增加，因此通常需要增加上下文窗口的大小。
+
+```python
+from lmdeploy import pipeline, TurbomindEngineConfig, ChatTemplateConfig
+from lmdeploy.vl import load_image
+from lmdeploy.vl.constants import IMAGE_TOKEN
+
+model = 'OpenGVLab/InternVL2-8B'
+system_prompt = '我是书生·万象，英文名是InternVL，是由上海人工智能实验室及多家合作单位联合开发的多模态基础模型。人工智能实验室致力于原始技术创新，开源开放，共享共创，推动科技进步和产业发展。'
+chat_template_config = ChatTemplateConfig('internlm2-chat')
+chat_template_config.meta_instruction = system_prompt
+pipe = pipeline(model, chat_template_config=chat_template_config,
+                backend_config=TurbomindEngineConfig(session_len=8192))
+
+image_urls=[
+    'https://raw.githubusercontent.com/open-mmlab/mmdeploy/main/demo/resources/human-pose.jpg',
+    'https://raw.githubusercontent.com/open-mmlab/mmdeploy/main/demo/resources/det.jpg'
+]
+
+images = [load_image(img_url) for img_url in image_urls]
+response = pipe((f'Image-1: {IMAGE_TOKEN} Image-2: {IMAGE_TOKEN}\ndescribe these two images', images))
+print(response.text)
+```
+
+#### 批量Prompt推理
+
+使用批量Prompt进行推理非常简单；只需将它们放在一个列表结构中：
+
+```python
+from lmdeploy import pipeline, TurbomindEngineConfig, ChatTemplateConfig
+from lmdeploy.vl import load_image
+
+model = 'OpenGVLab/InternVL2-8B'
+system_prompt = '我是书生·万象，英文名是InternVL，是由上海人工智能实验室及多家合作单位联合开发的多模态基础模型。人工智能实验室致力于原始技术创新，开源开放，共享共创，推动科技进步和产业发展。'
+chat_template_config = ChatTemplateConfig('internlm2-chat')
+chat_template_config.meta_instruction = system_prompt
+pipe = pipeline(model, chat_template_config=chat_template_config,
+                backend_config=TurbomindEngineConfig(session_len=8192))
+
+image_urls=[
+    "https://raw.githubusercontent.com/open-mmlab/mmdeploy/main/demo/resources/human-pose.jpg",
+    "https://raw.githubusercontent.com/open-mmlab/mmdeploy/main/demo/resources/det.jpg"
+]
+prompts = [('describe this image', load_image(img_url)) for img_url in image_urls]
+response = pipe(prompts)
+print(response)
+```
+
+#### 多轮对话
+
+使用管道进行多轮对话有两种方法。一种是根据 OpenAI 的格式构建消息并使用上述方法，另一种是使用 `pipeline.chat` 接口。
+
+```python
+from lmdeploy import pipeline, TurbomindEngineConfig, ChatTemplateConfig, GenerationConfig
+from lmdeploy.vl import load_image
+
+model = 'OpenGVLab/InternVL2-8B'
+system_prompt = '我是书生·万象，英文名是InternVL，是由上海人工智能实验室及多家合作单位联合开发的多模态基础模型。人工智能实验室致力于原始技术创新，开源开放，共享共创，推动科技进步和产业发展。'
+chat_template_config = ChatTemplateConfig('internlm2-chat')
+chat_template_config.meta_instruction = system_prompt
+pipe = pipeline(model, chat_template_config=chat_template_config,
+                backend_config=TurbomindEngineConfig(session_len=8192))
+
+image = load_image('https://raw.githubusercontent.com/open-mmlab/mmdeploy/main/demo/resources/human-pose.jpg')
+gen_config = GenerationConfig(top_k=40, top_p=0.8, temperature=0.8)
+sess = pipe.chat(('describe this image', image), gen_config=gen_config)
+print(sess.response.text)
+sess = pipe.chat('What is the woman doing?', session=sess, gen_config=gen_config)
+print(sess.response.text)
+```
+
 ## 开源许可证
 
 该项目采用 MIT 许可证发布，而 InternLM 则采用 Apache-2.0 许可证。