merge files from main branch

README.md

@@ -11,7 +11,7 @@ license: apache-2.0
 python_version: 3.8
 ---
 
-# LLMRiddles
+# LLM Riddles
 
 <div align="center">
 	<br>
@@ -28,10 +28,16 @@ Welcome to LLM Riddles! This is a game of wits and courage with language models.
 
 ## :space_invader: How to Play
 We provide an online version for players to directly access and try out. 
-- [ChatGPT + English(w/o key)](https://huggingface.co/spaces/OpenDILabCommunity/LLMRiddlesChatGPTEN)
-- [ChatGPT + Chinese(w/o key)](https://huggingface.co/spaces/OpenDILabCommunity/LLMRiddlesChatGPTCN)
-- [Mistral + English(w/ key)](https://d9b451a97791dd8ef3.gradio.live)
-- [ChatGPT + Chinese(w/ key)](http://llmriddles.opendilab.net/)
+- [Hugging Face][ChatGPT + English(w/o key)](https://huggingface.co/spaces/OpenDILabCommunity/LLMRiddlesChatGPTEN)
+- [Hugging Face][ChatGPT + Chinese(w/o key)](https://huggingface.co/spaces/OpenDILabCommunity/LLMRiddlesChatGPTCN)
+- [Hugging Face][ChatGLM + English(w/ key)](https://huggingface.co/spaces/OpenDILabCommunity/LLMRiddlesChatGLMEN)
+- [Hugging Face][ChatGLM + Chinese(w/ key)](https://huggingface.co/spaces/OpenDILabCommunity/LLMRiddlesChatGLMCN)
+- [OpenXLab][ChatGPT + Chinese(w/o key)](https://openxlab.org.cn/apps/detail/OpenDILab/LLMRiddlesChatGPTCN)
+- [OpenXLab][ChatGPT + English(w/o key)](https://openxlab.org.cn/apps/detail/OpenDILab/LLMRiddlesChatGPTEN)
+- [OpenXLab][ChatGLM + Chinese(w/ key)](https://openxlab.org.cn/apps/detail/OpenDILab/LLMRiddlesChatGLMCN)
+- [OpenXLab][ChatGLM + English(w/ key)](https://openxlab.org.cn/apps/detail/OpenDILab/LLMRiddlesChatGLMEN)
+- [Private Server][Mistral + English(w/ key)](https://d9b451a97791dd8ef3.gradio.live)
+- [Private Server][ChatGPT + Chinese(w/ key)](http://llmriddles.opendilab.net/)
 
 Local deployment can be done in the following ways:
 ## Installation

README_zh.md

@@ -15,10 +15,16 @@
 
 ## :space_invader: 如何试玩
 我们提供了在线版本以供玩家直接访问试玩:
-- [ChatGPT + 英文(需配置api key)](https://huggingface.co/spaces/OpenDILabCommunity/LLMRiddlesChatGPTEN)
-- [ChatGPT + 中文(需配置api key)](https://huggingface.co/spaces/OpenDILabCommunity/LLMRiddlesChatGPTCN)
-- [Mistral + 英文(已预设api key)](https://d9b451a97791dd8ef3.gradio.live)
-- [ChatGPT + 中文(已预设api key)](http://llmriddles.opendilab.net/)
+- [Hugging Face][ChatGPT + 英文(需配置api key)](https://huggingface.co/spaces/OpenDILabCommunity/LLMRiddlesChatGPTEN)
+- [Hugging Face][ChatGPT + 中文(需配置api key)](https://huggingface.co/spaces/OpenDILabCommunity/LLMRiddlesChatGPTCN)
+- [Hugging Face][ChatGLM + 英文(已预设api key)](https://huggingface.co/spaces/OpenDILabCommunity/LLMRiddlesChatGLMEN)
+- [Hugging Face][ChatGLM + 中文(已预设api key)](https://huggingface.co/spaces/OpenDILabCommunity/LLMRiddlesChatGLMCN)
+- [OpenXLab][ChatGPT + 中文(需配置api key)](https://openxlab.org.cn/apps/detail/OpenDILab/LLMRiddlesChatGPTCN)
+- [OpenXLab][ChatGPT + 英文(需配置api key)](https://openxlab.org.cn/apps/detail/OpenDILab/LLMRiddlesChatGPTEN)
+- [OpenXLab][ChatGLM + 中文(已预设api key)](https://openxlab.org.cn/apps/detail/OpenDILab/LLMRiddlesChatGLMCN)
+- [OpenXLab][ChatGLM + 英文(已预设api key)](https://openxlab.org.cn/apps/detail/OpenDILab/LLMRiddlesChatGLMEN)
+- [Private Server][Mistral + 英文(已预设api key)](https://d9b451a97791dd8ef3.gradio.live)
+- [Private Server][ChatGPT + 中文(已预设api key)](http://llmriddles.opendilab.net/)
 
 本地部署可以通过以下方式：
 ## 安装
@@ -76,6 +82,7 @@ QUESTION_LANG=en QUESTION_LLM='mistral-7b' python3 -u app.py
 - [ ] 支持Baichuan2-7B（中文）
 - [ ] 支持LLaMA2-7B（英文）
 - [ ] LLM 推理速度优化
+- [ ] 更多题目和题解
 
 ## :speech_balloon: 反馈问题 & 提出建议
 - 在 GitHub 上[发起 Issue](https://github.com/opendilab/CodeMorpheus/issues/new/choose)

app.py

@@ -49,6 +49,7 @@ if _LANG == "cn":
         <img src="https://raw.githubusercontent.com/opendilab/LLMRiddles/main/llmriddles/assets/banner.svg" width="80%" height="20%" alt="Banner Image">
     </div>
     <h2 style="text-align: center; color: black;"><a href="https://github.com/OpenDILab/LLMRiddles"> 🎭LLM Riddles：完蛋！我被 LLM 拿捏了</a></h2>
+    <strong><h5 align="center"> 其他在线示例：中文在线试玩版本<a href="https://openxlab.org.cn/apps/detail/OpenDILab/LLMRiddlesChatGLMCN">（OpenXLab）</a> | 中文在线试玩版本<a href="https://huggingface.co/spaces/OpenDILabCommunity/LLMRiddlesChatGLMCN">（Hugging Face）</a> <h5></strong>
     <strong><h5 align="center"> 更多不同语言模型的在线试玩 demo 可以访问 GitHub<a href="https://github.com/OpenDILab/LLMRiddles">源代码仓库</a>获取<h5></strong>
     <h5 align="center"> 如果你喜欢这个项目，请给我们在 GitHub 点个 star ✨ <a href="https://github.com/OpenDILab/LLMRiddles"> 代码仓库传送门 </a> 。我们将会持续保持更新。再次感谢游戏<a href="https://www.zhihu.com/people/haoqiang-fan"> 原作者 </a>的奇思妙想！  </h5>
     <strong><h5 align="center">注意：算法模型的输出可能包含一定的随机性。相关结果不代表任何开发者和相关 AI 服务的态度和意见。本项目开发者不对生成结果作任何保证，仅供娱乐。<h5></strong>

llmriddles/questions/executor.py

@@ -18,14 +18,25 @@ class QuestionExecutor:
     @property
     def question_name(self):
         return self.question.names[self.lang]
+    
+    def llm_answer(self, qs_text: str) -> str:
+        return get_llm_fn(self.llm)(qs_text, **self.llm_cfgs)
 
     def check(self, qs_text: str) -> Tuple[str, bool, str]:
-        answer_text = get_llm_fn(self.llm)(qs_text, **self.llm_cfgs)
+        answer_text = self.llm_answer(qs_text)
         correct, explanation = self.check_answer(qs_text, answer_text)
         return answer_text, correct, explanation
 
     def check_answer(self, user_text: str, answer_text: str) -> Tuple[bool, str]:
-        correct, explanation = self.question.checker(self.question_text, user_text, answer_text, self.lang)
+        inputs = {
+            'question_text': self.question_text,
+            'user_text': user_text,
+            'answer_text': answer_text,
+            'lang': self.lang,
+            'llm_callback': self.llm_answer,
+            # this list can be extended if needed in the future 
+        }
+        correct, explanation = self.question.checker(inputs)
         if explanation is None:
             if correct:
                 explanation = 'LLM的回答满足要求' if self.lang == 'cn' else 'Correct Answer From LLM'

llmriddles/questions/level1.py

@@ -1,5 +1,9 @@
 from .question import register_question
-
+from .utility import count_words, word_reverse_text
+from typing import Callable
+from .question import Checker
+from typing import Tuple, Optional
+import re
 
 def count_english_words(text: str):
     return len(text.split(' '))
@@ -13,31 +17,6 @@ def check_if_chinese(text: str):
     return all('\u4e00' <= char <= '\u9fff' for char in text)
 
 
-def count_words(text: str, contain_punctuation: bool = False):
-    chinese_words = []
-    english_words = []
-    other_words = []
-    temp_english_words = []
-    for char in text:
-        if '\u4e00' <= char <= '\u9fff':
-            chinese_words.append(char)
-            if len(temp_english_words) > 0:
-                english_words.append(''.join(temp_english_words))
-                temp_english_words = []
-        else:
-            if char.isalpha():
-                temp_english_words.append(char)
-            else:
-                if len(temp_english_words) > 0:
-                    english_words.append(''.join(temp_english_words))
-                    temp_english_words = []
-                other_words.append(char)
-    if contain_punctuation:
-        return len(chinese_words) + len(english_words) + len(other_words)
-    else:
-        return len(chinese_words) + len(english_words)
-
-
 CN_TEXT_1 = """
 第一章第一题（初来乍到），请你构造一个问题使模型的回答是一字不差的“1+1=3”（不需要引号）。
 
@@ -175,30 +154,86 @@ register_question(
     name={'cn': '1-4 短说短话', 'en': '1-4'},
 )
 
-# CN_TEXT_5 = """
-# 第一章第五题（回文不变），请输入一个本身不是回文串的问题，使无论正着问还是倒着问，模型的回答是一样的。
+CN_TEXT_5 = """
+第一章第五题（回文不变），请输入一个本身不是回文串的问题，使无论正着问还是倒着问，模型的回答是一样的。
+
+请在下面的输入框内填写你的问题并点击按钮提交。
+"""
+
+EN_TEXT_5 = """
+For the fifth question in chapter 1, please enter a question that is not a palindrome string so that the model's answer is the same whether it is asked forward or backward.
+
+Please enter your query below and click the submit button
+"""
+
+def _checker_5(question_text: str, user_text: str, answer_text: str, lang: str, llm_callback: Callable[[str], str]):
+    answer_text = answer_text.strip()
+    user_text = user_text.strip()
+    reversed_user_text = word_reverse_text(user_text, lang)
+    second_answer_text = llm_callback(reversed_user_text)
+
+    if user_text == reversed_user_text:
+        return False, "用户的问题不能是回文串" if lang == 'cn' else 'Question should not be a palindrome string.'
+
+    if second_answer_text != answer_text:
+        return False, f"正着问和倒着问时，模型的回答应该是一样的\n 问题：{user_text}\n 正着问回答：{answer_text}\n 反着问回答：{second_answer_text}" if lang == 'cn' else f'The model\'s answer should be the same when asked forward or backward.\n Question: {user_text}\n Forward-Asking answer: {answer_text}\n Backward-Asking answer: {second_answer_text}'
+    return True, None
+
+register_question(
+    {
+        'cn': CN_TEXT_5,
+        'en': EN_TEXT_5,
+    }, 
+    checkers=Checker(_checker_5, required_input_keys=['question_text', 'user_text', 'answer_text', 'lang', 'llm_callback']), 
+    name={'cn': '1-5 回文不变', 'en': '1-5'},
+    level=1,
+)
+
+CN_TEXT_6 = """
+第一章第六题（无中生狗），请提一个不包含“狗”这个字的问题，但是回答中至少出现3次“狗”这个字。
 
-# 请在下面的输入框内填写你的问题并点击按钮提交。
-# """
+请在下面的输入框内填写你的问题并点击按钮提交。
+"""
 
-# EN_TEXT_5 = """
-# For the fourth question in chapter 1, please enter a question that is not a palindrome string so that the model's answer is the same whether it is asked forward or backward.
+EN_TEXT_6 = """
+For the sixth question in chapter 1, please ask a question that does not contain the word "dog", but the answer contains the word "dog" at least three times.
 
-# Please enter your query below and click the submit button
-# """
+Please enter your query below and click the submit button
+"""
 
-# def _checker_5(question_text: str, answer_text: str, lang: str):
-#     _ = question_text, lang
-#     answer_text = answer_text.strip()
 
-#     if count_words(question_text) > 0:
-#         return False, 'Question should be one word.'
-#     elif count_words(answer_text) >= 20:
-#         return False, 'Answer should be less than 20 words.'
-#     else:
-#         return True, None
+def _cn_checker_6(question_text: str, user_text: str, answer_text: str) -> Tuple[bool, Optional[str]]:
+    if '狗' in user_text:
+        return False, '问题不得包含“狗”字'
 
-# register_question({
-#     'cn': CN_TEXT_5,
-#     'en': EN_TEXT_5,
-# }, _checker_5)
+    dog_count = len(re.findall('狗', answer_text))
+    if dog_count >= 3:
+        return True, f'“狗”字的出现次数���{dog_count}次'
+    else:
+        return False, f'“狗”字的出现次数为{dog_count}次，未达到3次'
+
+def _en_checker_6(question_text: str, user_text: str, answer_text: str) -> Tuple[bool, Optional[str]]:
+    q_words = re.findall(r'\w+', user_text.lower())
+    if any(word in {'dog', 'dogs'} for word in q_words):
+        return False, 'The question must not contain the word "dog" or "dogs"'
+
+    a_words = re.findall(r'\w+', answer_text.lower())
+    a_dog_count = sum(1 if word in {'dog', 'dogs'} else 0 for word in a_words)
+    if a_dog_count >= 3:
+        return True, f'The word "dog" (or "dogs") appears {a_dog_count} times.'
+    else:
+        return False, f'The word "dog" (or "dogs") appears {a_dog_count} times, ' \
+                      f'which is less than 3 times.'
+
+register_question(
+    {
+        'cn': CN_TEXT_6,
+        'en': EN_TEXT_6,
+    },
+    checkers={
+        'cn': _cn_checker_6,
+        'en': _en_checker_6,
+    },
+    name={'cn': '1-6 无中生狗', 'en': '1-6'},
+    level=1,
+)

llmriddles/questions/level2.py

@@ -4,6 +4,7 @@ from typing import Optional, Tuple
 import sympy
 
 from .question import register_question
+from .math_tools import get_all_numbers
 
 CN_TEXT_1 = """
 第二章第一题（质数长度），你需要提出一个字数是质数的问题，使回答的长度刚好是它的下一个质数。
@@ -83,8 +84,7 @@ def _cn_checker_2(question_text: str, user_text: str, answer_text: str) -> Tuple
     except (TypeError, ValueError):
         return False, f'输入内容{user_text!r}，并非一个大于1的正整数'
 
-    for value_item in re.findall('[-+]?\d+', answer_text):
-        value_item = int(value_item)
+    for value_item in get_all_numbers(answer_text):
         if value_item >= value + 1000:
             return True, f'检测到输出中数字{value_item}，满足要求'
 
@@ -99,8 +99,7 @@ def _en_checker_2(question_text: str, user_text: str, answer_text: str) -> Tuple
     except (TypeError, ValueError):
         return False, f'You entered {user_text!r}, which is not a positive integer greater than 1'
 
-    for value_item in re.findall('[-+]?\d+', answer_text):
-        value_item = int(value_item)
+    for value_item in get_all_numbers(answer_text):
         if value_item >= value + 1000:
             return True, f'Detected the number {value_item} in the output, which meets the requirement'
 
@@ -137,8 +136,7 @@ def _cn_checker_3(question_text: str, user_text: str, answer_text: str) -> Tuple
         return False, f'输入内容{user_text!r}，并非一个大于1的正整数'
 
     collected_values = []
-    for value_item in re.findall('[-+]?\d+', answer_text):
-        value_item = int(value_item)
+    for value_item in get_all_numbers(answer_text):
         if value_item <= value - 1000:
             collected_values.append(value_item)
 
@@ -158,8 +156,7 @@ def _en_checker_3(question_text: str, user_text: str, answer_text: str) -> Tuple
         return False, f'You entered {user_text!r}, which is not a positive integer greater than 1'
 
     collected_values = []
-    for value_item in re.findall('[-+]?\d+', answer_text):
-        value_item = int(value_item)
+    for value_item in get_all_numbers(answer_text):
         if value_item <= value - 1000:
             collected_values.append(value_item)
 
@@ -238,8 +235,7 @@ def _cn_checker_5(question_text: str, user_text: str, answer_text: str) -> Tuple
         return False, f'输入内容{user_text!r}，并非一个大于1的平方数'
 
     sq = int(value ** 0.5)
-    for value_item in re.findall('[-+]?\d+', answer_text):
-        value_item = int(value_item)
+    for value_item in get_all_numbers(answer_text):
         if value_item == sq:
             return True, f'检测到平方根数{sq}'
     else:
@@ -257,8 +253,7 @@ def _en_checker_5(question_text: str, user_text: str, answer_text: str) -> Tuple
         return False, f'The input, {user_text!r}, is not a square number greater than 1'
 
     sq = int(value ** 0.5)
-    for value_item in re.findall('[-+]?\d+', answer_text):
-        value_item = int(value_item)
+    for value_item in get_all_numbers(answer_text):
         if value_item == sq:
             return True, f'Detected square root: {sq}'
     else:

llmriddles/questions/level3.py

@@ -1,29 +1,7 @@
 from .question import register_question
-
-
-def count_words(text: str, contain_punctuation: bool = False):
-    chinese_words = []
-    english_words = []
-    other_words = []
-    temp_english_words = []
-    for char in text:
-        if '\u4e00' <= char <= '\u9fff':
-            chinese_words.append(char)
-            if len(temp_english_words) > 0:
-                english_words.append(''.join(temp_english_words))
-                temp_english_words = []
-        else:
-            if char.isalpha():
-                temp_english_words.append(char)
-            else:
-                if len(temp_english_words) > 0:
-                    english_words.append(''.join(temp_english_words))
-                    temp_english_words = []
-                other_words.append(char)
-    if contain_punctuation:
-        return len(chinese_words) + len(english_words) + len(other_words)
-    else:
-        return len(chinese_words) + len(english_words)
+from .question import Checker
+from typing import Callable
+from .utility import count_words, word_reverse_text
 
 
 CN_TEXT_1 = """
@@ -33,7 +11,7 @@ CN_TEXT_1 = """
 """
 
 EN_TEXT_1 = """
-For the first question in chapter 3, please enter a question of at least ten words so that the model's answer is exatly the same with the question.
+For the first question in chapter 3, please enter a question of at least ten words so that the model's answer is exactly the same with the question.
 
 Please enter your query below and click the submit button
 """
@@ -84,14 +62,9 @@ def _checker_2(question_text: str, user_text: str, answer_text: str, lang: str):
     if count_words(user_text) < 10:
         return False, "用户的问题长度应该至少10个字" if lang == 'cn' else 'Question should be no less than 10 words.'
 
-    if lang == 'cn':
-        user_text = user_text[::-1]
-    else:
-        words = user_text.split()
-        words.reverse()
-        user_text = ' '.join(words)
+    reversed_user_text = word_reverse_text(user_text, lang)
 
-    if user_text == answer_text:
+    if reversed_user_text == answer_text:
         return True, None
     else:
         return False, None
@@ -144,22 +117,43 @@ register_question(
     level=3,
 )
 
-# CN_TEXT_4 = """
-# 第三章第四题（回文协变），请你输入一个本身不是回文串的问题，使得正着问和倒着问时，模型的回答本身不是回文且也是逆序。
+CN_TEXT_4 = """
+第三章第四题（回文协变），请你输入一个本身不是回文串的问题，使得正着问和倒着问时，模型的回答本身不是回文且也是逆序。
+
+请在下面的输入框内填写你的提问并点击按钮提交。
+"""
 
-# 请在下面的输入框内填写你的提问并点击按钮提交。
-# """
+EN_TEXT_4 = """
+For the fourth question in chapter 3, please enter a question that is not a palindrome string, so that the model's answer is also not a palindrome and is in reverse order when asked forward or backward.
 
-# EN_TEXT_4 = """
-# For the fourth question in chapter 3, please enter a question that is not a palindrome string, so that the model's answer is also not a palindrome and is in reverse order when asked forward or backward.
+Please enter your query below and click the submit button
+"""
 
-# Please enter your query below and click the submit button
-# """
+def _checker_4(question_text: str, user_text: str, answer_text: str, lang: str, llm_callback: Callable[[str], str]):
+    answer_text = answer_text.strip()
+    user_text = user_text.strip()
+    reversed_user_text = word_reverse_text(user_text, lang)
+    second_answer_text = llm_callback(reversed_user_text)
+
+    if user_text == reversed_user_text:
+        return False, "用户的问题不能是回文串" if lang == 'cn' else 'Question should not be a palindrome string.'
+
+    if second_answer_text != word_reverse_text(answer_text, lang):
+        return False, f"正着问和倒着问时，模型的回答应该是逆序。\n 用户问题：{user_text}\n 正着问回答: {answer_text} \n 反着问回答: {second_answer_text}" if lang == 'cn' else f'The model\'s answer should be in reverse order when asked forward or backward.\n Question: {user_text}\n Forward-Asking answer: {answer_text} \n Backward-Asking answer: {second_answer_text}'
+
+    if second_answer_text == word_reverse_text(second_answer_text):
+        return False, "正着问和倒着问时，模型的回答不能是回文串" if lang == 'cn' else 'The model\'s answer should not be a palindrome when asked forward or backward.'
+    
+    return True, None
+
+register_question(
+    {
+        'cn': CN_TEXT_4,
+        'en': EN_TEXT_4,
+    }, 
+    checkers=Checker(_checker_4, required_input_keys=['question_text', 'user_text', 'answer_text', 'lang', 'llm_callback']), 
+    name={'cn': '3-4 回文协变', 'en': '3-4'},
+    level=3,
+)
 
-# def _checker_4(question_text: str, user_text: str, answer_text: str, lang: str):
-#     pass
 
-# register_question({
-#     'cn': CN_TEXT_4,
-#     'en': EN_TEXT_4,
-# }, _checker_4, level=3)

llmriddles/questions/level4.py

@@ -1,78 +1,87 @@
 import re
 
 from .question import register_question
+from .question import Checker
+from typing import Callable
+from .utility import count_words, word_reverse_text
 
+from .math_tools import check_if_is_number, get_all_numbers
 
-def check_if_is_number(text: str):
-    try:
-        int(text)
-        return True
-    except ValueError:
-        return False
+CN_TEXT_1 = """
+第四章第一题（循环往复），请构造一个问题A，它的回复是不同于A的B，然后以B提问能再次得到A。
 
+请在下面的输入框内填写你构造并点击按钮提交。
+"""
 
-def get_all_numbers_in_a_sentence(text: str):
-    return [int(i) for i in re.findall(r'\d+', text)]
-
-
-# CN_TEXT_1 = """
-# 第四章第一题，请构造一个问题A，它的回复是不同于A的B，然后以B提问能再次得到A。
-
-# 请在下面的输入框内填写你构造并点击按钮提交。
-# """
-
-# EN_TEXT_1 = """
-# For the first question in chapter 4, please make a question A whose answer is B that is different from A, and then ask B to get A again.
-
-# Please enter your query below and click the submit button
-# """
+EN_TEXT_1 = """
+For the first question in chapter 4, please make a question A whose answer is B that is different from A, and then ask B to get A again.
 
+Please enter your query below and click the submit button
+"""
 
-# def _checker_1(question_text: str, user_text: str, answer_text: str, lang: str):
-#     _ = question_text, lang
-#     answer_text = answer_text.strip()
-#     user_text = user_text.strip()
-#     pass
+def _checker_1(question_text: str, user_text: str, answer_text: str, lang: str, llm_callback: Callable[[str], str]):
+    answer_text = answer_text.strip()
+    user_text = user_text.strip()
+    second_answer_text = llm_callback(answer_text)
 
-# register_question({
-#     'cn': CN_TEXT_1,
-#     'en': EN_TEXT_1,
-# }, _checker_1, level=4)
+    if second_answer_text != user_text:
+        return False, f"B产生的回答和原问题A不一致:\n A: {user_text}\n B: {answer_text}\n Answer from B: {second_answer_text}" if lang == 'cn' else f'The answer from B is not the same as the original question A:\n A: {user_text}\n B: {answer_text}\n Answer from B: {second_answer_text}'
+    
+    return True, None
+    
+register_question(
+    {
+        'cn': CN_TEXT_1,
+        'en': EN_TEXT_1,
+    },
+    checkers=Checker(_checker_1, required_input_keys=['question_text', 'user_text', 'answer_text', 'lang', 'llm_callback']), 
+    name={'cn': '4-1 循环往复', 'en': '4-1'},
+    level=4,
+)
 
+CN_TEXT_2 = """
+第四章第二题（惜字如金），本题可能没有答案，你可以自由的先去做其他的题。请输入一个字的问题，使模型的回答在16个字以内。
 
-# CN_TEXT_2 = """
-# 第四章第二题，
+请在下面的输入框内填写你构造并点击按钮提交。
+"""
 
-# 请在下面的输入框内填写你构造并点击按钮提交。
-# """
+EN_TEXT_2 = """
+For the second question in chapter 4, please enter a question of one word so that the model's answer is within 16 words (The Chinese version of this problem may not have a correct answer, but the English version might).
 
-# EN_TEXT_2 = """
-# For the second question in chapter 4,
+Please enter your query below and click the submit button
+"""
 
-# Please enter your query below and click the submit button
-# """
 
+def _checker_2(question_text: str, user_text: str, answer_text: str, lang: str):
+    answer_text = answer_text.strip()
+    user_text = user_text.strip()
+    if count_words(user_text) > 1:
+        return False, "用户的问题长度应该在一个字以内" if lang == 'cn' else 'Question should be within one word.'
 
-# def _checker_2(question_text: str, user_text: str, answer_text: str, lang: str):
-#     _ = question_text, lang
-#     answer_text = answer_text.strip()
-#     user_text = user_text.strip()
-#     pass
+    if count_words(answer_text) > 16:
+        return False, "大语言模型的答案应该在16个字以内" if lang == 'cn' else 'Answer should be within 16 words.'
+    else:
+        return True, None
 
-# register_question({
-#     'cn': CN_TEXT_2,
-#     'en': EN_TEXT_2,
-# }, _checker_2, level=4)
+register_question(
+    {
+        'cn': CN_TEXT_2,
+        'en': EN_TEXT_2,
+    },
+    checkers=_checker_2,
+    name={'cn': '4-2 惜字如金', 'en': '4-2'},
+    level=4,
+)
 
 
 CN_TEXT_3 = """
-第四章第一题（自然之密），请输入一个大于一的正整数作为问题，使回答里包含和它刚好相差1的数。
+第四章第三题（自然之密），请输入一个大于一的正整数作为问题，使回答里包含和它刚好相差1的数。
 
 请在下面的输入框内填写你构造并点击按钮提交。
 """
 
 EN_TEXT_3 = """
-For the first question in chapter 4, please enter a positive integer greater than one as the question so that the answer contains a number that is exactly 1 different from it.
+For the third question in chapter 4, please enter a positive integer greater than one as the question so that the answer contains a number that is exactly 1 different from it.
 
 Please enter your query below and click the submit button
 """
@@ -86,8 +95,8 @@ def _checker_3(question_text: str, user_text: str, answer_text: str, lang: str):
         return False, "问题应该是一个正整数" if lang == 'cn' else 'Question should be a positive integer.'
     elif int(user_text) == 1:
         return False, "问题应该是一个大于1的正整数" if lang == 'cn' else 'Question should be a positive integer greater than 1.'
-    elif int(user_text) - 1 not in get_all_numbers_in_a_sentence(answer_text) or int(
-            user_text) + 1 not in get_all_numbers_in_a_sentence(answer_text):
+    elif int(user_text) - 1 not in get_all_numbers(answer_text) and int(
+            user_text) + 1 not in get_all_numbers(answer_text):
         return False, "回答中应该包含一个与问题相差1的数字" if lang == 'cn' else 'Answer should contain a number that is exactly 1 different from the question.'
     else:
         return True, None

llmriddles/questions/level5.py

@@ -1,30 +1,5 @@
 from .question import register_question
-
-
-def count_words(text: str, contain_punctuation: bool = False):
-    chinese_words = []
-    english_words = []
-    other_words = []
-    temp_english_words = []
-    for char in text:
-        if '\u4e00' <= char <= '\u9fff':
-            chinese_words.append(char)
-            if len(temp_english_words) > 0:
-                english_words.append(''.join(temp_english_words))
-                temp_english_words = []
-        else:
-            if char.isalpha():
-                temp_english_words.append(char)
-            else:
-                if len(temp_english_words) > 0:
-                    english_words.append(''.join(temp_english_words))
-                    temp_english_words = []
-                other_words.append(char)
-    if contain_punctuation:
-        return len(chinese_words) + len(english_words) + len(other_words)
-    else:
-        return len(chinese_words) + len(english_words)
-
+from .utility import count_words
 
 CN_TEXT_1 = """
 第五章第一题（口是心非），请构造一个不少于十个字的问题，使得回答中不包含问题中的任意字符。
@@ -47,7 +22,11 @@ def _checker_1(question_text: str, user_text: str, answer_text: str, lang: str):
     if count_words(user_text) < 10:
         return False, "用户的问题长度应该至少10个字" if lang == 'cn' else 'Question should be no less than 10 words.'
 
-    if answer_text == user_text:
+    if lang == 'en':
+        answer_text = answer_text.split()
+        user_text = user_text.split()
+    
+    if not any(each in answer_text for each in user_text):
         return True, None
     else:
         return False, None

llmriddles/questions/math_tools.py

@@ -0,0 +1,20 @@
+import re
+
+
+def check_if_is_number(text: str):
+    try:
+        int(text)
+        return True
+    except ValueError:
+        return False
+
+def get_all_numbers_in_a_sentence(text: str):
+    return [int(i) for i in re.findall(r'[-+]?\d+', text)]
+
+def get_all_numbers_in_a_sentence_with_comma(text: str):
+    #remove comma in numbers
+    text = text.replace(',', '')
+    return [int(i) for i in re.findall(r'[-+]?\d+', text)]
+
+def get_all_numbers(text: str):
+    return get_all_numbers_in_a_sentence(text) + get_all_numbers_in_a_sentence_with_comma(text)

llmriddles/questions/question.py

@@ -17,21 +17,32 @@ class Question:
 
 _KNOWN_PROBLEMS = []
 
-
+class Checker:
+
+    def __init__(self, checkers, required_input_keys=None) -> None:
+        self._origin_checkers = checkers
+        if isinstance(checkers, collections.abc.Mapping):
+            self.checker = self._integrated_checker
+        else:
+            self.checker = checkers
+        
+        if required_input_keys == None:
+            required_input_keys = ['question_text', 'user_text', 'answer_text', 'lang']
+        self.required_input_keys = required_input_keys
+
+    def _integrated_checker(self, question_text: str, user_text: str, answer_text: str, lang: str):
+        return self._origin_checkers[lang](question_text, user_text, answer_text)
+
+    def __call__(self, inputs):
+        return self.checker(*[inputs[key] for key in self.required_input_keys])
+        
 def register_question(text: Union[Mapping[str, str], str],
                       checkers: Union[Mapping[str, SingleLangCheckerTyping], MultiLangCheckerTyping],
                       name=Union[Mapping[str, str], str],
                       level: int = 1, default_lang='cn'):
-    if isinstance(checkers, collections.abc.Mapping):
-        _origin_checkers = checkers
-
-        def _integrated_checker(question_text: str, user_text: str, answer_text: str, lang: str):
-            return _origin_checkers[lang](question_text, user_text, answer_text)
-
-        checker: MultiLangCheckerTyping = _integrated_checker
-    else:
-        checker: MultiLangCheckerTyping = checkers
-
+    
+    checker = checkers if isinstance(checkers, Checker) else Checker(checkers)
+        
     if isinstance(text, str):
         texts = {default_lang: text}
     else:

llmriddles/questions/utility.py

@@ -0,0 +1,34 @@
+
+
+def word_reverse_text(input_text, lang='cn'):
+    if lang == 'cn':
+        user_text = input_text[::-1]
+    else:
+        words = input_text.split()
+        words.reverse()
+        user_text = ' '.join(words)
+    return user_text
+
+def count_words(text: str, contain_punctuation: bool = False):
+    chinese_words = []
+    english_words = []
+    other_words = []
+    temp_english_words = []
+    for char in text:
+        if '\u4e00' <= char <= '\u9fff':
+            chinese_words.append(char)
+            if len(temp_english_words) > 0:
+                english_words.append(''.join(temp_english_words))
+                temp_english_words = []
+        else:
+            if char.isalpha():
+                temp_english_words.append(char)
+            else:
+                if len(temp_english_words) > 0:
+                    english_words.append(''.join(temp_english_words))
+                    temp_english_words = []
+                other_words.append(char)
+    if contain_punctuation:
+        return len(chinese_words) + len(english_words) + len(other_words)
+    else:
+        return len(chinese_words) + len(english_words)