Add datetime to log

.github/workflows/sync-to-hf.yaml

@@ -14,8 +14,7 @@ jobs:
         with:
           fetch-depth: 0
           lfs: true
-          ref: hf-active 
       - name: Push to hub
         env:
           HF_TOKEN: ${{ secrets.HF_TOKEN }}
-        run: git push https://omkarenator:$HF_TOKEN@huggingface.co/spaces/AutoLLM/AutoAgents hf-active:main
+        run: git push https://omkarenator:$HF_TOKEN@huggingface.co/spaces/AutoLLM/AutoAgents main

LICENSE

@@ -1,21 +0,0 @@
-MIT License
-
-Copyright (c) 2023 AutoLLM
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.

README-main.md

@@ -1,103 +1 @@
-# AutoAgents
-
-<p align="center"><img src="https://raw.githubusercontent.com/AutoLLM/AutoAgents/assets/images/logo.png?raw=true" width=400/></p>
-
-Unlock complex question answering in LLMs with enhanced chain-of-thought reasoning and information-seeking capabilities.
-
-## 👉	Overview
-
-The purpose of this project is to extend LLMs ability to answer more complex questions through chain-of-thought reasoning and information-seeking actions.
-
-We are excited to release the initial version of AutoAgents, a proof-of-concept on what can be achieved with only well-written prompts. This is the initial step towards our first big milestone, releasing and open-sourcing the AutoAgents 7B model!
- 
-Come try out our [Huggingface Space](https://huggingface.co/spaces/AutoLLM/AutoAgents)!
-
-
-
-## 🤖 The AutoAgents Project
-
-This project demonstrates LLMs capability to execute a complex user goal: understand a user's goal, generate a plan, use proper tools, and deliver a final result.
-
-For simplicity, our first attempt starts with a Web Search Agent.
-
-
-
-## 💫 How it works:
-
-<p align="left"><img src="https://raw.githubusercontent.com/AutoLLM/AutoAgents/assets/images/agent.png" width=830/></p>
-
-
-
-## 📔 Examples
-
-Ask your AutoAgent to do what a real person would do using the internet:
-
-For example:
-
-*1. Recommend a kid friendly movie that is playing at a theater near Sunnyvale. Give me the showtimes and a link to purchase the tickets*
-
-*2. What is the average age of the past three president when they took office*
-
-*3. What is the mortgage rate right now and how does that compare to the past two years*
-
-
-
-## 💁 Roadmap
-
-* ~~HuggingFace Space demo using OpenAI models~~ [LINK](https://huggingface.co/spaces/AutoLLM/AutoAgents)
-* AutoAgents [7B] Model
-  * Initial Release:
-    * Finetune and release a 7B parameter fine-tuned search model
-* AutoAgents Dataset
-  * A high-quality dataset for a diverse set of search scenarios (why quality and diversity?<sup>[1](https://arxiv.org/abs/2305.11206)</sup>)
-* Reduce Model Inference Overhead
-* Affordance Modeling <sup>[2](https://en.wikipedia.org/wiki/Affordance)</sup>
-* Extend Support to Additional Tools
-* Customizable Document Search set (e.g. personal documents)
-* Support Multi-turn Dialogue
-* Advanced Flow Control in Plan Execution
-
-We are actively developing a few interesting things, check back here or follow us on [Twitter](https://twitter.com/AutoLLM) for any new development.
- 
-If you are interested in any other problems, feel free to shoot us an issue.
-
-
-
-## 🧭 How to use this repo?
-
-This repo contains the entire code to run the search agent from your local browser. All you need is an OpenAI API key to begin.
-
-To run the search agent locally:
-
-1. Clone the repo and change the directory
-
-  ```bash
-  git clone https://github.com/AutoLLM/AutoAgents.git
-  cd AutoAgents
-  ```
-
-2. Install the dependencies
-
-  ```bash
-  pip install -r requirements.txt
-  ```
-
-3. Install the `autoagents` package
-
-  ```bash
-  pip install -e .
-  ```
-
-4. Make sure you have your OpenAI API key set as an environment variable. Alternatively, you can also feed it through the input text-box on the sidebar.
-
-  ```bash
-  export OPENAI_API_KEY=sk-xxxxxx
-  ```
-
-5. Run the Streamlit app
-
-  ```bash
-  streamlit run autoagents/spaces/app.py
-  ```
-
-This should open a browser window where you can type your search query.
+# AutoAgents!

README.md

@@ -1,5 +1,5 @@
 ---
-title: AutoAgents
+title: Search Llm
 emoji: 🐢
 colorFrom: green
 colorTo: purple
@@ -7,7 +7,7 @@ sdk: streamlit
 sdk_version: 1.21.0
 python_version: 3.10.11
 app_file: autoagents/spaces/app.py
-pinned: true
+pinned: false
 ---
 
 Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference

autoagents/agents/search.py

@@ -4,7 +4,6 @@ import re
 from datetime import date
 import asyncio
 from collections import defaultdict
-import os
 
 from langchain.agents import Tool, AgentExecutor, LLMSingleActionAgent, AgentOutputParser
 from langchain.prompts import StringPromptTemplate
@@ -14,7 +13,7 @@ from langchain.schema import AgentAction, AgentFinish
 from langchain.callbacks import get_openai_callback
 from langchain.callbacks.base import AsyncCallbackHandler
 from langchain.callbacks.manager import AsyncCallbackManager
-from langchain.base_language import BaseLanguageModel
+
 
 from autoagents.tools.tools import search_tool, note_tool, rewrite_search_query
 from autoagents.utils.logger import InteractionsLogger
@@ -78,7 +77,7 @@ class CustomPromptTemplate(StringPromptTemplate):
             outputs += f"{action.log}\n"
         if len(intermediate_steps) > 0:
             action, observation = intermediate_steps[-1]
-            # self.ialogger.add_system({"action": action, "observation": observation})
+            self.ialogger.add_system({"action": action, "observation": observation})
             if action.tool not in ("Search", "Notepad"):
                 raise Exception("Invalid tool requested by the model.")
             if action.tool == "Notepad":
@@ -115,7 +114,9 @@ class CustomPromptTemplate(StringPromptTemplate):
         kwargs["tool_names"] = ", ".join([tool.name for tool in self.tools])
         kwargs["today"] = date.today()
         final_prompt = self.template.format(**kwargs)
-        self.ialogger.add_system({"value": final_prompt})
+        if not intermediate_steps:
+            # first iteration
+            self.ialogger.add_system({"prompt": final_prompt})
         return final_prompt
 
 
@@ -123,8 +124,9 @@ class CustomOutputParser(AgentOutputParser):
     class Config:
         arbitrary_types_allowed = True
     ialogger: InteractionsLogger
-    llm: BaseLanguageModel
+    api_key: str
     new_action_input: Optional[str]
+
     action_history = defaultdict(set)
 
     def parse(self, llm_output: str) -> Union[AgentAction, AgentFinish]:
@@ -152,8 +154,7 @@ class CustomOutputParser(AgentOutputParser):
         if action_input in self.action_history[action]:
             new_action_input = rewrite_search_query(action_input,
                                                     self.action_history[action],
-                                                    self.llm)
-            self.ialogger.add_message({"query_rewrite": True})
+                                                    self.api_key)
             self.new_action_input = new_action_input
             self.action_history[action].add(new_action_input)
             return AgentAction(tool=action, tool_input=new_action_input, log=llm_output)
@@ -164,10 +165,7 @@ class CustomOutputParser(AgentOutputParser):
 
 
 class ActionRunner:
-    def __init__(self,
-                 outputq,
-                 llm: BaseLanguageModel,
-                 persist_logs: bool = False):
+    def __init__(self, outputq, api_key: str, model_name: str, persist_logs=False):
         self.ialogger = InteractionsLogger(name=f"{uuid.uuid4().hex[:6]}", persist=persist_logs)
         tools = [search_tool, note_tool]
         prompt = CustomPromptTemplate(
@@ -176,7 +174,7 @@ class ActionRunner:
                 input_variables=["input", "intermediate_steps"],
                 ialogger=self.ialogger)
 
-        output_parser = CustomOutputParser(ialogger=self.ialogger, llm=llm)
+        output_parser = CustomOutputParser(ialogger=self.ialogger, api_key=api_key)
 
         class MyCustomHandler(AsyncCallbackHandler):
             def __init__(self):
@@ -222,6 +220,7 @@ class ActionRunner:
 
         handler = MyCustomHandler()
 
+        llm = ChatOpenAI(openai_api_key=api_key, temperature=0, model_name=model_name)
         llm_chain = LLMChain(llm=llm, prompt=prompt, callbacks=[handler])
         tool_names = [tool.name for tool in tools]
         for tool in tools:

autoagents/models/custom.py

@@ -1,33 +0,0 @@
-import requests
-
-from langchain.llms.base import LLM
-
-
-class CustomLLM(LLM):
-    @property
-    def _llm_type(self) -> str:
-        return "custom"
-
-    def _call(self, prompt: str, stop=None) -> str:
-        r = requests.post(
-            "http://localhost:8000/v1/chat/completions",
-            json={
-                "model": "283-vicuna-7b",
-                "messages": [{"role": "user", "content": prompt}],
-                "stop": stop
-            },
-        )
-        result = r.json()
-        return result["choices"][0]["message"]["content"]
-
-    async def _acall(self, prompt: str, stop=None) -> str:
-        r = requests.post(
-            "http://localhost:8000/v1/chat/completions",
-            json={
-                "model": "283-vicuna-7b",
-                "messages": [{"role": "user", "content": prompt}],
-                "stop": stop
-            },
-        )
-        result = r.json()
-        return result["choices"][0]["message"]["content"]

autoagents/spaces/app.py

@@ -1,7 +1,6 @@
 import os
 import asyncio
 import random
-from datetime import date, datetime, timezone, timedelta
 from ast import literal_eval
 
 import streamlit as st
@@ -10,8 +9,6 @@ import openai
 from autoagents.utils.constants import MAIN_HEADER, MAIN_CAPTION, SAMPLE_QUESTIONS
 from autoagents.agents.search import ActionRunner
 
-from langchain.chat_models import ChatOpenAI
-
 
 async def run():
     output_acc = ""
@@ -45,14 +42,13 @@ async def run():
         )
 
         # Ask the user to enter their OpenAI API key
-        if (api_key := st.sidebar.text_input("OpenAI api-key", type="password")):
-            api_org = None
-        else:
-            api_key, api_org = os.getenv("OPENAI_API_KEY"), os.getenv("OPENAI_API_ORG")
+        API_O = st.sidebar.text_input("OpenAI api-key", type="password") or os.getenv(
+            "OPENAI_API_KEY"
+        )
         with st.sidebar:
             model_dict = {
                 "gpt-3.5-turbo": "GPT-3.5-turbo",
-                "gpt-4": "GPT-4 (Better but slower)",
+                "gpt-4": "GPT-4 (Recommneded for better quality results)",
             }
             st.radio(
                 "OpenAI model",
@@ -61,29 +57,22 @@ async def run():
                 format_func=lambda x: model_dict[x],
             )
 
-            time_zone = str(datetime.now(timezone(timedelta(0))).astimezone().tzinfo)
-            st.markdown(f"**The system time zone is {time_zone} and the date is {date.today()}**")
-
             st.markdown("**Example Queries:**")
             for q in SAMPLE_QUESTIONS:
                 st.markdown(f"*{q}*")
 
-        if not api_key:
+        if not API_O:
             st.warning(
                 "API key required to try this app. The API key is not stored in any form. [This](https://help.openai.com/en/articles/4936850-where-do-i-find-my-secret-api-key) might help."
             )
-        elif api_org and st.session_state.model_name == "gpt-4":
-            st.warning(
-                "The free API key does not support GPT-4. Please switch to GPT-3.5-turbo or input your own API key."
-            )
         else:
             outputq = asyncio.Queue()
-            runner = ActionRunner(outputq,
-                                  ChatOpenAI(openai_api_key=api_key,
-                                             openai_organization=api_org,
-                                             temperature=0,
-                                             model_name=st.session_state.model_name),
-                                  persist_logs=True)  # log to HF-dataset
+            runner = ActionRunner(
+                outputq,
+                api_key=API_O,
+                model_name=st.session_state.model_name,
+                persist_logs=True,
+            )  # log to HF-dataset
 
             async def cleanup(e):
                 st.error(e)
@@ -111,9 +100,6 @@ async def run():
                             if isinstance(output, Exception):
                                 if isinstance(output, openai.error.AuthenticationError):
                                     await cleanup(f"AuthenticationError: Invalid OpenAI API key.")
-                                elif isinstance(output, openai.error.InvalidRequestError) \
-                                      and output._message == "The model: `gpt-4` does not exist":
-                                    await cleanup(f"The free API key does not support GPT-4. Please switch to GPT-3.5-turbo or input your own API key.")
                                 elif isinstance(output, openai.error.OpenAIError):
                                     await cleanup(output)
                                 elif isinstance(output, RuntimeWarning):

autoagents/tools/tools.py

@@ -1,9 +1,7 @@
-import os
-
-from duckpy import Client
 from langchain import PromptTemplate, OpenAI, LLMChain
 from langchain.agents import Tool
-from langchain.base_language import BaseLanguageModel
+from duckpy import Client
+from langchain.chat_models import ChatOpenAI
 
 
 MAX_SEARCH_RESULTS = 20  # Number of search results to observe at a time
@@ -53,12 +51,13 @@ note_tool = Tool(name="Notepad",
                    description=notepad_description)
 
 
-def rewrite_search_query(q: str, search_history, llm: BaseLanguageModel) -> str:
+def rewrite_search_query(q: str, search_history, api_key: str) -> str:
     history_string = '\n'.join(search_history)
     template ="""We are using the Search tool.
                  # Previous queries:
                  {history_string}. \n\n Rewrite query {action_input} to be
                  different from the previous ones."""
+    llm = ChatOpenAI(temperature=0, openai_api_key=api_key)
     prompt = PromptTemplate(template=template,
                             input_variables=["action_input", "history_string"])
     llm_chain = LLMChain(prompt=prompt, llm=llm)

autoagents/utils/constants.py

@@ -1,21 +1,13 @@
 MAIN_HEADER = "Web Search Agent"
 
-MAIN_CAPTION = """This is a proof-of-concept search agent that reasons, plans,
-and executes web searches to collect information on your behalf.  It aims to
-resolve your question by breaking it down into step-by-step subtasks. All the
-intermediate results will be presented.
-
-*DISCLAIMER*: We are collecting search queries, so please refrain from
-providing any personal information. If you wish to avoid this, you can run the
-app locally by following the instructions on our
-[Github](https://github.com/AutoLLM/AutoAgents)."""
+MAIN_CAPTION = """ This is a proof-of-concept search engine built on ReAct-style
+prompting which acts as a search agent that plans and executes web searches on
+your behalf. Given a high-level search query the agent tries to come up with a
+concluding answer based off multiple rounds of searches. You can observe all
+the intermediate interactions with the search engine below."""
 
 SAMPLE_QUESTIONS = [
-    "Recommend me a movie in theater now to watch with kids.",
-    "Who is the most recent NBA MVP? Which team does he play for? What are his career stats?",
-    "Who is the head coach of AC Milan now? How long has he been coaching the team?",
-    "What is the mortgage rate right now and how does that compare to the past two years?",
-    "What is the weather like in San Francisco today? What about tomorrow?",
-    "When and where is the upcoming concert for Taylor Swift? Share a link to purchase tickets.",
-    "Find me recent studies focusing on hallucination in large language models. Provide the link to each study found.",
+    "What has David Sacks written about SAAS? Can you provide some links?",
+    "Where is the all-in summit 2023 being held and how much are the tickets?",
+    "Did Stan Druckenmiller buy Nvidia recently?",
 ]

autoagents/utils/logger.py

@@ -57,4 +57,4 @@ class InteractionsLogger:
             commit_url = self.repo.push_to_hub()
 
     def add_cost(self, cost):
-        self.messages.append({"metrics": cost})
+        self.messages.append({"metrics": cost})

requirements.txt

@@ -1,5 +1,5 @@
 openai>=0.27.7
-langchain>=0.0.193
+langchain
 duckpy
 huggingface_hub
-pytz
+pytz

setup.py

@@ -1,7 +0,0 @@
-from setuptools import setup, find_packages
-
-setup(
-    name='autoagents',
-    version='0.1.0',
-    packages=find_packages(include=['autoagents', 'autoagents.*'])
-)

test.py

@@ -1,22 +1,16 @@
 import os
 import asyncio
-
+from action import ActionRunner
+from langchain.callbacks import get_openai_callback
 from pprint import pprint
+import pdb
 from ast import literal_eval
-from multiprocessing import Pool, TimeoutError
-
-from autoagents.agents.search import ActionRunner
-from langchain.callbacks import get_openai_callback
-from langchain.chat_models import ChatOpenAI
 
-
-async def work(user_input):
+async def main(user_input):
     outputq = asyncio.Queue()
-    llm = ChatOpenAI(openai_api_key=os.getenv("OPENAI_API_KEY"),
-                     openai_organization=os.getenv("OPENAI_API_ORG"),
-                     temperature=0,
-                     model_name="gpt-3.5-turbo")
-    runner = ActionRunner(outputq, llm=llm)
+
+    API_O = os.getenv("OPENAI_API_KEY")
+    runner = ActionRunner(outputq, api_key=API_O, model_name="gpt-3.5-turbo")
     task = asyncio.create_task(runner.run(user_input, outputq))
 
     while True:
@@ -33,11 +27,12 @@ async def work(user_input):
             break
     await task
 
+
+# "list 5 cities and their current populations where Paramore is playing this year.",
+# "Who is Leo DiCaprio's girlfriend? What is her current age raised to the 0.43 power?",
+# "How many watermelons can fit in a Tesla Model S?",
+# "Recommend me some laptops suitable for UI designers under $2000. Please include brand and price."
 Q = [
-     "list 5 cities and their current populations where Paramore is playing this year.",
-     "Who is Leo DiCaprio's girlfriend? What is her current age raised to the 0.43 power?",
-     "How many watermelons can fit in a Tesla Model S?",
-     "Recommend me some laptops suitable for UI designers under $2000. Please include brand and price."
      "Build me a vacation plan for Rome and Milan this summer for seven days. Include place to visit and hotels to stay. ",
      "What is the sum of ages of the wives of Barack Obama and Donald Trump?",
      "Who is the most recent NBA MVP? Which team does he play for? What is his season stats?",
@@ -49,9 +44,6 @@ Q = [
      "Who are some top researchers in the field of machine learning systems nowadays?"
      ]
 
-def main(q):
-    asyncio.run(work(q))
-
-if __name__ == "__main__":
-    with Pool(processes=10) as pool:
-        print(pool.map(main, Q))
+loop = asyncio.new_event_loop()
+for i in range(len(Q)):
+    loop.run_until_complete(main(Q[i]))