■ load_summarize_chain 함수를 사용해 MAP-REDUCE 요약 체인을 만드는 방법을 보여준다. (MapReduceDocumentsChain 객체)
▶ main.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
import os from langchain_community.document_loaders import WebBaseLoader from langchain_openai import ChatOpenAI from langchain.chains.summarize import load_summarize_chain os.environ["OPENAI_API_KEY"] = "<OPENAI_API_KEY>" webBaseLoader = WebBaseLoader("https://lilianweng.github.io/posts/2023-06-23-agent/") documentList = webBaseLoader.load() chatOpenAI = ChatOpenAI(model_name = "gpt-3.5-turbo-1106", temperature = 0) mapReduceDocumentsChain = load_summarize_chain(chatOpenAI, chain_type = "map_reduce") resultDictionary = mapReduceDocumentsChain.invoke(documentList) print(resultDictionary["output_text"]) """ The article explores the use of LLM-powered autonomous agents, which utilize large language models as their main controllers. It discusses the components of these agents, including planning, memory, and tool use, and provides case studies and proof-of-concept examples. The challenges and limitations of using natural language interfaces for these agents are also addressed, with citations and references for further reading included. """ |
▶ requirements.txt
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
aiohttp==3.9.5 aiosignal==1.3.1 annotated-types==0.7.0 anyio==4.4.0 async-timeout==4.0.3 attrs==23.2.0 beautifulsoup4==4.12.3 bs4==0.0.2 certifi==2024.6.2 charset-normalizer==3.3.2 dataclasses-json==0.6.7 distro==1.9.0 exceptiongroup==1.2.1 frozenlist==1.4.1 greenlet==3.0.3 h11==0.14.0 httpcore==1.0.5 httpx==0.27.0 idna==3.7 jsonpatch==1.33 jsonpointer==3.0.0 langchain==0.2.5 langchain-community==0.2.5 langchain-core==0.2.7 langchain-openai==0.1.8 langchain-text-splitters==0.2.1 langsmith==0.1.77 marshmallow==3.21.3 multidict==6.0.5 mypy-extensions==1.0.0 numpy==1.26.4 openai==1.34.0 orjson==3.10.5 packaging==24.1 pydantic==2.7.4 pydantic_core==2.18.4 PyYAML==6.0.1 regex==2024.5.15 requests==2.32.3 sniffio==1.3.1 soupsieve==2.5 SQLAlchemy==2.0.30 tenacity==8.3.0 tiktoken==0.7.0 tqdm==4.66.4 typing-inspect==0.9.0 typing_extensions==4.12.2 urllib3==2.2.1 yarl==1.9.4 |
※ pip install langchain langchain-community langchain-openai bs4 명령을 실행했다.