■ RunnableWithMessageHistory 클래스의 stream 메소드를 사용해 응답 스트리밍을 처리하는 방법을 보여준다.
• LLM이 응답하는 데 시간이 걸릴 수 있으므로 사용자 경험을 개선하기 위해 대부분의 애플리케이션에서 수행하는 작업 중 하나는 생성된 각 토큰을 다시 스트리밍하는 것이다.
• 이를 통해 사용자는 진행 상황을 볼 수 있다.
▶ main.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 |
import os from langchain_core.messages import HumanMessage from langchain_core.chat_history import BaseChatMessageHistory from langchain_core.runnables.history import RunnableWithMessageHistory from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder from langchain_community.chat_message_histories import ChatMessageHistory from langchain_openai import ChatOpenAI os.environ["OPENAI_API_KEY"] = "<OPENAI_API_KEY>" chatPromptTemplate = ChatPromptTemplate.from_messages( [ ("system", "You are a helpful assistant. Answer all questions to the best of your ability in {language}."), MessagesPlaceholder(variable_name = "messages"), ] ) chatOpenAI = ChatOpenAI(model = "gpt-3.5-turbo") runnableSequence = chatPromptTemplate | chatOpenAI storeDictionary = {} def GetChatMessageHistory(sessionID : str) -> BaseChatMessageHistory: if sessionID not in storeDictionary: storeDictionary[sessionID] = ChatMessageHistory() return storeDictionary[sessionID] runnableWithMessageHistory = RunnableWithMessageHistory( runnableSequence, GetChatMessageHistory, input_messages_key = "messages", ) configuratonDictionary = {"configurable" : {"session_id" : "session1"}} for aiMessageChunk in runnableWithMessageHistory.stream( { "messages" : [HumanMessage(content = "hi! I'm todd. tell me a joke")], "language" : "Korean" }, config = configuratonDictionary, ): print(aiMessageChunk.content, end = "") print() """ 안녕하세요, 토드님! 어떤 유머를 선호하시나요?재미있는 농담을 듣고 싶으세요? """ |
▶ requirements.txt
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 |
aiohttp==3.9.5 aiosignal==1.3.1 annotated-types==0.7.0 anyio==4.4.0 async-timeout==4.0.3 attrs==23.2.0 certifi==2024.6.2 charset-normalizer==3.3.2 dataclasses-json==0.6.6 distro==1.9.0 exceptiongroup==1.2.1 frozenlist==1.4.1 greenlet==3.0.3 h11==0.14.0 httpcore==1.0.5 httpx==0.27.0 idna==3.7 jsonpatch==1.33 jsonpointer==2.4 langchain==0.2.3 langchain-community==0.2.4 langchain-core==0.2.5 langchain-openai==0.1.8 langchain-text-splitters==0.2.1 langsmith==0.1.75 marshmallow==3.21.3 multidict==6.0.5 mypy-extensions==1.0.0 numpy==1.26.4 openai==1.33.0 orjson==3.10.3 packaging==23.2 pydantic==2.7.3 pydantic_core==2.18.4 PyYAML==6.0.1 regex==2024.5.15 requests==2.32.3 sniffio==1.3.1 SQLAlchemy==2.0.30 tenacity==8.3.0 tiktoken==0.7.0 tqdm==4.66.4 typing-inspect==0.9.0 typing_extensions==4.12.2 urllib3==2.2.1 yarl==1.9.4 |
※ pip install langchain langchain-openai langchain_community 명령을 실행했다.