■ StreamingStdOutCallbackHandler 클래스를 사용해 표준 출력을 스트리밍하는 방법을 보여준다.
▶ main.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
from langchain_core.prompts import PromptTemplate from langchain_core.callbacks import StreamingStdOutCallbackHandler from langchain_community.llms import GPT4All # 프롬프트 템플리트를 설정한다. templateString = """Question : {question} Answer : Let's think step by step.""" promptTemplate = PromptTemplate.from_template(templateString) # GPT4All 모델을 설정한다. callbackHandlerList = [StreamingStdOutCallbackHandler()] gpt4All = GPT4All(model = "./llama-2-13b-chat.Q4_0.gguf", backend = "llama", callbacks = callbackHandlerList, verbose = True) # 실행 체인을 설정한다. runnableSequence = promptTemplate | gpt4All # 질의 응답을 실행한다. question = "What NFL team won the Super Bowl in the year Justin Bieber was born?" resultString = runnableSequence.invoke(question) print() print("질문 : ", question) print() print("답변 : ", resultString.strip()) """ 질문 : What NFL team won the Super Bowl in the year Justin Bieber was born? 답변 : Justin Bieber was born on March 1, 1994. The Super Bowl that year was played on January 29, 1994. So, the NFL team that won the Super Bowl in the year Justin Bieber was born is... ! """ |
※ llama-2-13b-chat.Q4_0.gguf 모델 파일은 https://gpt4all.io 웹 사이트에서 다운로드 받은 [Desktop Chat Client] 프로그램에서 다운로드 받을 수 있다.
▶ requirements.txt
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
aiohttp==3.9.5 aiosignal==1.3.1 annotated-types==0.7.0 async-timeout==4.0.3 attrs==23.2.0 certifi==2024.6.2 charset-normalizer==3.3.2 dataclasses-json==0.6.7 frozenlist==1.4.1 gpt4all==2.7.0 greenlet==3.0.3 idna==3.7 jsonpatch==1.33 jsonpointer==3.0.0 langchain==0.2.3 langchain-community==0.2.4 langchain-core==0.2.5 langchain-text-splitters==0.2.1 langsmith==0.1.77 marshmallow==3.21.3 multidict==6.0.5 mypy-extensions==1.0.0 numpy==1.26.4 orjson==3.10.4 packaging==23.2 pydantic==2.7.4 pydantic_core==2.18.4 PyYAML==6.0.1 requests==2.32.3 SQLAlchemy==2.0.30 tenacity==8.3.0 tqdm==4.66.4 typing-inspect==0.9.0 typing_extensions==4.12.2 urllib3==2.2.1 yarl==1.9.4 |
※ pip install langchain langchain-community gpt4all 명령을 실행했다.