[PYTHON/LANGCHAIN] ChatOpenAI 클래스 : with_fallbacks 메소드를 사용해 모델 오류시 다른 모델로 대체하는 RunnableWithFallbacks 객체 만들기 (프롬프트 템플리트 추가)

■ ChatOpenAI 클래스의 with_fallbacks 메소드를 사용해 모델 오류시 다른 모델로 대체하는 RunnableWithFallbacks 객체를 만드는 방법을 보여준다. (프롬프트 템플리트 추가)

※ 본 예제에서는 ChatOpenAI 모델에 강제로 오류를 발생시켜서 테스트한다.
※ OPENAI_API_KEY 환경 변수 값은 .env 파일에 정의한다.

▶ main.py


import httpx

from unittest.mock            import patch
from dotenv                   import load_dotenv
from openai                   import RateLimitError
from langchain_core.prompts   import ChatPromptTemplate
from langchain_community.llms import GPT4All
from langchain_openai         import ChatOpenAI

load_dotenv()

request = httpx.Request("GET", "/")

response = httpx.Response(200, request = request)

rateLimitError = RateLimitError("rate limit", response = response, body = "")

chatPromptTemplate = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You're a nice assistant who always includes a compliment in your response",
        ),
        ("human", "Why did the {animal} cross the road?"),
    ]
)

gpt4All = GPT4All(model = "./llama-2-13b-chat.Q4_0.gguf", backend = "llama")

chatOpenAI = ChatOpenAI(model = "gpt-3.5-turbo-0125", max_retries = 0)

runnableWithFallbacks = chatOpenAI.with_fallbacks([gpt4All])

runnableSequence = chatPromptTemplate | runnableWithFallbacks

with patch("openai.resources.chat.completions.Completions.create", side_effect = RateLimitError):
    try:
        print(runnableSequence.invoke({"animal" : "kangaroo"}))
    except RateLimitError:
        print("Hit error")

"""

System: Oh my goodness, that is such a clever question! I just love how you think outside the box and come up with these hilarious scenarios. The answer to your question is... (insert witty response here).
"""

import httpx

from unittest.mock import patch

from dotenv import load_dotenv

from openai import RateLimitError

from langchain_core.prompts import ChatPromptTemplate

from langchain_community.llms import GPT4All

from langchain_openai import ChatOpenAI

load_dotenv()

request = httpx.Request("GET", "/")

response = httpx.Response(200, request = request)

rateLimitError = RateLimitError("rate limit", response = response, body = "")

chatPromptTemplate = ChatPromptTemplate.from_messages(

[

(

"system",

"You're a nice assistant who always includes a compliment in your response",

("human", "Why did the {animal} cross the road?"),

]

)

gpt4All = GPT4All(model = "./llama-2-13b-chat.Q4_0.gguf", backend = "llama")

chatOpenAI = ChatOpenAI(model = "gpt-3.5-turbo-0125", max_retries = 0)

runnableWithFallbacks = chatOpenAI.with_fallbacks([gpt4All])

runnableSequence = chatPromptTemplate | runnableWithFallbacks

with patch("openai.resources.chat.completions.Completions.create", side_effect = RateLimitError):

try:

print(runnableSequence.invoke({"animal" : "kangaroo"}))

except RateLimitError:

print("Hit error")

"""

System: Oh my goodness, that is such a clever question! I just love how you think outside the box and come up with these hilarious scenarios. The answer to your question is... (insert witty response here).

"""

※ llama-2-13b-chat.Q4_0.gguf 모델 파일은 https://gpt4all.io 웹 사이트에서 다운로드 받은 [Desktop Chat Client] 프로그램에서 다운로드 받을 수 있다.

▶ requirements.txt


aiohttp==3.9.5
aiosignal==1.3.1
annotated-types==0.7.0
anyio==4.4.0
async-timeout==4.0.3
attrs==23.2.0
certifi==2024.6.2
charset-normalizer==3.3.2
dataclasses-json==0.6.7
distro==1.9.0
exceptiongroup==1.2.1
frozenlist==1.4.1
gpt4all==2.7.0
greenlet==3.0.3
h11==0.14.0
httpcore==1.0.5
httpx==0.27.0
idna==3.7
jsonpatch==1.33
jsonpointer==3.0.0
langchain==0.2.5
langchain-community==0.2.5
langchain-core==0.2.9
langchain-openai==0.1.9
langchain-text-splitters==0.2.1
langsmith==0.1.81
marshmallow==3.21.3
multidict==6.0.5
mypy-extensions==1.0.0
numpy==1.26.4
openai==1.35.3
orjson==3.10.5
packaging==24.1
pydantic==2.7.4
pydantic_core==2.18.4
python-dotenv==1.0.1
PyYAML==6.0.1
regex==2024.5.15
requests==2.32.3
sniffio==1.3.1
SQLAlchemy==2.0.31
tenacity==8.4.1
tiktoken==0.7.0
tqdm==4.66.4
typing-inspect==0.9.0
typing_extensions==4.12.2
urllib3==2.2.2
yarl==1.9.4

aiohttp==3.9.5

aiosignal==1.3.1

annotated-types==0.7.0

anyio==4.4.0

async-timeout==4.0.3

attrs==23.2.0

certifi==2024.6.2

charset-normalizer==3.3.2

dataclasses-json==0.6.7

distro==1.9.0

exceptiongroup==1.2.1

frozenlist==1.4.1

gpt4all==2.7.0

greenlet==3.0.3

h11==0.14.0

httpcore==1.0.5

httpx==0.27.0

idna==3.7

jsonpatch==1.33

jsonpointer==3.0.0

langchain==0.2.5

langchain-community==0.2.5

langchain-core==0.2.9

langchain-openai==0.1.9

langchain-text-splitters==0.2.1

langsmith==0.1.81

marshmallow==3.21.3

multidict==6.0.5

mypy-extensions==1.0.0

numpy==1.26.4

openai==1.35.3

orjson==3.10.5

packaging==24.1

pydantic==2.7.4

pydantic_core==2.18.4

python-dotenv==1.0.1

PyYAML==6.0.1

regex==2024.5.15

requests==2.32.3

sniffio==1.3.1

SQLAlchemy==2.0.31

tenacity==8.4.1

tiktoken==0.7.0

tqdm==4.66.4

typing-inspect==0.9.0

typing_extensions==4.12.2

urllib3==2.2.2

yarl==1.9.4

※ psp install httpx python-dotenv langchain langchain-community langchain-openai openai gpt4all 명령을 실행했다.