[PYTHON/LANGCHAIN] ChatPromptTemplate 클래스 : from_messages 정적 메소드를 사용해 멀티모달 프롬프트 만들기

■ ChatPromptTemplate 클래스의 from_messages 정적 메소드를 사용해 멀티모달 프롬프트를 만드는 방법을 보여준다.

※ OPENAI_API_KEY 환경 변수 값은 .env 파일에 정의한다.

▶ main.py


import httpx
import base64

from dotenv                 import load_dotenv
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai       import ChatOpenAI

load_dotenv()

imageURL = "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"

imageDataString = base64.b64encode(httpx.get(imageURL).content).decode("utf-8")

chatPromptTemplate = ChatPromptTemplate.from_messages(
    [
        ("system", "Describe the image provided"),
        (
            "user",
            [
                {
                    "type"      : "image_url",
                    "image_url" : {"url" : "data:image/jpeg;base64,{image_data}"},
                }
            ],
        ),
    ]
)

chatOpenAI = ChatOpenAI(model = "gpt-4o")

runnableSequence = chatPromptTemplate | chatOpenAI

responseAIMessage = runnableSequence.invoke({"image_data": imageDataString})

print(responseAIMessage.content)

"""
The image depicts a serene landscape featuring a wooden boardwalk leading through a lush, green field. The field is filled with tall grass and surrounded by bushes and trees in the distance. The sky above is a vivid blue with wispy, white clouds scattered throughout, creating a peaceful and open atmosphere. The overall scene conveys a sense of tranquility and natural beauty.
"""

import httpx

import base64

from dotenv import load_dotenv

from langchain_core.prompts import ChatPromptTemplate

from langchain_openai import ChatOpenAI

load_dotenv()

imageURL = "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"

imageDataString = base64.b64encode(httpx.get(imageURL).content).decode("utf-8")

chatPromptTemplate = ChatPromptTemplate.from_messages(

[

("system", "Describe the image provided"),

(

"user",

[

{

"type" : "image_url",

"image_url" : {"url" : "data:image/jpeg;base64,{image_data}"},

}

]

)

chatOpenAI = ChatOpenAI(model = "gpt-4o")

runnableSequence = chatPromptTemplate | chatOpenAI

responseAIMessage = runnableSequence.invoke({"image_data": imageDataString})

print(responseAIMessage.content)

"""

The image depicts a serene landscape featuring a wooden boardwalk leading through a lush, green field. The field is filled with tall grass and surrounded by bushes and trees in the distance. The sky above is a vivid blue with wispy, white clouds scattered throughout, creating a peaceful and open atmosphere. The overall scene conveys a sense of tranquility and natural beauty.

"""

▶ requirements.txt


annotated-types==0.7.0
anyio==4.6.0
certifi==2024.8.30
charset-normalizer==3.3.2
colorama==0.4.6
distro==1.9.0
h11==0.14.0
httpcore==1.0.6
httpx==0.27.2
idna==3.10
jiter==0.5.0
jsonpatch==1.33
jsonpointer==3.0.0
langchain-core==0.3.8
langchain-openai==0.2.1
langsmith==0.1.131
openai==1.51.0
orjson==3.10.7
packaging==24.1
pydantic==2.9.2
pydantic_core==2.23.4
python-dotenv==1.0.1
PyYAML==6.0.2
regex==2024.9.11
requests==2.32.3
requests-toolbelt==1.0.0
sniffio==1.3.1
tenacity==8.5.0
tiktoken==0.8.0
tqdm==4.66.5
typing_extensions==4.12.2
urllib3==2.2.3

annotated-types==0.7.0

anyio==4.6.0

certifi==2024.8.30

charset-normalizer==3.3.2

colorama==0.4.6

distro==1.9.0

h11==0.14.0

httpcore==1.0.6

httpx==0.27.2

idna==3.10

jiter==0.5.0

jsonpatch==1.33

jsonpointer==3.0.0

langchain-core==0.3.8

langchain-openai==0.2.1

langsmith==0.1.131

openai==1.51.0

orjson==3.10.7

packaging==24.1

pydantic==2.9.2

pydantic_core==2.23.4

python-dotenv==1.0.1

PyYAML==6.0.2

regex==2024.9.11

requests==2.32.3

requests-toolbelt==1.0.0

sniffio==1.3.1

tenacity==8.5.0

tiktoken==0.8.0

tqdm==4.66.5

typing_extensions==4.12.2

urllib3==2.2.3

※ pip install python-dotenv langchain-openai 명령을 실행했다.