■ Chroma 클래스의 생성자에서 collection_name/embedding_function 인자를 사용해 Chroma 객체를 만드는 방법을 보여준다. ▶ 예제 코드 (PY)
|
1 2 3 4 5 6 7 8 |
from langchain_openai import OpenAIEmbeddings from langchain_chroma import Chroma openAIEmbeddings = OpenAIEmbeddings() chroma = Chroma(collection_name = "full_documents", embedding_function = openAIEmbeddings) |
※ pip install langchain-openai langchain-chroma
■ TextLoader 클래스의 생성자에서 encoding 인자를 사용해 UTF-8 인코딩 파일을 로드하는 방법을 보여준다. ▶ 예제 코드 (PY)
|
1 2 3 4 5 |
from langchain_community.document_loaders import TextLoader textLoader = TextLoader("paul_graham_essay.txt" , encoding = "utf-8") |
※ pip install langchain-community
■ LongContextReorder 클래스를 사용해 검색된 결과를 재정렬해 "중간에서 잃어버린" 효과를 완화하는 방법을 보여준다. ▶ main.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 |
from langchain_huggingface import HuggingFaceEmbeddings from langchain_chroma import Chroma from langchain_community.document_transformers import LongContextReorder from langchain_openai import OpenAI from langchain_core.prompts import PromptTemplate from langchain.chains.combine_documents import create_stuff_documents_chain huggingFaceEmbeddings = HuggingFaceEmbeddings(model_name = "all-MiniLM-L6-v2") textList = [ "Basquetball is a great sport.", "Fly me to the moon is one of my favourite songs.", "The Celtics are my favourite team.", "This is a document about the Boston Celtics", "I simply love going to the movies", "The Boston Celtics won the game by 20 points", "This is just a random text.", "Elden Ring is one of the best games in the last 15 years.", "L. Kornet is one of the best Celtics players.", "Larry Bird was an iconic NBA player." ] chroma = Chroma.from_texts(textList, embedding = huggingFaceEmbeddings) vectorStoreRetriever = chroma.as_retriever(search_kwargs = {"k" : 10}) query = "What can you tell me about the Celtics?" documentList = vectorStoreRetriever.invoke(query) longContextReorder = LongContextReorder() reorderedDocumentList = longContextReorder.transform_documents(documentList) openAI = OpenAI() templateString = """ Given these texts: ----- {context} ----- Please answer the following question: {query} """ promptTemplate = PromptTemplate( template = templateString, input_variables = ["context", "query"] ) runnableBinding = create_stuff_documents_chain(openAI, promptTemplate) responseString = runnableBinding.invoke({"context" : reorderedDocumentList, "query" : query}) print(responseString) |
▶ requirements.txt
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 |
aiohappyeyeballs==2.4.0 aiohttp==3.10.5 aiosignal==1.3.1 annotated-types==0.7.0 anyio==4.4.0 asgiref==3.8.1 attrs==24.2.0 backoff==2.2.1 bcrypt==4.2.0 build==1.2.2 cachetools==5.5.0 certifi==2024.8.30 charset-normalizer==3.3.2 chroma-hnswlib==0.7.3 chromadb==0.5.3 click==8.1.7 colorama==0.4.6 coloredlogs==15.0.1 dataclasses-json==0.6.7 Deprecated==1.2.14 distro==1.9.0 fastapi==0.114.1 filelock==3.16.0 flatbuffers==24.3.25 frozenlist==1.4.1 fsspec==2024.9.0 google-auth==2.34.0 googleapis-common-protos==1.65.0 greenlet==3.1.0 grpcio==1.66.1 h11==0.14.0 httpcore==1.0.5 httptools==0.6.1 httpx==0.27.2 huggingface-hub==0.24.7 humanfriendly==10.0 idna==3.8 importlib_metadata==8.4.0 importlib_resources==6.4.5 Jinja2==3.1.4 jiter==0.5.0 joblib==1.4.2 jsonpatch==1.33 jsonpointer==3.0.0 kubernetes==30.1.0 langchain==0.2.16 langchain-chroma==0.1.3 langchain-community==0.2.16 langchain-core==0.2.39 langchain-huggingface==0.0.3 langchain-openai==0.1.23 langchain-text-splitters==0.2.4 langsmith==0.1.120 markdown-it-py==3.0.0 MarkupSafe==2.1.5 marshmallow==3.22.0 mdurl==0.1.2 mmh3==4.1.0 monotonic==1.6 mpmath==1.3.0 multidict==6.1.0 mypy-extensions==1.0.0 networkx==3.3 numpy==1.26.4 oauthlib==3.2.2 onnxruntime==1.19.2 openai==1.45.0 opentelemetry-api==1.27.0 opentelemetry-exporter-otlp-proto-common==1.27.0 opentelemetry-exporter-otlp-proto-grpc==1.27.0 opentelemetry-instrumentation==0.48b0 opentelemetry-instrumentation-asgi==0.48b0 opentelemetry-instrumentation-fastapi==0.48b0 opentelemetry-proto==1.27.0 opentelemetry-sdk==1.27.0 opentelemetry-semantic-conventions==0.48b0 opentelemetry-util-http==0.48b0 orjson==3.10.7 overrides==7.7.0 packaging==24.1 pillow==10.4.0 posthog==3.6.5 protobuf==4.25.4 pyasn1==0.6.1 pyasn1_modules==0.4.1 pydantic==2.9.1 pydantic_core==2.23.3 Pygments==2.18.0 PyPika==0.48.9 pyproject_hooks==1.1.0 pyreadline3==3.4.3 python-dateutil==2.9.0.post0 python-dotenv==1.0.1 PyYAML==6.0.2 regex==2024.9.11 requests==2.32.3 requests-oauthlib==2.0.0 rich==13.8.1 rsa==4.9 safetensors==0.4.5 scikit-learn==1.5.2 scipy==1.14.1 sentence-transformers==3.1.0 setuptools==74.1.2 shellingham==1.5.4 six==1.16.0 sniffio==1.3.1 SQLAlchemy==2.0.34 starlette==0.38.5 sympy==1.13.2 tenacity==8.5.0 threadpoolctl==3.5.0 tiktoken==0.7.0 tokenizers==0.19.1 torch==2.4.1 tqdm==4.66.5 transformers==4.44.2 typer==0.12.5 typing-inspect==0.9.0 typing_extensions==4.12.2 urllib3==2.2.3 uvicorn==0.30.6 watchfiles==0.24.0 websocket-client==1.8.0 websockets==13.0.1 wrapt==1.16.0 yarl==1.11.1 zipp==3.20.1 |
※ pip install
■ LongContextReorder 클래스의 transform_documents 메소드를 사용해 검색 문서를 재정렬하는 방법을 보여준다. ▶ main.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
from langchain_huggingface import HuggingFaceEmbeddings from langchain_chroma import Chroma from langchain_community.document_transformers import LongContextReorder huggingFaceEmbeddings = HuggingFaceEmbeddings(model_name = "all-MiniLM-L6-v2") textList = [ "Basquetball is a great sport.", "Fly me to the moon is one of my favourite songs.", "The Celtics are my favourite team.", "This is a document about the Boston Celtics", "I simply love going to the movies", "The Boston Celtics won the game by 20 points", "This is just a random text.", "Elden Ring is one of the best games in the last 15 years.", "L. Kornet is one of the best Celtics players.", "Larry Bird was an iconic NBA player." ] chroma = Chroma.from_texts(textList, embedding = huggingFaceEmbeddings) vectorStoreRetriever = chroma.as_retriever(search_kwargs = {"k" : 10}) documentList = vectorStoreRetriever.invoke("What can you tell me about the Celtics?") for document in documentList: print(document.page_content) print() longContextReorder = LongContextReorder() reorderedDocumentList = longContextReorder.transform_documents(documentList) for reorderedDocument in reorderedDocumentList: print(reorderedDocument.page_content) |
▶ requirements.txt
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 |
aiohappyeyeballs==2.4.0 aiohttp==3.10.5 aiosignal==1.3.1 annotated-types==0.7.0 anyio==4.4.0 asgiref==3.8.1 attrs==24.2.0 backoff==2.2.1 bcrypt==4.2.0 build==1.2.2 cachetools==5.5.0 certifi==2024.8.30 charset-normalizer==3.3.2 chroma-hnswlib==0.7.3 chromadb==0.5.3 click==8.1.7 colorama==0.4.6 coloredlogs==15.0.1 dataclasses-json==0.6.7 Deprecated==1.2.14 distro==1.9.0 fastapi==0.114.1 filelock==3.16.0 flatbuffers==24.3.25 frozenlist==1.4.1 fsspec==2024.9.0 google-auth==2.34.0 googleapis-common-protos==1.65.0 greenlet==3.1.0 grpcio==1.66.1 h11==0.14.0 httpcore==1.0.5 httptools==0.6.1 httpx==0.27.2 huggingface-hub==0.24.7 humanfriendly==10.0 idna==3.8 importlib_metadata==8.4.0 importlib_resources==6.4.5 Jinja2==3.1.4 jiter==0.5.0 joblib==1.4.2 jsonpatch==1.33 jsonpointer==3.0.0 kubernetes==30.1.0 langchain==0.2.16 langchain-chroma==0.1.3 langchain-community==0.2.16 langchain-core==0.2.39 langchain-huggingface==0.0.3 langchain-openai==0.1.23 langchain-text-splitters==0.2.4 langsmith==0.1.120 markdown-it-py==3.0.0 MarkupSafe==2.1.5 marshmallow==3.22.0 mdurl==0.1.2 mmh3==4.1.0 monotonic==1.6 mpmath==1.3.0 multidict==6.1.0 mypy-extensions==1.0.0 networkx==3.3 numpy==1.26.4 oauthlib==3.2.2 onnxruntime==1.19.2 openai==1.45.0 opentelemetry-api==1.27.0 opentelemetry-exporter-otlp-proto-common==1.27.0 opentelemetry-exporter-otlp-proto-grpc==1.27.0 opentelemetry-instrumentation==0.48b0 opentelemetry-instrumentation-asgi==0.48b0 opentelemetry-instrumentation-fastapi==0.48b0 opentelemetry-proto==1.27.0 opentelemetry-sdk==1.27.0 opentelemetry-semantic-conventions==0.48b0 opentelemetry-util-http==0.48b0 orjson==3.10.7 overrides==7.7.0 packaging==24.1 pillow==10.4.0 posthog==3.6.5 protobuf==4.25.4 pyasn1==0.6.1 pyasn1_modules==0.4.1 pydantic==2.9.1 pydantic_core==2.23.3 Pygments==2.18.0 PyPika==0.48.9 pyproject_hooks==1.1.0 pyreadline3==3.4.3 python-dateutil==2.9.0.post0 python-dotenv==1.0.1 PyYAML==6.0.2 regex==2024.9.11 requests==2.32.3 requests-oauthlib==2.0.0 rich==13.8.1 rsa==4.9 safetensors==0.4.5 scikit-learn==1.5.2 scipy==1.14.1 sentence-transformers==3.1.0 setuptools==74.1.2 shellingham==1.5.4 six==1.16.0 sniffio==1.3.1 SQLAlchemy==2.0.34 starlette==0.38.5 sympy==1.13.2 tenacity==8.5.0 threadpoolctl==3.5.0 tiktoken==0.7.0 tokenizers==0.19.1 torch==2.4.1 tqdm==4.66.5 transformers==4.44.2 typer==0.12.5 typing-inspect==0.9.0 typing_extensions==4.12.2 urllib3==2.2.3 uvicorn==0.30.6 watchfiles==0.24.0 websocket-client==1.8.0 websockets==13.0.1 wrapt==1.16.0 yarl==1.11.1 zipp==3.20.1 |
※ pip install langchain-community langchain-huggingface
■ HuggingFaceEmbeddings 클래스의 생성자에서 model_name 인자를 사용해 HuggingFaceEmbeddings 객체를 만드는 방법을 보여준다. ▶ main.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
from langchain_huggingface import HuggingFaceEmbeddings from langchain_chroma import Chroma huggingFaceEmbeddings = HuggingFaceEmbeddings(model_name = "all-MiniLM-L6-v2") textList = [ "Basquetball is a great sport.", "Fly me to the moon is one of my favourite songs.", "The Celtics are my favourite team.", "This is a document about the Boston Celtics", "I simply love going to the movies", "The Boston Celtics won the game by 20 points", "This is just a random text.", "Elden Ring is one of the best games in the last 15 years.", "L. Kornet is one of the best Celtics players.", "Larry Bird was an iconic NBA player.", ] chroma = Chroma.from_texts(textList, embedding = huggingFaceEmbeddings) vectorStoreRetriever = chroma.as_retriever(search_kwargs = {"k" : 10}) documentList = vectorStoreRetriever.invoke("What can you tell me about the Celtics?") for document in documentList: print(document.page_content) |
▶ requirements.txt
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 |
annotated-types==0.7.0 anyio==4.4.0 asgiref==3.8.1 backoff==2.2.1 bcrypt==4.2.0 build==1.2.2 cachetools==5.5.0 certifi==2024.8.30 charset-normalizer==3.3.2 chroma-hnswlib==0.7.3 chromadb==0.5.3 click==8.1.7 colorama==0.4.6 coloredlogs==15.0.1 Deprecated==1.2.14 distro==1.9.0 fastapi==0.114.1 filelock==3.16.0 flatbuffers==24.3.25 fsspec==2024.9.0 google-auth==2.34.0 googleapis-common-protos==1.65.0 grpcio==1.66.1 h11==0.14.0 httpcore==1.0.5 httptools==0.6.1 httpx==0.27.2 huggingface-hub==0.24.7 humanfriendly==10.0 idna==3.8 importlib_metadata==8.4.0 importlib_resources==6.4.5 Jinja2==3.1.4 jiter==0.5.0 joblib==1.4.2 jsonpatch==1.33 jsonpointer==3.0.0 kubernetes==30.1.0 langchain-chroma==0.1.3 langchain-core==0.2.39 langchain-huggingface==0.0.3 langchain-openai==0.1.23 langsmith==0.1.120 markdown-it-py==3.0.0 MarkupSafe==2.1.5 mdurl==0.1.2 mmh3==4.1.0 monotonic==1.6 mpmath==1.3.0 networkx==3.3 numpy==1.26.4 oauthlib==3.2.2 onnxruntime==1.19.2 openai==1.45.0 opentelemetry-api==1.27.0 opentelemetry-exporter-otlp-proto-common==1.27.0 opentelemetry-exporter-otlp-proto-grpc==1.27.0 opentelemetry-instrumentation==0.48b0 opentelemetry-instrumentation-asgi==0.48b0 opentelemetry-instrumentation-fastapi==0.48b0 opentelemetry-proto==1.27.0 opentelemetry-sdk==1.27.0 opentelemetry-semantic-conventions==0.48b0 opentelemetry-util-http==0.48b0 orjson==3.10.7 overrides==7.7.0 packaging==24.1 pillow==10.4.0 posthog==3.6.5 protobuf==4.25.4 pyasn1==0.6.1 pyasn1_modules==0.4.1 pydantic==2.9.1 pydantic_core==2.23.3 Pygments==2.18.0 PyPika==0.48.9 pyproject_hooks==1.1.0 pyreadline3==3.4.3 python-dateutil==2.9.0.post0 python-dotenv==1.0.1 PyYAML==6.0.2 regex==2024.9.11 requests==2.32.3 requests-oauthlib==2.0.0 rich==13.8.1 rsa==4.9 safetensors==0.4.5 scikit-learn==1.5.2 scipy==1.14.1 sentence-transformers==3.1.0 setuptools==74.1.2 shellingham==1.5.4 six==1.16.0 sniffio==1.3.1 starlette==0.38.5 sympy==1.13.2 tenacity==8.5.0 threadpoolctl==3.5.0 tiktoken==0.7.0 tokenizers==0.19.1 torch==2.4.1 tqdm==4.66.5 transformers==4.44.2 typer==0.12.5 typing_extensions==4.12.2 urllib3==2.2.3 uvicorn==0.30.6 watchfiles==0.24.0 websocket-client==1.8.0 websockets==13.0.1 wrapt==1.16.0 zipp==3.20.1 |
※ pip install langchain-huggingface
■ VectorStoreRetriever 클래스의 configurable_fields 메소드에서 search_kwargs 인자에 ConfigurableField 객체를 설정하는 방법을 보여준다. ▶ main.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
from langchain_community.retrievers import BM25Retriever from langchain_openai import OpenAIEmbeddings from langchain_community.vectorstores import FAISS from langchain_core.runnables import ConfigurableField from langchain.retrievers import EnsembleRetriever stringList1 = [ "I like apples", "I like oranges", "Apples and oranges are fruits" ] bm25Retriever = BM25Retriever.from_texts(stringList1, metadatas = [{"source" : 1}] * len(stringList1)) bm25Retriever.k = 2 stringList2 = [ "You like apples", "You like oranges" ] openAIEmbeddings = OpenAIEmbeddings() faiss = FAISS.from_texts(stringList2, openAIEmbeddings, metadatas = [{"source" : 2}] * len(stringList2)) vectorStoreRetriever = faiss.as_retriever(search_kwargs = {"k" : 2}) runnableConfigurableFields = vectorStoreRetriever.configurable_fields( search_kwargs = ConfigurableField( id = "search_kwargs_faiss", name = "Search Kwargs", description = "The search kwargs to use", ) ) ensembleRetriever = EnsembleRetriever(retrievers = [bm25Retriever, runnableConfigurableFields], weights = [0.5, 0.5]) configurationDictionary = {"configurable" : {"search_kwargs_faiss" : {"k" : 1}}} documentList = ensembleRetriever.invoke("apples", config = configurationDictionary) for document in documentList: print(document.page_content) |
▶ requirements.txt
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
aiohappyeyeballs==2.4.0 aiohttp==3.10.5 aiosignal==1.3.1 annotated-types==0.7.0 anyio==4.4.0 attrs==24.2.0 certifi==2024.8.30 charset-normalizer==3.3.2 colorama==0.4.6 dataclasses-json==0.6.7 distro==1.9.0 faiss-cpu==1.8.0.post1 frozenlist==1.4.1 greenlet==3.1.0 h11==0.14.0 httpcore==1.0.5 httpx==0.27.2 idna==3.8 jiter==0.5.0 jsonpatch==1.33 jsonpointer==3.0.0 langchain==0.2.16 langchain-community==0.2.16 langchain-core==0.2.39 langchain-openai==0.1.23 langchain-text-splitters==0.2.4 langsmith==0.1.120 marshmallow==3.22.0 multidict==6.1.0 mypy-extensions==1.0.0 numpy==1.26.4 openai==1.45.0 orjson==3.10.7 packaging==24.1 pydantic==2.9.1 pydantic_core==2.23.3 PyYAML==6.0.2 rank-bm25==0.2.2 regex==2024.9.11 requests==2.32.3 sniffio==1.3.1 SQLAlchemy==2.0.34 tenacity==8.5.0 tiktoken==0.7.0 tqdm==4.66.5 typing-inspect==0.9.0 typing_extensions==4.12.2 urllib3==2.2.3 yarl==1.11.1 |
※ install langchain-community langchain-openai
■ EnsembleRetriever 클래스의 생성자에서 retrievers/weights 인자를 사용해 EnsembleRetriever 객체를 만드는 방법을 보여준다. ▶ main.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
from langchain_community.retrievers import BM25Retriever from langchain_openai import OpenAIEmbeddings from langchain_community.vectorstores import FAISS from langchain.retrievers import EnsembleRetriever stringList1 = [ "I like apples", "I like oranges", "Apples and oranges are fruits" ] bm25Retriever = BM25Retriever.from_texts(stringList1, metadatas = [{"source" : 1}] * len(stringList1)) bm25Retriever.k = 2 stringList2 = [ "You like apples", "You like oranges" ] openAIEmbeddings = OpenAIEmbeddings() faiss = FAISS.from_texts(stringList2, openAIEmbeddings, metadatas = [{"source" : 2}] * len(stringList2)) vectorStoreRetriever = faiss.as_retriever(search_kwargs = {"k" : 2}) ensembleRetriever = EnsembleRetriever(retrievers = [bm25Retriever, vectorStoreRetriever], weights = [0.5, 0.5]) documentList = ensembleRetriever.invoke("apples") for document in documentList: print(document.page_content) |
▶ requirements.txt
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
aiohappyeyeballs==2.4.0 aiohttp==3.10.5 aiosignal==1.3.1 annotated-types==0.7.0 anyio==4.4.0 attrs==24.2.0 certifi==2024.8.30 charset-normalizer==3.3.2 colorama==0.4.6 dataclasses-json==0.6.7 distro==1.9.0 faiss-cpu==1.8.0.post1 frozenlist==1.4.1 greenlet==3.1.0 h11==0.14.0 httpcore==1.0.5 httpx==0.27.2 idna==3.8 jiter==0.5.0 jsonpatch==1.33 jsonpointer==3.0.0 langchain==0.2.16 langchain-community==0.2.16 langchain-core==0.2.39 langchain-openai==0.1.23 langchain-text-splitters==0.2.4 langsmith==0.1.120 marshmallow==3.22.0 multidict==6.1.0 mypy-extensions==1.0.0 numpy==1.26.4 openai==1.45.0 orjson==3.10.7 packaging==24.1 pydantic==2.9.1 pydantic_core==2.23.3 PyYAML==6.0.2 rank-bm25==0.2.2 regex==2024.9.11 requests==2.32.3 sniffio==1.3.1 SQLAlchemy==2.0.34 tenacity==8.5.0 tiktoken==0.7.0 tqdm==4.66.5 typing-inspect==0.9.0 typing_extensions==4.12.2 urllib3==2.2.3 yarl==1.11.1 |
※ install langchain-community langchain-openai
■ BM25Retriever 클래스의 from_texts 정적 메소드를 사용해 BM25Retriever 객체를 만드는 방법을 보여준다. ▶ main.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
from langchain_community.retrievers import BM25Retriever stringList = [ "I like apples", "I like oranges", "Apples and oranges are fruits" ] bm25Retriever = BM25Retriever.from_texts(stringList, metadatas = [{"source" : 1}] * len(stringList1)) bm25Retriever.k = 2 |
▶ requirements.txt
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 |
aiohappyeyeballs==2.4.0 aiohttp==3.10.5 aiosignal==1.3.1 annotated-types==0.7.0 anyio==4.4.0 attrs==24.2.0 certifi==2024.8.30 charset-normalizer==3.3.2 dataclasses-json==0.6.7 frozenlist==1.4.1 greenlet==3.1.0 h11==0.14.0 httpcore==1.0.5 httpx==0.27.2 idna==3.8 jsonpatch==1.33 jsonpointer==3.0.0 langchain==0.2.16 langchain-community==0.2.16 langchain-core==0.2.39 langchain-text-splitters==0.2.4 langsmith==0.1.120 marshmallow==3.22.0 multidict==6.1.0 mypy-extensions==1.0.0 numpy==1.26.4 orjson==3.10.7 packaging==24.1 pydantic==2.9.1 pydantic_core==2.23.3 PyYAML==6.0.2 rank-bm25==0.2.2 requests==2.32.3 sniffio==1.3.1 SQLAlchemy==2.0.34 tenacity==8.5.0 typing-inspect==0.9.0 typing_extensions==4.12.2 urllib3==2.2.3 yarl==1.11.1 |
※ install langchain-community rank_bm25
■ MultiVectorRetriever 클래스를 사용해 여러 벡터를 단일 문서와 연결해 부모 문서를 검색하는 방법을 보여준다. ※ OPENAI_API_KEY 환경 변수 값은 .env 파일에 정의한다.
■ SelfQueryRetriever 클래스를 사용해 LLM을 사용해 잠재적으로 구조화된 쿼리를 생성하는 방법을 보여준다. ※ OPENAI_API_KEY 환경 변수 값은 .env 파일에 정의한다. ▶ main.py
■ Chroma 클래스의 similarity_search_with_score 메소드를 사용해 검색 결과에 점수를 추가하는 방법을 보여준다. ※ OPENAI_API_KEY 환경 변수 값은 .env 파일에 정의한다. ▶ main.py
■ BaseRetriever 클래스의 astream_events 메소드를 사용하는 방법을 보여준다. ▶ main.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 |
import asyncio from langchain_core.retrievers import BaseRetriever from typing import List from langchain_core.documents import Document from langchain_core.callbacks import CallbackManagerForRetrieverRun from langchain import globals class CustomRetriever(BaseRetriever): """사용자 쿼리를 포함하는 상위 k 문서를 포함하는 장난감 검색기이다. 이 검색기는 동기화 메서드 _get_relevant_documents만 구현한다. 검색기가 파일 액세스 또는 네트워크 액세스를 포함하는 경우 `_aget_relevant_documents`의 네이티브 비동기 구현에서 이점을 얻을 수 있다. 평소와 같이 Runnables에는 다른 스레드에서 실행되는 동기화 구현에 위임하는 기본 비동기 구현이 제공된다.""" documents : List[Document] """List of documents to retrieve from.""" k : int """Number of top results to return""" def _get_relevant_documents(self, query : str, *, run_manager : CallbackManagerForRetrieverRun) -> List[Document]: """검색기에 대한 동기화를 구현한다.""" matchingDocumentList = [] for document in self.documents: if len(matchingDocumentList) > self.k: return matchingDocumentList if query.lower() in document.page_content.lower(): matchingDocumentList.append(document) return matchingDocumentList documentList = [ Document( page_content = "Dogs are great companions, known for their loyalty and friendliness.", metadata = {"type" : "dog", "trait" : "loyalty"} ), Document( page_content = "Cats are independent pets that often enjoy their own space.", metadata = {"type" : "cat", "trait" : "independence"} ), Document( page_content = "Goldfish are popular pets for beginners, requiring relatively simple care.", metadata = {"type" : "fish", "trait" : "low maintenance"} ), Document( page_content = "Parrots are intelligent birds capable of mimicking human speech.", metadata = {"type" : "bird", "trait" : "intelligence"} ), Document( page_content = "Rabbits are social animals that need plenty of space to hop around.", metadata = {"type" : "rabbit", "trait" : "social"} ) ] async def main(): globals.set_debug(False) customRetriever = CustomRetriever(documents = documentList, k = 3) async for event in customRetriever.astream_events("bar", version = "v1"): print(event) asyncio.run(main()) |
▶ requirements.txt
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 |
aiohappyeyeballs==2.4.0 aiohttp==3.10.5 aiosignal==1.3.1 annotated-types==0.7.0 anyio==4.4.0 attrs==24.2.0 certifi==2024.8.30 charset-normalizer==3.3.2 frozenlist==1.4.1 greenlet==3.1.0 h11==0.14.0 httpcore==1.0.5 httpx==0.27.2 idna==3.8 jsonpatch==1.33 jsonpointer==3.0.0 langchain==0.2.16 langchain-core==0.2.39 langchain-text-splitters==0.2.4 langsmith==0.1.118 multidict==6.1.0 numpy==1.26.4 orjson==3.10.7 packaging==24.1 pydantic==2.9.1 pydantic_core==2.23.3 PyYAML==6.0.2 requests==2.32.3 sniffio==1.3.1 SQLAlchemy==2.0.34 tenacity==8.5.0 typing_extensions==4.12.2 urllib3==2.2.3 yarl==1.11.1 |
※ pip install langchain 명령을 실행했다.
■ BaseRetriever 클래스의 batch 메소드를 사용하는 방법을 보여준다. ▶ main.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 |
from langchain_core.retrievers import BaseRetriever from typing import List from langchain_core.documents import Document from langchain_core.callbacks import CallbackManagerForRetrieverRun from langchain import globals class CustomRetriever(BaseRetriever): """사용자 쿼리를 포함하는 상위 k 문서를 포함하는 장난감 검색기이다. 이 검색기는 동기화 메서드 _get_relevant_documents만 구현한다. 검색기가 파일 액세스 또는 네트워크 액세스를 포함하는 경우 `_aget_relevant_documents`의 네이티브 비동기 구현에서 이점을 얻을 수 있다. 평소와 같이 Runnables에는 다른 스레드에서 실행되는 동기화 구현에 위임하는 기본 비동기 구현이 제공된다.""" documents : List[Document] """List of documents to retrieve from.""" k : int """Number of top results to return""" def _get_relevant_documents(self, query : str, *, run_manager : CallbackManagerForRetrieverRun) -> List[Document]: """검색기에 대한 동기화를 구현한다.""" matchingDocumentList = [] for document in self.documents: if len(matchingDocumentList) > self.k: return matchingDocumentList if query.lower() in document.page_content.lower(): matchingDocumentList.append(document) return matchingDocumentList documentList = [ Document( page_content = "Dogs are great companions, known for their loyalty and friendliness.", metadata = {"type" : "dog", "trait" : "loyalty"} ), Document( page_content = "Cats are independent pets that often enjoy their own space.", metadata = {"type" : "cat", "trait" : "independence"} ), Document( page_content = "Goldfish are popular pets for beginners, requiring relatively simple care.", metadata = {"type" : "fish", "trait" : "low maintenance"} ), Document( page_content = "Parrots are intelligent birds capable of mimicking human speech.", metadata = {"type" : "bird", "trait" : "intelligence"} ), Document( page_content = "Rabbits are social animals that need plenty of space to hop around.", metadata = {"type" : "rabbit", "trait" : "social"} ) ] globals.set_debug(False) customRetriever = CustomRetriever(documents = documentList, k = 3) resultDocumentList = customRetriever.batch(["dog", "cat"]) print(resultDocumentList) |
▶ requirements.txt
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 |
aiohappyeyeballs==2.4.0 aiohttp==3.10.5 aiosignal==1.3.1 annotated-types==0.7.0 anyio==4.4.0 attrs==24.2.0 certifi==2024.8.30 charset-normalizer==3.3.2 frozenlist==1.4.1 greenlet==3.1.0 h11==0.14.0 httpcore==1.0.5 httpx==0.27.2 idna==3.8 jsonpatch==1.33 jsonpointer==3.0.0 langchain==0.2.16 langchain-core==0.2.39 langchain-text-splitters==0.2.4 langsmith==0.1.118 multidict==6.1.0 numpy==1.26.4 orjson==3.10.7 packaging==24.1 pydantic==2.9.1 pydantic_core==2.23.3 PyYAML==6.0.2 requests==2.32.3 sniffio==1.3.1 SQLAlchemy==2.0.34 tenacity==8.5.0 typing_extensions==4.12.2 urllib3==2.2.3 yarl==1.11.1 |
※ pip install langchain 명령을 실행했다.
■ BaseRetriever 클래스의 invoke 메소드를 사용하는 방법을 보여준다. ▶ main.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 |
from langchain_core.retrievers import BaseRetriever from typing import List from langchain_core.documents import Document from langchain_core.callbacks import CallbackManagerForRetrieverRun class CustomRetriever(BaseRetriever): """사용자 쿼리를 포함하는 상위 k 문서를 포함하는 장난감 검색기이다. 이 검색기는 동기화 메서드 _get_relevant_documents만 구현한다. 검색기가 파일 액세스 또는 네트워크 액세스를 포함하는 경우 `_aget_relevant_documents`의 네이티브 비동기 구현에서 이점을 얻을 수 있다. 평소와 같이 Runnables에는 다른 스레드에서 실행되는 동기화 구현에 위임하는 기본 비동기 구현이 제공된다.""" documents : List[Document] """List of documents to retrieve from.""" k : int """Number of top results to return""" def _get_relevant_documents(self, query : str, *, run_manager : CallbackManagerForRetrieverRun) -> List[Document]: """검색기에 대한 동기화를 구현한다.""" matchingDocumentList = [] for document in self.documents: if len(matchingDocumentList) > self.k: return matchingDocumentList if query.lower() in document.page_content.lower(): matchingDocumentList.append(document) return matchingDocumentList documentList = [ Document( page_content = "Dogs are great companions, known for their loyalty and friendliness.", metadata = {"type" : "dog", "trait" : "loyalty"} ), Document( page_content = "Cats are independent pets that often enjoy their own space.", metadata = {"type" : "cat", "trait" : "independence"} ), Document( page_content = "Goldfish are popular pets for beginners, requiring relatively simple care.", metadata = {"type" : "fish", "trait" : "low maintenance"} ), Document( page_content = "Parrots are intelligent birds capable of mimicking human speech.", metadata = {"type" : "bird", "trait" : "intelligence"} ), Document( page_content = "Rabbits are social animals that need plenty of space to hop around.", metadata = {"type" : "rabbit", "trait" : "social"} ) ] customRetriever = CustomRetriever(documents = documentList, k = 3) resultDocumentList = customRetriever.invoke("that") print(resultDocumentList) |
▶ requirements.txt
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 |
aiohappyeyeballs==2.4.0 aiohttp==3.10.5 aiosignal==1.3.1 annotated-types==0.7.0 anyio==4.4.0 attrs==24.2.0 certifi==2024.8.30 charset-normalizer==3.3.2 frozenlist==1.4.1 greenlet==3.1.0 h11==0.14.0 httpcore==1.0.5 httpx==0.27.2 idna==3.8 jsonpatch==1.33 jsonpointer==3.0.0 langchain==0.2.16 langchain-core==0.2.39 langchain-text-splitters==0.2.4 langsmith==0.1.118 multidict==6.1.0 numpy==1.26.4 orjson==3.10.7 packaging==24.1 pydantic==2.9.1 pydantic_core==2.23.3 PyYAML==6.0.2 requests==2.32.3 sniffio==1.3.1 SQLAlchemy==2.0.34 tenacity==8.5.0 typing_extensions==4.12.2 urllib3==2.2.3 yarl==1.11.1 |
※ pip install langchain 명령을 실행했다.
■ BaseRetriever 클래스를 사용해 커스텀 검색기를 만드는 방법을 보여준다. ▶ main.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 |
from langchain_core.retrievers import BaseRetriever from typing import List from langchain_core.documents import Document from langchain_core.callbacks import CallbackManagerForRetrieverRun class CustomRetriever(BaseRetriever): """사용자 쿼리를 포함하는 상위 k 문서를 포함하는 장난감 검색기이다. 이 검색기는 동기화 메서드 _get_relevant_documents만 구현한다. 검색기가 파일 액세스 또는 네트워크 액세스를 포함하는 경우 `_aget_relevant_documents`의 네이티브 비동기 구현에서 이점을 얻을 수 있다. 평소와 같이 Runnables에는 다른 스레드에서 실행되는 동기화 구현에 위임하는 기본 비동기 구현이 제공된다.""" documents : List[Document] """List of documents to retrieve from.""" k : int """Number of top results to return""" def _get_relevant_documents(self, query : str, *, run_manager : CallbackManagerForRetrieverRun) -> List[Document]: """검색기에 대한 동기화를 구현한다.""" matchingDocumentList = [] for document in self.documents: if len(matchingDocumentList) > self.k: return matchingDocumentList if query.lower() in document.page_content.lower(): matchingDocumentList.append(document) return matchingDocumentList documentList = [ Document( page_content = "Dogs are great companions, known for their loyalty and friendliness.", metadata = {"type" : "dog", "trait" : "loyalty"} ), Document( page_content = "Cats are independent pets that often enjoy their own space.", metadata = {"type" : "cat", "trait" : "independence"} ), Document( page_content = "Goldfish are popular pets for beginners, requiring relatively simple care.", metadata = {"type" : "fish", "trait" : "low maintenance"} ), Document( page_content = "Parrots are intelligent birds capable of mimicking human speech.", metadata = {"type" : "bird", "trait" : "intelligence"} ), Document( page_content = "Rabbits are social animals that need plenty of space to hop around.", metadata = {"type" : "rabbit", "trait" : "social"} ) ] customRetriever = CustomRetriever(documents = documentList, k = 3) |
▶ requirements.txt
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 |
aiohappyeyeballs==2.4.0 aiohttp==3.10.5 aiosignal==1.3.1 annotated-types==0.7.0 anyio==4.4.0 attrs==24.2.0 certifi==2024.8.30 charset-normalizer==3.3.2 frozenlist==1.4.1 greenlet==3.1.0 h11==0.14.0 httpcore==1.0.5 httpx==0.27.2 idna==3.8 jsonpatch==1.33 jsonpointer==3.0.0 langchain==0.2.16 langchain-core==0.2.39 langchain-text-splitters==0.2.4 langsmith==0.1.118 multidict==6.1.0 numpy==1.26.4 orjson==3.10.7 packaging==24.1 pydantic==2.9.1 pydantic_core==2.23.3 PyYAML==6.0.2 requests==2.32.3 sniffio==1.3.1 SQLAlchemy==2.0.34 tenacity==8.5.0 typing_extensions==4.12.2 urllib3==2.2.3 yarl==1.11.1 |
※ pip install langchain 명령을 실행했다.
■ Docx2txtLoader 클래스의 load 메소드를 사용해 MS WORD 문서를 로드하는 방법을 보여준다. ▶ main.py
|
1 2 3 4 5 6 7 8 9 10 11 |
from langchain_community.document_loaders import Docx2txtLoader docx2txtLoader = Docx2txtLoader("sample.docx") documentList = docx2txtLoader.load() document = documentList[0] print(document.page_content) |
▶ requirements.txt
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 |
aiohappyeyeballs==2.4.0 aiohttp==3.10.5 aiosignal==1.3.1 annotated-types==0.7.0 anyio==4.4.0 attrs==24.2.0 certifi==2024.8.30 charset-normalizer==3.3.2 dataclasses-json==0.6.7 docx2txt==0.8 frozenlist==1.4.1 greenlet==3.1.0 h11==0.14.0 httpcore==1.0.5 httpx==0.27.2 idna==3.8 jsonpatch==1.33 jsonpointer==3.0.0 langchain==0.2.16 langchain-community==0.2.16 langchain-core==0.2.39 langchain-text-splitters==0.2.4 langsmith==0.1.118 marshmallow==3.22.0 multidict==6.1.0 mypy-extensions==1.0.0 numpy==1.26.4 orjson==3.10.7 packaging==24.1 pydantic==2.9.1 pydantic_core==2.23.3 PyYAML==6.0.2 requests==2.32.3 sniffio==1.3.1 SQLAlchemy==2.0.34 tenacity==8.5.0 typing-inspect==0.9.0 typing_extensions==4.12.2 urllib3==2.2.2 yarl==1.11.1 |
※ pip install langchain-community
■ UnstructuredURLLoader 클래스의 생성자에서 urls 인자를 사용해 UnstructuredURLLoader 객체를 만드는 방법을 보여준다. ▶ main.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
from langchain_community.document_loaders import UnstructuredURLLoader urlList = [ "https://n.news.naver.com/news/articles/092/0002307222?sid=105", "https://n.news.naver.com/news/articles/052/0001944792?sid=105" ] unstructuredURLLoader = UnstructuredURLLoader(urls = urlList) documentList = unstructuredURLLoader.load() document = documentList[0] print(document.page_content) |
▶ requirements.txt
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 |
aiohappyeyeballs==2.4.0 aiohttp==3.10.5 aiosignal==1.3.1 annotated-types==0.7.0 anyio==4.4.0 attrs==24.2.0 backoff==2.2.1 beautifulsoup4==4.12.3 certifi==2024.8.30 chardet==5.2.0 charset-normalizer==3.3.2 click==8.1.7 colorama==0.4.6 dataclasses-json==0.6.7 deepdiff==8.0.1 emoji==2.12.1 filetype==1.2.0 frozenlist==1.4.1 greenlet==3.1.0 h11==0.14.0 httpcore==1.0.5 httpx==0.27.2 idna==3.8 joblib==1.4.2 jsonpatch==1.33 jsonpath-python==1.0.6 jsonpointer==3.0.0 langchain==0.2.16 langchain-community==0.2.16 langchain-core==0.2.39 langchain-text-splitters==0.2.4 langdetect==1.0.9 langsmith==0.1.118 lxml==5.3.0 marshmallow==3.22.0 multidict==6.1.0 mypy-extensions==1.0.0 nest-asyncio==1.6.0 nltk==3.9.1 numpy==1.26.4 olefile==0.47 orderly-set==5.2.2 orjson==3.10.7 packaging==24.1 psutil==6.0.0 pydantic==2.9.1 pydantic_core==2.23.3 pypdf==4.3.1 python-dateutil==2.9.0.post0 python-iso639==2024.4.27 python-magic==0.4.27 python-oxmsg==0.0.1 PyYAML==6.0.2 rapidfuzz==3.9.7 regex==2024.9.11 requests==2.32.3 requests-toolbelt==1.0.0 six==1.16.0 sniffio==1.3.1 soupsieve==2.6 SQLAlchemy==2.0.34 tabulate==0.9.0 tenacity==8.5.0 tqdm==4.66.5 typing-inspect==0.9.0 typing_extensions==4.12.2 unstructured==0.15.10 unstructured-client==0.25.8 urllib3==2.2.2 wrapt==1.16.0 yarl==1.11.1 |
※ pip install langchain-community
■ OpenAI 클래스에서 CommaSeparatedListOutputParser 객체를 사용하는 방법을 보여준다. ※ OPENAI_API_KEY 환경 변수 값은 .env 파일에 정의한다. ▶ main.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
from dotenv import load_dotenv from langchain_core.output_parsers import CommaSeparatedListOutputParser from langchain_core.prompts import PromptTemplate from langchain_openai import OpenAI load_dotenv() commaSeparatedListOutputParser = CommaSeparatedListOutputParser() formatInstructionString = commaSeparatedListOutputParser.get_format_instructions() promptTemplate = PromptTemplate( template = "{주제} 5개를 추천해주세요.\n{format_instructions}", input_variables = ["주제"], partial_variables = {"format_instructions" : formatInstructionString} ) openAI = OpenAI(temperature = 1) runnableSequence = promptTemplate | openAI | commaSeparatedListOutputParser responseList = runnableSequence.invoke({"주제" : "영화"}) for response in responseList: print(response) |
▶ requirements.txt
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
annotated-types==0.7.0 anyio==4.4.0 certifi==2024.8.30 charset-normalizer==3.3.2 colorama==0.4.6 distro==1.9.0 h11==0.14.0 httpcore==1.0.5 httpx==0.27.2 idna==3.8 jiter==0.5.0 jsonpatch==1.33 jsonpointer==3.0.0 langchain-core==0.2.39 langchain-openai==0.1.23 langsmith==0.1.117 openai==1.44.1 orjson==3.10.7 packaging==24.1 pydantic==2.9.1 pydantic_core==2.23.3 python-dotenv==1.0.1 PyYAML==6.0.2 regex==2024.7.24 requests==2.32.3 sniffio==1.3.1 tenacity==8.5.0 tiktoken==0.7.0 tqdm==4.66.5 typing_extensions==4.12.2 urllib3==2.2.2 |
■ CommaSeparatedListOutputParser 클래스의 get_format_instructions 메소드를 사용해 포맷 명령어 문자열을 구하는 방법을 보여준다. ▶ main.py
|
1 2 3 4 5 6 7 |
from langchain_core.output_parsers import CommaSeparatedListOutputParser commaSeparatedListOutputParser = CommaSeparatedListOutputParser() formatInstructionString = commaSeparatedListOutputParser.get_format_instructions() |
▶ requirements.txt
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 |
aiohappyeyeballs==2.4.0 aiohttp==3.10.5 aiosignal==1.3.1 annotated-types==0.7.0 anyio==4.4.0 attrs==24.2.0 certifi==2024.8.30 charset-normalizer==3.3.2 frozenlist==1.4.1 greenlet==3.1.0 h11==0.14.0 httpcore==1.0.5 httpx==0.27.2 idna==3.8 jsonpatch==1.33 jsonpointer==3.0.0 langchain==0.2.16 langchain-core==0.2.39 langchain-text-splitters==0.2.4 langsmith==0.1.117 multidict==6.1.0 numpy==1.26.4 orjson==3.10.7 packaging==24.1 pydantic==2.9.1 pydantic_core==2.23.3 PyYAML==6.0.2 requests==2.32.3 sniffio==1.3.1 SQLAlchemy==2.0.34 tenacity==8.5.0 typing_extensions==4.12.2 urllib3==2.2.2 yarl==1.11.1 |
※ pip install langchain
■ ChatOpenAI 클래스에서 SemanticSimilarityExampleSelector 객체를 사용해 채팅하는 방법을 보여준다. ※ OPENAI_API_KEY 환경 변수 값은 .env 파일에 정의한다. ▶ main.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 |
from dotenv import load_dotenv from langchain_openai import OpenAIEmbeddings from langchain_chroma import Chroma from langchain_core.example_selectors import SemanticSimilarityExampleSelector from langchain_core.prompts import PromptTemplate from langchain_core.prompts.few_shot import FewShotPromptTemplate from langchain_openai import ChatOpenAI load_dotenv() exampleList = [ {"input" : "행복" , "output" : "슬픔" }, {"input" : "흥미" , "output" : "지루" }, {"input" : "불안" , "output" : "안정" }, {"input" : "긴 기차", "output" : "짧은 기차"}, {"input" : "큰 공" , "output" : "작은 공" } ] openAIEmbeddings = OpenAIEmbeddings() semanticSimilarityExampleSelector = SemanticSimilarityExampleSelector.from_examples( exampleList, openAIEmbeddings, Chroma, k = 1 ) examplePromptTemplate = PromptTemplate( input_variables = ["input", "output"], template = "Input : {input}\nOutput : {output}" ) fewShotPromptTemplate = FewShotPromptTemplate( example_selector = semanticSimilarityExampleSelector, example_prompt = examplePromptTemplate, prefix = "주어진 입력에 대해 반대의 의미를 가진 단어를 출력해주세요.", suffix = "Input : {단어}\nOutput : ", input_variables = ["단어"] ) chatOpenAI = ChatOpenAI(model_name = "gpt-4o-mini") runnableSequence = fewShotPromptTemplate | chatOpenAI responseAIMessage = runnableSequence.invoke({"단어" : "무서운"}) print(responseAIMessage.content) |
▶ requirements.txt
■ FewShotPromptTemplate 클래스의 생성자에서 example_selector 인자를 사용해 SemanticSimilarityExampleSelector 객체를 설정하는 방법을 보여준다. ※ OPENAI_API_KEY 환경 변수 값은 .env 파일에 정의한다. ▶ main.py
■ SemanticSimilarityExampleSelector 클래스의 from_examples 정적 메소드를 사용해 SemanticSimilarityExampleSelector 객체를 만드는 방법을 보여준다. ※ OPENAI_API_KEY 환경 변수 값은 .env 파일에 정의한다. ▶ main.py
■ ChatOpenAI 클래스에서 FewShotPromptTemplate 객체를 사용해 채팅하는 방법을 보여준다. ※ OPENAI_API_KEY 환경 변수 값은 .env 파일에 정의한다. ▶ main.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 |
from dotenv import load_dotenv from langchain_core.prompts import PromptTemplate from langchain_core.prompts.few_shot import FewShotPromptTemplate from langchain_openai import ChatOpenAI load_dotenv() exampleList = [ { "question" : "아이유로 삼행시 만들어주세요.", "answer" : """ 아 : 아이유는 이 : 이런 강의를 들을 이 유 : 유가 없다. """ }, { "question" : "김민수로 삼행시를 만들어주세요.", "answer" : """ 김 : 김치는 맛있다. 민 : 민달팽이도 좋아하는 김치! 수 : 수억을 줘도 김치는 내꺼! """ } ] examplePromptTemplate = PromptTemplate(input_variables = ["quesiton", "answer"], template = "Question : {question}\n{answer}") fewShotPromptTemplate = FewShotPromptTemplate( examples = exampleList, example_prompt = examplePromptTemplate, suffix = "Question : {input}", input_variables = ["input"] ) chatOpenAI = ChatOpenAI(model_name = "gpt-4o-mini") runnableSequence = fewShotPromptTemplate | chatOpenAI responseAIMessage = runnableSequence.invoke({"input" : "홍길동으로 삼행시를 만들어주세요."}) print(responseAIMessage.content) |
▶ requirements.txt
■ FewShotPromptTemplate 클래스의 생성자에서 examples/example_prompt/suffix/input_variables 인자를 사용해 FewShotPromptTemplate 객체를 만드는 방법을 보여준다. ▶ main.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
from langchain_core.prompts import PromptTemplate from langchain_core.prompts.few_shot import FewShotPromptTemplate exampleList = [ { "question" : "아이유로 삼행시 만들어주세요.", "answer" : """ 아 : 아이유는 이 : 이런 강의를 들을 이 유 : 유가 없다. """ }, { "question" : "김민수로 삼행시를 만들어주세요.", "answer" : """ 김 : 김치는 맛있다. 민 : 민달팽이도 좋아하는 김치! 수 : 수억을 줘도 김치는 내꺼! """ } ] examplePromptTemplate = PromptTemplate(input_variables = ["quesiton", "answer"], template = "Question : {question}\n{answer}") fewShotPromptTemplate = FewShotPromptTemplate( examples = exampleList, example_prompt = examplePromptTemplate, suffix = "Question : {input}", input_variables = ["input"] ) |
▶ requirements.txt
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 |
aiohappyeyeballs==2.4.0 aiohttp==3.10.5 aiosignal==1.3.1 annotated-types==0.7.0 anyio==4.4.0 attrs==24.2.0 certifi==2024.8.30 charset-normalizer==3.3.2 frozenlist==1.4.1 greenlet==3.1.0 h11==0.14.0 httpcore==1.0.5 httpx==0.27.2 idna==3.8 jsonpatch==1.33 jsonpointer==3.0.0 langchain==0.2.16 langchain-core==0.2.39 langchain-text-splitters==0.2.4 langsmith==0.1.117 multidict==6.1.0 numpy==1.26.4 orjson==3.10.7 packaging==24.1 pydantic==2.9.1 pydantic_core==2.23.3 PyYAML==6.0.2 requests==2.32.3 sniffio==1.3.1 SQLAlchemy==2.0.34 tenacity==8.5.0 typing_extensions==4.12.2 urllib3==2.2.2 yarl==1.11.1 |
※ pip install langchain
■ ChatPromptTemplate 클래스의 from_messages 정적 메소드에서 SystemMessagePromptTemplate/HumanMessagePromptTemplate 객체 리스를 사용해 ChatPromptTemplate 객체를 만드는 방법을 보여준다. ※ OPENAI_API_KEY 환경 변수 값은 .env 파일에