■ ConditionalPromptSelector 클래스를 사용해 모델에 따라 프롬프트 템플리트를 설정하는 방법을 보여준다.
▶ main.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 |
from langchain_community.llms import LlamaCpp from langchain.chains.prompt_selector import ConditionalPromptSelector from langchain_core.prompts import PromptTemplate llm = LlamaCpp( model_path = "./llama-2-13b-chat.Q4_0.gguf", # llama-2-13b-chat.Q4_0.gguf 파일 경로를 설정한다. n_gpu_layers = 1, # GPU 메모리에 로드할 레이어 수, 모델의 한 레이어만 GPU 메모리에 로드된다(1개이면 충분하다). n_batch = 2048, # 모델이 병렬로 처리해야 하는 토큰 수, 1과 n_ctx 사이의 값을 선택하는 것이 좋다 n_ctx = 2048, # 토큰 컨텍스트 창, 모델은 한 번에 2048개의 토큰 창을 고려한다. f16_kv = True, # 모델이 키/값 캐시에 반정밀도를 사용해야 하는지 여부를 설정한다. # 모델은 메모리 효율성이 더 높은 반정밀도를 사용한다. # 반드시 True로 설정해야 하며, 그렇지 않으면 몇 번의 호출 후에 문제가 발생하게 된다. verbose = False ) defaultLlamaSearchPromptTemplate = PromptTemplate( input_variables = ["question"], template = """<<SYS>> \n You are an assistant tasked with improving Google search \ results. \n <</SYS>> \n\n [INST] Generate THREE Google search queries that \ are similar to this question. The output should be a numbered list of questions \ and each should have a question mark at the end :\n\n{question} [/INST]""" ) defaultSearchPromptTemplate = PromptTemplate( input_variables = ["question"], template = """You are an assistant tasked with improving Google search \ results. Generate THREE Google search queries that are similar to \ this question. The output should be a numbered list of questions and each \ should have a question mark at the end : {question}""" ) conditionalPromptSelector = ConditionalPromptSelector( default_prompt = defaultSearchPromptTemplate, conditionals = [(lambda llm : isinstance(llm, LlamaCpp), defaultLlamaSearchPromptTemplate)] ) promptTemplate = conditionalPromptSelector.get_prompt(llm) runnableSequence = promptTemplate | llm responseString = runnableSequence.invoke("The first man on the moon was ... Let's think step by step") print(responseString) """ Sure, here are three Google search queries that are similar to the original question: 1. Who was the first person to walk on the moon?... 2. What was the name of the astronaut who first stepped on the moon?... 3. Can you name the Apollo mission that included the first manned lunar landing?.... """ |
▶ requirements.txt
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
aiohttp==3.9.5 aiosignal==1.3.1 annotated-types==0.7.0 async-timeout==4.0.3 attrs==23.2.0 certifi==2024.6.2 charset-normalizer==3.3.2 dataclasses-json==0.6.7 diskcache==5.6.3 frozenlist==1.4.1 greenlet==3.0.3 idna==3.7 Jinja2==3.1.4 jsonpatch==1.33 jsonpointer==3.0.0 langchain==0.2.6 langchain-community==0.2.6 langchain-core==0.2.10 langchain-text-splitters==0.2.2 langsmith==0.1.82 llama_cpp_python==0.2.79 MarkupSafe==2.1.5 marshmallow==3.21.3 multidict==6.0.5 mypy-extensions==1.0.0 numpy==1.26.4 orjson==3.10.5 packaging==24.1 pydantic==2.7.4 pydantic_core==2.18.4 PyYAML==6.0.1 requests==2.32.3 SQLAlchemy==2.0.31 tenacity==8.4.2 typing-inspect==0.9.0 typing_extensions==4.12.2 urllib3==2.2.2 yarl==1.9.4 |
※ pip install langchain-community llama-cpp-python 명령을 실행했다.