LLAMA3.1 Archives - icodebroker

[PYTHON/LANGCHAIN] 라마 3.1 한국어 모델 채팅 서버/클라이언트 프로그램 만들기 2

■ 라마 3.1 한국어 모델과 채팅하는 서버/클라이언트 프로그램을 만드는 방법을 보여준다. ※ Ollama 설치/실행과 llama3.1:8b-instruct-q2_K 모델이 다운로드 되어 있어야 한다. ▶ server.py

[PYTHON/LANGCHAIN] 라마 3.1 한국어 모델 채팅 서버/클라이언트 프로그램 만들기 1

[TOOL/OLLAMA] ollama 명령 : create 옵션을 사용해 ggml-model-q4_k_m.gguf 파일에서 모델 생성하기

■ ollama 명령의 create 옵션을 사용해 .gguf 파일에서 모델을 생성하는 방법을 보여준다. 1. 명령 프롬프트를 실행한다. 2. 아래 스크립트를 실행한다. ▶ 실행

[TOOL/OLLAMA] ollama 명령 : run 옵션을 사용해 benedict/linkbricks-llama3.1-korean:8b 라마 3.1 한글 모델 실행하기

■ ollama 명령의 run 옵션을 사용해 benedict/linkbricks-llama3.1-korean:8b 라마 3.1 한글 모델을 실행하는 방법을 보여준다. 1. 명령 프롬프트를 실행한다. 2. 아래 스크립트를 실행한다.

[PYTHON/LANGCHAIN] LlamaCpp 클래스 : Llama 3.1 모델을 사용해 로컬 RAG 애플리케이션 만들기

■ LlamaCpp 클래스에서 Llama 3.1 모델을 사용해 로컬 RAG 애플리케이션을 만드는 방법을 보여준다. ▶ main.py


from langchain_community.llms             import LlamaCpp
from langchain.memory                     import ConversationBufferMemory
from langchain_core.runnables             import RunnablePassthrough
from langchain_core.prompts               import MessagesPlaceholder
from langchain_core.prompts               import ChatPromptTemplate
from langchain_core.output_parsers        import StrOutputParser

modelFilePath = "./meta-llama-3.1-8b-instruct-abliterated.bf16.gguf"

llamaCpp = LlamaCpp(
    model_path  = modelFilePath,
    temperature = 0.75,
    max_tokens  = 2000,
    top_p       = 1,
    verbose     = False
)

conversationBufferMemory = ConversationBufferMemory(return_messages = True, memory_key = "chat_history")

runnableAssign = RunnablePassthrough.assign(chat_history = lambda x : conversationBufferMemory.load_memory_variables({})["chat_history"])

chatPromptTemplate = ChatPromptTemplate.from_messages(
    [
        ("system", "You are a python programmer."),
        MessagesPlaceholder(variable_name = "chat_history"),
        ("human", "{input}"),
    ]
)
runnableSequence = runnableAssign | chatPromptTemplate | llamaCpp | StrOutputParser()

while True:
    print()
    userInputString = input("User : ")
    if userInputString.lower() in ["quit", "exit", "bye", "종료"]:
        break
    chunkStringList = []
    for chunkString in runnableSequence.stream({"input" : userInputString}):
        print(chunkString, end = "", flush = True)
        chunkStringList += chunkString
    print()
    responseString = ''.join(chunkStringList)
    conversationBufferMemory.save_context({"input" : userInputString}, {"output" : responseString})

from langchain_community.llms import LlamaCpp

from langchain.memory import ConversationBufferMemory

from langchain_core.runnables import RunnablePassthrough

from langchain_core.prompts import MessagesPlaceholder

from langchain_core.prompts import ChatPromptTemplate

from langchain_core.output_parsers import StrOutputParser

modelFilePath = "./meta-llama-3.1-8b-instruct-abliterated.bf16.gguf"

llamaCpp = LlamaCpp(

model_path = modelFilePath,

temperature = 0.75,

max_tokens = 2000,

top_p = 1,

verbose = False

)

conversationBufferMemory = ConversationBufferMemory(return_messages = True, memory_key = "chat_history")

runnableAssign = RunnablePassthrough.assign(chat_history = lambda x : conversationBufferMemory.load_memory_variables({})["chat_history"])

chatPromptTemplate = ChatPromptTemplate.from_messages(

[

("system", "You are a python programmer."),

MessagesPlaceholder(variable_name = "chat_history"),

("human", "{input}"),

]

)

runnableSequence = runnableAssign | chatPromptTemplate | llamaCpp | StrOutputParser()

while True:

print()

userInputString = input("User : ")

if userInputString.lower() in ["quit", "exit", "bye", "종료"]:

break

chunkStringList = []

for chunkString in runnableSequence.stream({"input" : userInputString}):

print(chunkString, end = "", flush = True)

chunkStringList += chunkString

print()

responseString = ''.join(chunkStringList)

conversationBufferMemory.save_context({"input" : userInputString}, {"output" : responseString})

▶ requirements.txt


aiohappyeyeballs==2.3.7
aiohttp==3.10.4
aiosignal==1.3.1
annotated-types==0.7.0
async-timeout==4.0.3
attrs==24.2.0
certifi==2024.7.4
charset-normalizer==3.3.2
dataclasses-json==0.6.7
diskcache==5.6.3
frozenlist==1.4.1
greenlet==3.0.3
idna==3.7
Jinja2==3.1.4
jsonpatch==1.33
jsonpointer==3.0.0
langchain==0.2.14
langchain-community==0.2.12
langchain-core==0.2.33
langchain-text-splitters==0.2.2
langsmith==0.1.99
llama_cpp_python==0.2.88
MarkupSafe==2.1.5
marshmallow==3.21.3
multidict==6.0.5
mypy-extensions==1.0.0
numpy==1.26.4
orjson==3.10.7
packaging==24.1
pydantic==2.8.2
pydantic_core==2.20.1
PyYAML==6.0.2
requests==2.32.3
SQLAlchemy==2.0.32
tenacity==8.5.0
typing-inspect==0.9.0
typing_extensions==4.12.2
urllib3==2.2.2
yarl==1.9.4

aiohappyeyeballs==2.3.7

aiohttp==3.10.4

aiosignal==1.3.1

annotated-types==0.7.0

async-timeout==4.0.3

attrs==24.2.0

certifi==2024.7.4

charset-normalizer==3.3.2

dataclasses-json==0.6.7

diskcache==5.6.3

frozenlist==1.4.1

greenlet==3.0.3

idna==3.7

Jinja2==3.1.4

jsonpatch==1.33

jsonpointer==3.0.0

langchain==0.2.14

langchain-community==0.2.12

langchain-core==0.2.33

langchain-text-splitters==0.2.2

langsmith==0.1.99

llama_cpp_python==0.2.88

MarkupSafe==2.1.5

marshmallow==3.21.3

multidict==6.0.5

mypy-extensions==1.0.0

numpy==1.26.4

orjson==3.10.7

packaging==24.1

pydantic==2.8.2

pydantic_core==2.20.1

PyYAML==6.0.2

requests==2.32.3

SQLAlchemy==2.0.32

tenacity==8.5.0

typing-inspect==0.9.0

typing_extensions==4.12.2

urllib3==2.2.2

yarl==1.9.4

※ pip install

[PYTHON/TRANSFORMERS] TextGenerationPipeline 클래스 : Llama 3.1 모델을 사용해 대화하기

■ TextGenerationPipeline 클래스에서 Llama 3.1 모델을 사용해 대화하는 방법을 보여준다. ▶ main.py


import transformers
import torch

textGenerationPipeline = transformers.pipeline(
    "text-generation",
    model        = "meta-llama/Meta-Llama-3.1-8B-Instruct",
    model_kwargs = {"torch_dtype" : torch.bfloat16},
    device_map   = "auto",
)

messageList = [
    {"role" : "system", "content" : "You are a pirate chatbot who always responds in pirate speak!"},
    {"role" : "user"  , "content" : "Who are you?"                                                 }
]

resultList = textGenerationPipeline(
    messageList,
    max_new_tokens = 256,
)

print(resultList[0]["generated_text"][-1])

"""
{'role': 'assistant', 'content': "Arrrr, me hearty! I be Captain Chat, a swashbucklin' pirate chatbot, here to serve ye with me treasure trove o' knowledge! Me and me trusty keyboard be ready to set sail fer a sea o' conversation, answerin' yer questions and tellin' tales o' adventure on the seven seas! So hoist the sails and let's set course fer a fun-filled journey, matey!"}
"""

import transformers

import torch

textGenerationPipeline = transformers.pipeline(

"text-generation",

model = "meta-llama/Meta-Llama-3.1-8B-Instruct",

model_kwargs = {"torch_dtype" : torch.bfloat16},

device_map = "auto",

)

messageList = [

{"role" : "system", "content" : "You are a pirate chatbot who always responds in pirate speak!"},

{"role" : "user" , "content" : "Who are you?" }

]

resultList = textGenerationPipeline(

messageList,

max_new_tokens = 256,

)

print(resultList[0]["generated_text"][-1])

"""

{'role': 'assistant', 'content': "Arrrr, me hearty! I be Captain Chat, a swashbucklin' pirate chatbot, here to serve ye with me treasure trove o' knowledge! Me and me trusty keyboard be ready to set sail fer a sea o' conversation, answerin' yer questions and tellin' tales o' adventure on the seven seas! So hoist the sails and let's set course fer a fun-filled journey, matey!"}

"""

▶ requirements.txt


accelerate==0.33.0
certifi==2024.7.4
charset-normalizer==3.3.2
filelock==3.15.4
fsspec==2024.6.1
huggingface-hub==0.24.5
idna==3.7
Jinja2==3.1.4
MarkupSafe==2.1.5
mpmath==1.3.0
networkx==3.3
numpy==1.26.4
nvidia-cublas-cu12==12.1.3.1
nvidia-cuda-cupti-cu12==12.1.105
nvidia-cuda-nvrtc-cu12==12.1.105
nvidia-cuda-runtime-cu12==12.1.105
nvidia-cudnn-cu12==9.1.0.70
nvidia-cufft-cu12==11.0.2.54
nvidia-curand-cu12==10.3.2.106
nvidia-cusolver-cu12==11.4.5.107
nvidia-cusparse-cu12==12.1.0.106
nvidia-nccl-cu12==2.20.5
nvidia-nvjitlink-cu12==12.6.20
nvidia-nvtx-cu12==12.1.105
packaging==24.1
psutil==6.0.0
PyYAML==6.0.2
regex==2024.7.24
requests==2.32.3
safetensors==0.4.4
sympy==1.13.2
tokenizers==0.19.1
torch==2.4.0
tqdm==4.66.5
transformers==4.44.0
triton==3.0.0
typing_extensions==4.12.2
urllib3==2.2.2

accelerate==0.33.0

certifi==2024.7.4

charset-normalizer==3.3.2

filelock==3.15.4

fsspec==2024.6.1

huggingface-hub==0.24.5

idna==3.7

Jinja2==3.1.4

MarkupSafe==2.1.5

mpmath==1.3.0

networkx==3.3

numpy==1.26.4

nvidia-cublas-cu12==12.1.3.1

nvidia-cuda-cupti-cu12==12.1.105

nvidia-cuda-nvrtc-cu12==12.1.105

nvidia-cuda-runtime-cu12==12.1.105

nvidia-cudnn-cu12==9.1.0.70

nvidia-cufft-cu12==11.0.2.54

nvidia-curand-cu12==10.3.2.106

nvidia-cusolver-cu12==11.4.5.107

nvidia-cusparse-cu12==12.1.0.106

nvidia-nccl-cu12==2.20.5

nvidia-nvjitlink-cu12==12.6.20

nvidia-nvtx-cu12==12.1.105

packaging==24.1

psutil==6.0.0

PyYAML==6.0.2

regex==2024.7.24

requests==2.32.3

safetensors==0.4.4

sympy==1.13.2

tokenizers==0.19.1

torch==2.4.0

tqdm==4.66.5

transformers==4.44.0

triton==3.0.0

typing_extensions==4.12.2

urllib3==2.2.2

※ pip install transformers torch accelerate

[PYTHON/TRANSFORMERS] pipeline 함수 : Llama 3.1 모델을 설정한 TextGenerationPipeline 객체 만들기

■ pipeline 함수를 사용해 Llama 3.1 모델을 설정한 TextGenerationPipeline 객체를 만드는 방법을 보여준다. ▶ 예제 코드 (PY)


import transformers
import torch

textGenerationPipeline = transformers.pipeline(
    "text-generation",
    model        = "meta-llama/Meta-Llama-3.1-8B-Instruct",
    model_kwargs = {"torch_dtype" : torch.bfloat16},
    device_map   = "auto",
)

import transformers

import torch

textGenerationPipeline = transformers.pipeline(

"text-generation",

model = "meta-llama/Meta-Llama-3.1-8B-Instruct",

model_kwargs = {"torch_dtype" : torch.bfloat16},

device_map = "auto",

)

▶ requirements.txt


accelerate==0.33.0
certifi==2024.7.4
charset-normalizer==3.3.2
filelock==3.15.4
fsspec==2024.6.1
huggingface-hub==0.24.5
idna==3.7
Jinja2==3.1.4
MarkupSafe==2.1.5
mpmath==1.3.0
networkx==3.3
numpy==1.26.4
nvidia-cublas-cu12==12.1.3.1
nvidia-cuda-cupti-cu12==12.1.105
nvidia-cuda-nvrtc-cu12==12.1.105
nvidia-cuda-runtime-cu12==12.1.105
nvidia-cudnn-cu12==9.1.0.70
nvidia-cufft-cu12==11.0.2.54
nvidia-curand-cu12==10.3.2.106
nvidia-cusolver-cu12==11.4.5.107
nvidia-cusparse-cu12==12.1.0.106
nvidia-nccl-cu12==2.20.5
nvidia-nvjitlink-cu12==12.6.20
nvidia-nvtx-cu12==12.1.105
packaging==24.1
psutil==6.0.0
PyYAML==6.0.2
regex==2024.7.24
requests==2.32.3
safetensors==0.4.4
sympy==1.13.2
tokenizers==0.19.1
torch==2.4.0
tqdm==4.66.5
transformers==4.44.0
triton==3.0.0
typing_extensions==4.12.2
urllib3==2.2.2

accelerate==0.33.0

certifi==2024.7.4

charset-normalizer==3.3.2

filelock==3.15.4

fsspec==2024.6.1

huggingface-hub==0.24.5

idna==3.7

Jinja2==3.1.4

MarkupSafe==2.1.5

mpmath==1.3.0

networkx==3.3

numpy==1.26.4

nvidia-cublas-cu12==12.1.3.1

nvidia-cuda-cupti-cu12==12.1.105

nvidia-cuda-nvrtc-cu12==12.1.105

nvidia-cuda-runtime-cu12==12.1.105

nvidia-cudnn-cu12==9.1.0.70

nvidia-cufft-cu12==11.0.2.54

nvidia-curand-cu12==10.3.2.106

nvidia-cusolver-cu12==11.4.5.107

nvidia-cusparse-cu12==12.1.0.106

nvidia-nccl-cu12==2.20.5

nvidia-nvjitlink-cu12==12.6.20

nvidia-nvtx-cu12==12.1.105

packaging==24.1

psutil==6.0.0

PyYAML==6.0.2

regex==2024.7.24

requests==2.32.3

safetensors==0.4.4

sympy==1.13.2

tokenizers==0.19.1

torch==2.4.0

tqdm==4.66.5

transformers==4.44.0

triton==3.0.0

typing_extensions==4.12.2

urllib3==2.2.2

※

[PYTHON/HUGGINGFACE] huggingface-cli 명령 : download 인자를 사용해 Llama 3.1 모델 다운로드하기

■ huggingface-cli 명령에서 download 인자를 사용해 Llama 3.1 모델을 다운로드하는 방법을 보여준다. ▶ 실행 명령


huggingface-cli download meta-llama/Meta-Llama-3.1-8B-Instruct --include "original/*" --local-dir Meta-Llama-3.1-8B-Instruct

huggingface-cli download meta-llama/Meta-Llama-3.1-8B-Instruct --include "original/*" --local-dir Meta-Llama-3.1-8B-Instruct