LLAMA3 Archives - icodebroker

[PYTHON/LANGCHAIN] LlamaCpp 클래스 : Llama 3 모델을 사용해 질문하기

■ LlamaCpp 클래스에서 Llama 3 모델을 사용해 질문하는 방법을 보여준다. ▶ main.py


from langchain_community.llms      import LlamaCpp
from langchain_core.output_parsers import StrOutputParser

llamaCpp = LlamaCpp(
    model_path   = "./Meta-Llama-3-8B-Instruct.Q4_0.gguf",
    n_gpu_layers = 1,
    n_batch      = 512,
    n_ctx        = 2048,
    f16_kv       = True,
    verbose      = False,
    max_tokens   = 512
)

runnableSequence = llamaCpp | StrOutputParser()

for chunkString in runnableSequence.stream("윈도우즈 운영 체제에서 크롬 브라우저를 실행하는 C# 코드를 작성해주세요."):
    print(chunkString, end = "", flush = True)
print()

"""
```csharp
using System;
using System.Diagnostics;

class Program
{
    [STAThread]
    static void Main(string[] args)
    {
        // Chrome 브라우저의 경로를 설정합니다.
        string chromePath = @"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe";

        // C:\Users\<username>\AppData\Local\Google\Chrome\User Data\Default folder의 Default.ini file을 복사하여 새 파일에 붙여넣습니다.
        string defaultIniPath = @"C:\Users\<username>\AppData\Local\Google\Chrome\User Data\Default\default.ini";
        File.Copy(defaultIniPath, @"C:\temp\default.ini", true);

        // Chrome 브라우저를 실행합니다.
        ProcessStartInfo startInfo = new ProcessStartInfo();
        startInfo.FileName = chromePath;
        startInfo.Arguments = " --user-data-dir=C:\\temp\\chrome_user_data";
        startInfo.UseShellExecute = false;
        startInfo.CreateNoWindow = true;
        startInfo.RedirectStandardError = true;

        using (Process process = Process.Start(startInfo))
        {
            // Chrome 브라우저가 실행되면, 이 부분은 실행되지 않습니다.
            process.WaitForExit();
        }
    }
}
```

위 코드는 C#에서 크롬 브라우저를 실행하는 예시입니다.

1. Chrome 브라우저의 경로를 설정합니다.
2. C:\Users\<username>\AppData\Local\Google\Chrome\User Data\Default folder의 Default.ini file을 복사하여 새 파일에 붙여넣습니다.
3. Chrome 브라우저를 실행합니다. 실행할 때는, User Agent String이 포함된 Argument를 추가하여야 합니다.

위 코드를 실행하면, 크롬 브라우저가 실행되는 것을 볼 수 있습니다. 그러나 실제로는, 크롬 브라우저의 설정을 변경하거나, 새로운 브라우저 창을 열 수 있습니다. 따라서, 이 코드는 실제로는 크롬 브라우저를 실행하는 데 사용할 수 없습니다. 이를 해결하기 위해서는, 추가적인 로직이나, C#에서 브라우저와 같은 프로세스를 생성하여야 합니다. 이 경우에는, C#에서는 브라우저와 같은 프로세스를 생성하여야 합니다. 이러한 경우에는, C# 코드에서는 브라우저와 같은 프로세스를 생성하여야 합니다. 이를 해결하기 위해서는,
"""

from langchain_community.llms import LlamaCpp

from langchain_core.output_parsers import StrOutputParser

llamaCpp = LlamaCpp(

model_path = "./Meta-Llama-3-8B-Instruct.Q4_0.gguf",

n_gpu_layers = 1,

n_batch = 512,

n_ctx = 2048,

f16_kv = True,

verbose = False,

max_tokens = 512

)

runnableSequence = llamaCpp | StrOutputParser()

for chunkString in runnableSequence.stream("윈도우즈 운영 체제에서 크롬 브라우저를 실행하는 C# 코드를 작성해주세요."):

print(chunkString, end = "", flush = True)

print()

"""

```csharp

using System;

using System.Diagnostics;

class Program

{

[STAThread]

static void Main(string[] args)

{

// Chrome 브라우저의 경로를 설정합니다.

string chromePath = @"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe";

// C:\Users\<username>\AppData\Local\Google\Chrome\User Data\Default folder의 Default.ini file을 복사하여 새 파일에 붙여넣습니다.

string defaultIniPath = @"C:\Users\<username>\AppData\Local\Google\Chrome\User Data\Default\default.ini";

File.Copy(defaultIniPath, @"C:\temp\default.ini", true);

// Chrome 브라우저를 실행합니다.

ProcessStartInfo startInfo = new ProcessStartInfo();

startInfo.FileName = chromePath;

startInfo.Arguments = " --user-data-dir=C:\\temp\\chrome_user_data";

startInfo.UseShellExecute = false;

startInfo.CreateNoWindow = true;

startInfo.RedirectStandardError = true;

using (Process process = Process.Start(startInfo))

{

// Chrome 브라우저가 실행되면, 이 부분은 실행되지 않습니다.

process.WaitForExit();

}

```

위 코드는 C#에서 크롬 브라우저를 실행하는 예시입니다.

1. Chrome 브라우저의 경로를 설정합니다.

2. C:\Users\<username>\AppData\Local\Google\Chrome\User Data\Default folder의 Default.ini file을 복사하여 새 파일에 붙여넣습니다.

3. Chrome 브라우저를 실행합니다. 실행할 때는, User Agent String이 포함된 Argument를 추가하여야 합니다.

위 코드를 실행하면, 크롬 브라우저가 실행되는 것을 볼 수 있습니다. 그러나 실제로는, 크롬 브라우저의 설정을 변경하거나, 새로운 브라우저 창을 열 수 있습니다. 따라서, 이 코드는 실제로는 크롬 브라우저를 실행하는 데 사용할 수 없습니다. 이를 해결하기 위해서는, 추가적인 로직이나, C#에서 브라우저와 같은 프로세스를 생성하여야 합니다. 이 경우에는, C#에서는 브라우저와 같은 프로세스를 생성하여야 합니다. 이러한 경우에는, C# 코드에서는 브라우저와 같은 프로세스를 생성하여야 합니다. 이를 해결하기 위해서는,

"""

▶ requirements.txt


aiohappyeyeballs==2.3.7
aiohttp==3.10.4
aiosignal==1.3.1
annotated-types==0.7.0
async-timeout==4.0.3
attrs==24.2.0
certifi==2024.7.4
charset-normalizer==3.3.2
dataclasses-json==0.6.7
diskcache==5.6.3
frozenlist==1.4.1
greenlet==3.0.3
idna==3.7
Jinja2==3.1.4
jsonpatch==1.33
jsonpointer==3.0.0
langchain==0.2.14
langchain-community==0.2.12
langchain-core==0.2.33
langchain-text-splitters==0.2.2
langsmith==0.1.99
llama_cpp_python==0.2.88
MarkupSafe==2.1.5
marshmallow==3.21.3
multidict==6.0.5
mypy-extensions==1.0.0
numpy==1.26.4
orjson==3.10.7
packaging==24.1
pydantic==2.8.2
pydantic_core==2.20.1
PyYAML==6.0.2
requests==2.32.3
SQLAlchemy==2.0.32
tenacity==8.5.0
typing-inspect==0.9.0
typing_extensions==4.12.2
urllib3==2.2.2
yarl==1.9.4

aiohappyeyeballs==2.3.7

aiohttp==3.10.4

aiosignal==1.3.1

annotated-types==0.7.0

async-timeout==4.0.3

attrs==24.2.0

certifi==2024.7.4

charset-normalizer==3.3.2

dataclasses-json==0.6.7

diskcache==5.6.3

frozenlist==1.4.1

greenlet==3.0.3

idna==3.7

Jinja2==3.1.4

jsonpatch==1.33

jsonpointer==3.0.0

langchain==0.2.14

langchain-community==0.2.12

langchain-core==0.2.33

langchain-text-splitters==0.2.2

langsmith==0.1.99

llama_cpp_python==0.2.88

MarkupSafe==2.1.5

marshmallow==3.21.3

multidict==6.0.5

mypy-extensions==1.0.0

numpy==1.26.4

orjson==3.10.7

packaging==24.1

pydantic==2.8.2

pydantic_core==2.20.1

PyYAML==6.0.2

requests==2.32.3

SQLAlchemy==2.0.32

tenacity==8.5.0

typing-inspect==0.9.0

typing_extensions==4.12.2

urllib3==2.2.2

yarl==1.9.4

※ pip install langchain langchain-community llama-cpp-python

[C#/COMMON/.NET8] HttpClient 클래스 : OLLAMA에게 LLAMA 3 로컬 LLM 통신하기 (스트림)

■ HttpClient 클래스를 사용해 OLLAMA에게 LLAMA 3 로컬 LLM 통신하는 방법을 보여준다. (스트림) ▶ RequestMessage.cs


using Newtonsoft.Json;

namespace TestProject;

/// <summary>
/// 요청 메시지
/// </summary>
public class RequestMessage
{
    //////////////////////////////////////////////////////////////////////////////////////////////////// Property
    ////////////////////////////////////////////////////////////////////////////////////////// Public

    #region 모델 - Model

    /// <summary>
    /// 모델
    /// </summary>
    [JsonProperty("model")]
    public string Model { get; set; }

    #endregion
    #region 프롬프트 - Prompt

    /// <summary>
    /// 프롬프트
    /// </summary>
    [JsonProperty("prompt")]
    public string Prompt { get; set; }

    #endregion
}

using Newtonsoft.Json;

namespace TestProject;

/// <summary>

/// 요청 메시지

/// </summary>

public class RequestMessage

{

//////////////////////////////////////////////////////////////////////////////////////////////////// Property

////////////////////////////////////////////////////////////////////////////////////////// Public

#region 모델 - Model

/// <summary>

/// 모델

/// </summary>

[JsonProperty("model")]

public string Model { get; set; }

#endregion

#region 프롬프트 - Prompt

/// <summary>

/// 프롬프트

/// </summary>

[JsonProperty("prompt")]

public string Prompt { get; set; }

#endregion

}

▶ ResponseMessage.cs


using Newtonsoft.Json;

namespace TestProject;

/// <summary>
/// 응답 메시지
/// </summary>
public class ResponseMessage
{
    //////////////////////////////////////////////////////////////////////////////////////////////////// Property
    ////////////////////////////////////////////////////////////////////////////////////////// Public

    #region 모델 - Model

    /// <summary>
    /// 모델
    /// </summary>
    [JsonProperty("model")]
    public string Model { get; set; }

    #endregion
    #region 생성 시간 - CreateTime

    /// <summary>
    /// 생성 시간
    /// </summary>
    [JsonProperty("created_at")]
    public string CreateTime { get; set; }

    #endregion
    #region 응답 - Response

    /// <summary>
    /// 응답
    /// </summary>
    [JsonProperty("response")]
    public string Response { get; set; }

    #endregion
    #region 완료 여부 - Done

    /// <summary>
    /// 완료 여부
    /// </summary>
    [JsonProperty("done")]
    public bool Done { get; set; }

    #endregion
}

using Newtonsoft.Json;

namespace TestProject;

/// <summary>

/// 응답 메시지

/// </summary>

public class ResponseMessage

{

//////////////////////////////////////////////////////////////////////////////////////////////////// Property

////////////////////////////////////////////////////////////////////////////////////////// Public

#region 모델 - Model

/// <summary>

/// 모델

/// </summary>

[JsonProperty("model")]

public string Model { get; set; }

#endregion

#region 생성 시간 - CreateTime

/// <summary>

/// 생성 시간

/// </summary>

[JsonProperty("created_at")]

public string CreateTime { get; set; }

#endregion

#region 응답 - Response

/// <summary>

/// 응답

/// </summary>

[JsonProperty("response")]

public string Response { get; set; }

#endregion

#region 완료 여부 - Done

/// <summary>

/// 완료 여부

/// </summary>

[JsonProperty("done")]

public bool Done { get; set; }

#endregion

}

▶ APIClient.cs


using System.Text;
using System.Net.Http.Headers;

using Newtonsoft.Json;

namespace TestProject;

/// <summary>
/// API 클라이언트
/// </summary>
public class APIClient : IDisposable
{
    //////////////////////////////////////////////////////////////////////////////////////////////////// Event
    ////////////////////////////////////////////////////////////////////////////////////////// Public

    #region 메시지 도착시 이벤트 - MessageArrived

    /// <summary>
    /// 메시지 도착시 이벤트
    /// </summary>
    public event EventHandler<ResponseMessage> MessageArrived;

    #endregion

    //////////////////////////////////////////////////////////////////////////////////////////////////// Field
    ////////////////////////////////////////////////////////////////////////////////////////// Private

    #region Field

    /// <summary>
    /// 리소스 해제 여부
    /// </summary>
    private bool disposed = false;

    /// <summary>
    /// BASE URL
    /// </summary>
    private const string BASE_URL = "http://localhost:11434";

    /// <summary>
    /// HTTP 클라이언트
    /// </summary>
    private HttpClient client;

    /// <summary>
    /// 품질을 갖는 스트림 미디어 타입 헤더 값
    /// </summary>
    private MediaTypeWithQualityHeaderValue streamMediaTypeWithQualityHeaderValue;

    #endregion

    //////////////////////////////////////////////////////////////////////////////////////////////////// Property
    ////////////////////////////////////////////////////////////////////////////////////////// Public

    #region HTTP 클라이언트 - Client

    /// <summary>
    /// HTTP 클라이언트
    /// </summary>
    public HttpClient Client
    {
        get
        {
            return this.client;
        }
    }

    #endregion

    //////////////////////////////////////////////////////////////////////////////////////////////////// Constructor
    ////////////////////////////////////////////////////////////////////////////////////////// Public

    #region API 클라이언트 - APIClient()

    /// <summary>
    /// API 클라이언트
    /// </summary>
    public APIClient()
    {
        this.client = new HttpClient();

        this.streamMediaTypeWithQualityHeaderValue = new MediaTypeWithQualityHeaderValue("text/event-stream");
    }

    #endregion

    //////////////////////////////////////////////////////////////////////////////////////////////////// Method
    ////////////////////////////////////////////////////////////////////////////////////////// Public

    #region 채팅 요청하기 (비동기) - RequestChatAsync(requestMessage)

    /// <summary>
    /// 채팅 요청하기 (비동기)
    /// </summary>
    /// <param name="requestMessage">요청 메시지</param>
    /// <returns>응답 메시지 태스크</returns>
    public async Task<List<ResponseMessage>> RequestChatAsync(RequestMessage requestMessage)
    {
        if(this.client.DefaultRequestHeaders.Accept.Contains(this.streamMediaTypeWithQualityHeaderValue))
        {
            this.client.DefaultRequestHeaders.Accept.Remove(this.streamMediaTypeWithQualityHeaderValue);
        }

        string requestJSON = JsonConvert.SerializeObject(requestMessage);

        StringContent content = new StringContent(requestJSON, Encoding.UTF8, "application/json");

        HttpResponseMessage responseMessage = await this.client.PostAsync($"{BASE_URL}/api/generate", content);

        responseMessage.EnsureSuccessStatusCode();

        string responseString = await responseMessage.Content.ReadAsStringAsync();

        return JsonConvert.DeserializeObject<List<ResponseMessage>>($"[{responseString}]");
    }

    #endregion
    #region 채팅 요청하기 (스트림) (비동기) - RequestChatStreamAsync(requestMessage)

    /// <summary>
    /// 채팅 요청하기 (스트림) (비동기)
    /// </summary>
    /// <param name="requestMessage">요청 메시지</param>
    /// <returns>스트리밍 메시지 리스트 태스크</returns>
    public async Task<List<ResponseMessage>> RequestChatStreamAsync(RequestMessage requestMessage)
    {
        if(!this.client.DefaultRequestHeaders.Accept.Contains(this.streamMediaTypeWithQualityHeaderValue))
        {
            this.client.DefaultRequestHeaders.Accept.Add(this.streamMediaTypeWithQualityHeaderValue);
        }

        string requestJSON = JsonConvert.SerializeObject(requestMessage);

        StringContent content = new StringContent(requestJSON, Encoding.UTF8, "application/json");

        using var request = new HttpRequestMessage(HttpMethod.Post, $"{BASE_URL}/api/generate");

        request.Content = content;

        using HttpResponseMessage httpResponseMessage = await client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);

        httpResponseMessage.EnsureSuccessStatusCode();

        using Stream stream = await httpResponseMessage.Content.ReadAsStreamAsync();

        using StreamReader reader = new StreamReader(stream);

        List<ResponseMessage> responseMessageList = new List<ResponseMessage>();

        while(!reader.EndOfStream)
        {
            string line = await reader.ReadLineAsync();

            if(string.IsNullOrWhiteSpace(line))
            {
                continue;
            }

            ResponseMessage responseMessage = JsonConvert.DeserializeObject<ResponseMessage>(line);

            responseMessageList.Add(responseMessage);

            MessageArrived?.Invoke(this, responseMessage);
        }

        return responseMessageList;
    }

    #endregion

    #region 리소스 해제하기 - Dispose()

    /// <summary>
    /// 리소스 해제하기
    /// </summary>
    public void Dispose()
    {
        Dispose(true);

        GC.SuppressFinalize(this);
    }

    #endregion

    ////////////////////////////////////////////////////////////////////////////////////////// Protected

    #region 리소스 해제하기 - Dispose(disposing)

    /// <summary>
    /// 리소스 해제하기
    /// </summary>
    /// <param name="disposing">리소스 해제 여부</param>
    protected virtual void Dispose(bool disposing)
    {
        if(!this.disposed)
        {
            if(disposing)
            {
                if(this.client != null)
                {
                    this.client.Dispose();
                }

                this.client = null;

                this.disposed = true;
            }
        }
    }

    #endregion
}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

using System.Text;

using System.Net.Http.Headers;

using Newtonsoft.Json;

namespace TestProject;

/// <summary>

/// API 클라이언트

/// </summary>

public class APIClient : IDisposable

{

//////////////////////////////////////////////////////////////////////////////////////////////////// Event

////////////////////////////////////////////////////////////////////////////////////////// Public

#region 메시지 도착시 이벤트 - MessageArrived

/// <summary>

/// 메시지 도착시 이벤트

/// </summary>

public event EventHandler<ResponseMessage> MessageArrived;

#endregion

//////////////////////////////////////////////////////////////////////////////////////////////////// Field

////////////////////////////////////////////////////////////////////////////////////////// Private

#region Field

/// <summary>

/// 리소스 해제 여부

/// </summary>

private bool disposed = false;

/// <summary>

/// BASE URL

/// </summary>

private const string BASE_URL = "http://localhost:11434";

/// <summary>

/// HTTP 클라이언트

/// </summary>

private HttpClient client;

/// <summary>

/// 품질을 갖는 스트림 미디어 타입 헤더 값

/// </summary>

private MediaTypeWithQualityHeaderValue streamMediaTypeWithQualityHeaderValue;

#endregion

//////////////////////////////////////////////////////////////////////////////////////////////////// Property

////////////////////////////////////////////////////////////////////////////////////////// Public

#region HTTP 클라이언트 - Client

/// <summary>

/// HTTP 클라이언트

/// </summary>

public HttpClient Client

{

get

{

return this.client;

}

#endregion

//////////////////////////////////////////////////////////////////////////////////////////////////// Constructor

////////////////////////////////////////////////////////////////////////////////////////// Public

#region API 클라이언트 - APIClient()

/// <summary>

/// API 클라이언트

/// </summary>

public APIClient()

{

this.client = new HttpClient();

this.streamMediaTypeWithQualityHeaderValue = new MediaTypeWithQualityHeaderValue("text/event-stream");

}

#endregion

//////////////////////////////////////////////////////////////////////////////////////////////////// Method

////////////////////////////////////////////////////////////////////////////////////////// Public

#region 채팅 요청하기 (비동기) - RequestChatAsync(requestMessage)

/// <summary>

/// 채팅 요청하기 (비동기)

/// </summary>

/// <param name="requestMessage">요청 메시지</param>

/// <returns>응답 메시지 태스크</returns>

public async Task<List<ResponseMessage>> RequestChatAsync(RequestMessage requestMessage)

{

if(this.client.DefaultRequestHeaders.Accept.Contains(this.streamMediaTypeWithQualityHeaderValue))

{

this.client.DefaultRequestHeaders.Accept.Remove(this.streamMediaTypeWithQualityHeaderValue);

}

string requestJSON = JsonConvert.SerializeObject(requestMessage);

StringContent content = new StringContent(requestJSON, Encoding.UTF8, "application/json");

HttpResponseMessage responseMessage = await this.client.PostAsync($"{BASE_URL}/api/generate", content);

responseMessage.EnsureSuccessStatusCode();

string responseString = await responseMessage.Content.ReadAsStringAsync();

return JsonConvert.DeserializeObject<List<ResponseMessage>>($"[{responseString}]");

}

#endregion

#region 채팅 요청하기 (스트림) (비동기) - RequestChatStreamAsync(requestMessage)

/// <summary>

/// 채팅 요청하기 (스트림) (비동기)

/// </summary>

/// <param name="requestMessage">요청 메시지</param>

/// <returns>스트리밍 메시지 리스트 태스크</returns>

public async Task<List<ResponseMessage>> RequestChatStreamAsync(RequestMessage requestMessage)

{

if(!this.client.DefaultRequestHeaders.Accept.Contains(this.streamMediaTypeWithQualityHeaderValue))

{

this.client.DefaultRequestHeaders.Accept.Add(this.streamMediaTypeWithQualityHeaderValue);

}

string requestJSON = JsonConvert.SerializeObject(requestMessage);

StringContent content = new StringContent(requestJSON, Encoding.UTF8, "application/json");

using var request = new HttpRequestMessage(HttpMethod.Post, $"{BASE_URL}/api/generate");

request.Content = content;

using HttpResponseMessage httpResponseMessage = await client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);

httpResponseMessage.EnsureSuccessStatusCode();

using Stream stream = await httpResponseMessage.Content.ReadAsStreamAsync();

using StreamReader reader = new StreamReader(stream);

List<ResponseMessage> responseMessageList = new List<ResponseMessage>();

while(!reader.EndOfStream)

{

string line = await reader.ReadLineAsync();

if(string.IsNullOrWhiteSpace(line))

{

continue;

}

ResponseMessage responseMessage = JsonConvert.DeserializeObject<ResponseMessage>(line);

responseMessageList.Add(responseMessage);

MessageArrived?.Invoke(this, responseMessage);

}

return responseMessageList;

}

#endregion

#region 리소스 해제하기 - Dispose()

/// <summary>

/// 리소스 해제하기

/// </summary>

public void Dispose()

{

Dispose(true);

GC.SuppressFinalize(this);

}

#endregion

////////////////////////////////////////////////////////////////////////////////////////// Protected

#region 리소스 해제하기 - Dispose(disposing)

/// <summary>

/// 리소스 해제하기

/// </summary>

/// <param name="disposing">리소스 해제 여부</param>

protected virtual void Dispose(bool disposing)

{

if(!this.disposed)

{

if(disposing)

{

if(this.client != null)

{

this.client.Dispose();

}

this.client = null;

this.disposed = true;

}

#endregion

}

[C#/COMMON/.NET8] HttpClient 클래스 : OLLAMA에게 LLAMA 3 로컬 LLM 통신하기

■ HttpClient 클래스를 사용해 OLLAMA에게 LLAMA 3 로컬 LLM 통신하는 방법을 보여준다. ▶ RequestMessage.cs


using Newtonsoft.Json;

namespace TestProject;

/// <summary>
/// 요청 메시지
/// </summary>
public class RequestMessage
{
    //////////////////////////////////////////////////////////////////////////////////////////////////// Property
    ////////////////////////////////////////////////////////////////////////////////////////// Public

    #region 모델 - Model

    /// <summary>
    /// 모델
    /// </summary>
    [JsonProperty("model")]
    public string Model { get; set; }

    #endregion
    #region 프롬프트 - Prompt

    /// <summary>
    /// 프롬프트
    /// </summary>
    [JsonProperty("prompt")]
    public string Prompt { get; set; }

    #endregion
}

using Newtonsoft.Json;

namespace TestProject;

/// <summary>

/// 요청 메시지

/// </summary>

public class RequestMessage

{

//////////////////////////////////////////////////////////////////////////////////////////////////// Property

////////////////////////////////////////////////////////////////////////////////////////// Public

#region 모델 - Model

/// <summary>

/// 모델

/// </summary>

[JsonProperty("model")]

public string Model { get; set; }

#endregion

#region 프롬프트 - Prompt

/// <summary>

/// 프롬프트

/// </summary>

[JsonProperty("prompt")]

public string Prompt { get; set; }

#endregion

}

▶ Program.cs


using System.Text;

using Newtonsoft.Json;

namespace TestProject;

/// <summary>
/// 프로그램
/// </summary>
class Program
{
    //////////////////////////////////////////////////////////////////////////////////////////////////// Method
    ////////////////////////////////////////////////////////////////////////////////////////// Static
    //////////////////////////////////////////////////////////////////////////////// Private

    #region 프로그램 시작하기 - Main()

    /// <summary>
    /// 프로그램 시작하기
    /// </summary>
    /// <returns>태스크</returns>
    static async Task Main()
    {
        using HttpClient httpClient = new HttpClient();

        httpClient.BaseAddress = new Uri("http://localhost:11434");

        RequestMessage requestMessage = new() { Model = "llama3", Prompt = "Hello, how are you?" };

        StringContent stringContent = new
        (
            JsonConvert.SerializeObject(requestMessage),
            Encoding.UTF8,
            "application/json"
        );

        HttpResponseMessage httpResponseMessage = await httpClient.PostAsync("/api/generate", stringContent);

        string responseString = await httpResponseMessage.Content.ReadAsStringAsync();

        Console.WriteLine(responseString);
    }

    #endregion
}

using System.Text;

using Newtonsoft.Json;

namespace TestProject;

/// <summary>

/// 프로그램

/// </summary>

class Program

{

//////////////////////////////////////////////////////////////////////////////////////////////////// Method

////////////////////////////////////////////////////////////////////////////////////////// Static

//////////////////////////////////////////////////////////////////////////////// Private

#region 프로그램 시작하기 - Main()

/// <summary>

/// 프로그램 시작하기

/// </summary>

/// <returns>태스크</returns>

static async Task Main()

{

using HttpClient httpClient = new HttpClient();

httpClient.BaseAddress = new Uri("http://localhost:11434");

RequestMessage requestMessage = new() { Model = "llama3", Prompt = "Hello, how are you?" };

StringContent stringContent = new

(

JsonConvert.SerializeObject(requestMessage),

Encoding.UTF8,

"application/json"

);

HttpResponseMessage httpResponseMessage = await httpClient.PostAsync("/api/generate", stringContent);

string responseString = await httpResponseMessage.Content.ReadAsStringAsync();

Console.WriteLine(responseString);

}

#endregion

}

TestProject.zip

[OS/UBUNTU] ollama pull 명령 : 모델을 레지스트리에서 다운로드하기

■ ollama pull 명령 : 모델을 레지스트리에서 다운로드하는 방법을 보여준다. 1. CTRL + ALT + T 키를 눌러서 [터미널]을 실행한다. 2. [터미널]에서

[OS/UBUNTU] ollama run 명령 : 모델 실행하기

■ ollama run 명령 : 모델을 실행하는 방법을 보여준다. 1. CTRL + ALT + T 키를 눌러서 [터미널]을 실행한다. 2. [터미널]에서 아래

[PYTHON/LANGCHAIN] Ollama 클래스 : 생성자에서 callback_manager 인자를 사용해 스트리밍 표준 출력 콜백 핸들러 설정하기

■ Ollama 클래스의 생성자에서 callback_manager 인자를 사용해 스트리밍 표준 출력 콜백 핸들러를 설정하는 방법을 보여준다. ▶ main.py


from langchain_community.llms import Ollama
from langchain_core.callbacks import StreamingStdOutCallbackHandler
from langchain_core.callbacks import CallbackManager

ollama = Ollama(model = "llama3", callback_manager = CallbackManager([StreamingStdOutCallbackHandler()]))

ollama.invoke("The first man on the moon was ...")

"""
A classic question!

The answer is: Neil Armstrong.

On July 20, 1969, NASA's Apollo 11 mission successfully landed two astronauts on the surface of the Moon. The first person to set foot on the Moon was Neil Armstrong, who famously declared: "That's one small step for man, one giant leap for mankind" as he stepped onto the lunar surface.

Armstrong was followed by fellow astronaut Edwin "Buzz" Aldrin, who also walked on the Moon during the mission.
"""

from langchain_community.llms import Ollama

from langchain_core.callbacks import StreamingStdOutCallbackHandler

from langchain_core.callbacks import CallbackManager

ollama = Ollama(model = "llama3", callback_manager = CallbackManager([StreamingStdOutCallbackHandler()]))

ollama.invoke("The first man on the moon was ...")

"""

A classic question!

The answer is: Neil Armstrong.

On July 20, 1969, NASA's Apollo 11 mission successfully landed two astronauts on the surface of the Moon. The first person to set foot on the Moon was Neil Armstrong, who famously declared: "That's one small step for man, one giant leap for mankind" as he stepped onto the lunar surface.

Armstrong was followed by fellow astronaut Edwin "Buzz" Aldrin, who also walked on the Moon during the mission.

"""

▶ requirements.txt


aiohttp==3.9.5
aiosignal==1.3.1
annotated-types==0.7.0
async-timeout==4.0.3
attrs==23.2.0
certifi==2024.6.2
charset-normalizer==3.3.2
dataclasses-json==0.6.7
frozenlist==1.4.1
greenlet==3.0.3
idna==3.7
jsonpatch==1.33
jsonpointer==3.0.0
langchain==0.2.6
langchain-community==0.2.6
langchain-core==0.2.10
langchain-text-splitters==0.2.2
langsmith==0.1.82
marshmallow==3.21.3
multidict==6.0.5
mypy-extensions==1.0.0
numpy==1.26.4
orjson==3.10.5
packaging==24.1
pydantic==2.7.4
pydantic_core==2.18.4
PyYAML==6.0.1
requests==2.32.3
SQLAlchemy==2.0.31
tenacity==8.4.2
typing-inspect==0.9.0
typing_extensions==4.12.2
urllib3==2.2.2
yarl==1.9.4

aiohttp==3.9.5

aiosignal==1.3.1

annotated-types==0.7.0

async-timeout==4.0.3

attrs==23.2.0

certifi==2024.6.2

charset-normalizer==3.3.2

dataclasses-json==0.6.7

frozenlist==1.4.1

greenlet==3.0.3

idna==3.7

jsonpatch==1.33

jsonpointer==3.0.0

langchain==0.2.6

langchain-community==0.2.6

langchain-core==0.2.10

langchain-text-splitters==0.2.2

langsmith==0.1.82

marshmallow==3.21.3

multidict==6.0.5

mypy-extensions==1.0.0

numpy==1.26.4

orjson==3.10.5

packaging==24.1

pydantic==2.7.4

pydantic_core==2.18.4

PyYAML==6.0.1

requests==2.32.3

SQLAlchemy==2.0.31

tenacity==8.4.2

typing-inspect==0.9.0

typing_extensions==4.12.2

urllib3==2.2.2

yarl==1.9.4

※

[PYTHON/OLLAMA] generate 함수 : LLAMA 3 모델에서 질의 응답하기

■ generate 함수를 사용해 LLAMA 3 모델에서 질의 응답하는 방법을 보여준다. ▶ main.py


import ollama

responseDictionary = ollama.generate(model = "llama3", prompt = "what is 4 + 1000?")

print(responseDictionary["response"])

"""
The answer to 4 + 1000 is 1004.
"""

import ollama

responseDictionary = ollama.generate(model = "llama3", prompt = "what is 4 + 1000?")

print(responseDictionary["response"])

"""

The answer to 4 + 1000 is 1004.

"""

▶ requirements.txt


anyio==4.4.0
certifi==2024.6.2
exceptiongroup==1.2.1
h11==0.14.0
httpcore==1.0.5
httpx==0.27.0
idna==3.7
ollama==0.2.1
sniffio==1.3.1
typing_extensions==4.12.2

anyio==4.4.0

certifi==2024.6.2

exceptiongroup==1.2.1

h11==0.14.0

httpcore==1.0.5

httpx==0.27.0

idna==3.7

ollama==0.2.1

sniffio==1.3.1

typing_extensions==4.12.2

※ pip install ollama 명령을

[PYTHON/LANGCHAIN] Ollama 클래스 : invoke 메소드를 사용해 Llama 3 모델에서 질의 응답하기

■ Ollama 클래스의 invoke 메소드를 사용해 Llama 3 모델에서 질의 응답하는 방법을 보여준다. ※ 참조 : "OLLAMA 설치하기 (UBUNTU)" ▶ main.py


from langchain_community.llms import Ollama

ollama = Ollama(model = "llama3")

responseString = ollama.invoke("The first man on the moon was ...")

print(responseString)

"""
A classic question!

According to historical records and NASA, the first humans to set foot on the Moon were Neil Armstrong and Edwin "Buzz" Aldrin. They landed on July 20, 1969, as part of the Apollo 11 mission.

Neil Armstrong stepped out of the lunar module Eagle and onto the Moon's surface at 2:56 UTC on July 21, famously declaring, "That's one small step for man, one giant leap for mankind." Buzz Aldrin joined him shortly afterwards.
"""

from langchain_community.llms import Ollama

ollama = Ollama(model = "llama3")

responseString = ollama.invoke("The first man on the moon was ...")

print(responseString)

"""

A classic question!

According to historical records and NASA, the first humans to set foot on the Moon were Neil Armstrong and Edwin "Buzz" Aldrin. They landed on July 20, 1969, as part of the Apollo 11 mission.

Neil Armstrong stepped out of the lunar module Eagle and onto the Moon's surface at 2:56 UTC on July 21, famously declaring, "That's one small step for man, one giant leap for mankind." Buzz Aldrin joined him shortly afterwards.

"""

[PYTHON/OLLAMA] OLLAMA 설치하기 (UBUNTU)

■ OLLAMA를 설치하는 방법을 보여준다. (UBUNTU) 1. CTRL + ALT + T 키를 눌러서 [터미널]을 실행한다. 2. [터미널]에서 아래 명령을 실행해 ollama