FILE Archives - 19 중 3 번째 페이지

[PYTHON/COMMON] PosixPath 클래스 : read_text 메소드를 사용해 텍스트 파일 읽기

■ PosixPath 클래스의 read_text 메소드를 사용해 텍스트 파일을 읽는 방법을 보여준다. ▶ 예제 코드 (PY)


from pathlib import Path

posixPath = Path("settings.json")

jsonString = posixPath.read_text()

print(jsonString)

"""
{
  "MainFormLeft": 0,
  "MainFormTop": 0,
  "MainFormWidth": 920,
  "MainFormHeight": 690,
  "MainFormState": "Maximized",
  "SkinStyle": "Office 2016 Colorful",
  "RibbonMinimized": true,
  "SidebarStyle": "NavigationPane",
  "SidebarWidth": 100,
  "SidebarCollapsable": true,
  "SidebarVisible": true
}
"""

from pathlib import Path

posixPath = Path("settings.json")

jsonString = posixPath.read_text()

print(jsonString)

"""

{

"MainFormLeft": 0,

"MainFormTop": 0,

"MainFormWidth": 920,

"MainFormHeight": 690,

"MainFormState": "Maximized",

"SkinStyle": "Office 2016 Colorful",

"RibbonMinimized": true,

"SidebarStyle": "NavigationPane",

"SidebarWidth": 100,

"SidebarCollapsable": true,

"SidebarVisible": true

}

"""

[PYTHON/COMMON] Path 클래스 : 생성자를 사용해 PosixPath 객체 만들기

■ Path 클래스의 생성자를 사용해 PosixPath 객체를 만드는 방법을 보여준다. ▶ 예제 코드 (PY)


from pathlib import Path

posixPath = Path("./settings.json")

from pathlib import Path

posixPath = Path("./settings.json")

[PYTHON/LANGCHAIN] UnstructuredHTMLLoader 클래스 : load 메소드를 사용해 HTML 파일 문서 로드하기

■ UnstructuredHTMLLoader 클래스의 load 메소드를 사용해 HTML 파일 문서를 로드하는 방법을 보여준다. ▶ main.py


from langchain_community.document_loaders import UnstructuredHTMLLoader

unstructuredHTMLLoader = UnstructuredHTMLLoader("source.html")

documentList = unstructuredHTMLLoader.load()

for document in documentList:
    print(document)
    print()

"""
page_content='HTML을 로드하는 방법\n\n하이퍼텍스트 마크업 언어(HyperText Markup Language, HTML)는 웹 브라우저에 표시되도록 설계된 문서의 표준 마크업 언어입니다.\n\n이는 다운스트림에서 사용할 수 있는 LangChain 문서 개체에 HTML 문서를 로드하는 방법을 다룹니다.\n\nHTML 파일을 구문 분석하려면 특수 도구가 필요한 경우가 많습니다. 여기서는 pip를 통해 설치할 수 있는 Unstructured 및 BeautifulSoup4를 통한 구문 분석을 보여줍니다. Azure AI Document Intelligence 또는 FireCrawl과 같은 추가 서비스와의 통합을 찾으려면 통합 페이지로 이동하세요.' metadata={'source': 'source.html'}
"""

from langchain_community.document_loaders import UnstructuredHTMLLoader

unstructuredHTMLLoader = UnstructuredHTMLLoader("source.html")

documentList = unstructuredHTMLLoader.load()

for document in documentList:

print(document)

print()

"""

page_content='HTML을 로드하는 방법\n\n하이퍼텍스트 마크업 언어(HyperText Markup Language, HTML)는 웹 브라우저에 표시되도록 설계된 문서의 표준 마크업 언어입니다.\n\n이는 다운스트림에서 사용할 수 있는 LangChain 문서 개체에 HTML 문서를 로드하는 방법을 다룹니다.\n\nHTML 파일을 구문 분석하려면 특수 도구가 필요한 경우가 많습니다. 여기서는 pip를 통해 설치할 수 있는 Unstructured 및 BeautifulSoup4를 통한 구문 분석을 보여줍니다. Azure AI Document Intelligence 또는 FireCrawl과 같은 추가 서비스와의 통합을 찾으려면 통합 페이지로 이동하세요.' metadata={'source': 'source.html'}

"""

▶ requirements.txt


aiohttp==3.9.5
aiosignal==1.3.1
annotated-types==0.7.0
anyio==4.4.0
async-timeout==4.0.3
attrs==23.2.0
backoff==2.2.1
beautifulsoup4==4.12.3
certifi==2024.6.2
chardet==5.2.0
charset-normalizer==3.3.2
click==8.1.7
dataclasses-json==0.6.7
deepdiff==7.0.1
emoji==2.12.1
exceptiongroup==1.2.1
filetype==1.2.0
frozenlist==1.4.1
greenlet==3.0.3
h11==0.14.0
httpcore==1.0.5
httpx==0.27.0
idna==3.7
joblib==1.4.2
jsonpatch==1.33
jsonpath-python==1.0.6
jsonpointer==3.0.0
langchain==0.2.6
langchain-community==0.2.6
langchain-core==0.2.10
langchain-text-splitters==0.2.2
langdetect==1.0.9
langsmith==0.1.82
lxml==5.2.2
marshmallow==3.21.3
multidict==6.0.5
mypy-extensions==1.0.0
nest-asyncio==1.6.0
nltk==3.8.1
numpy==1.26.4
ordered-set==4.1.0
orjson==3.10.5
packaging==24.1
pydantic==2.7.4
pydantic_core==2.18.4
pypdf==4.2.0
python-dateutil==2.9.0.post0
python-iso639==2024.4.27
python-magic==0.4.27
PyYAML==6.0.1
rapidfuzz==3.9.3
regex==2024.5.15
requests==2.32.3
requests-toolbelt==1.0.0
six==1.16.0
sniffio==1.3.1
soupsieve==2.5
SQLAlchemy==2.0.31
tabulate==0.9.0
tenacity==8.4.2
tqdm==4.66.4
typing-inspect==0.9.0
typing_extensions==4.12.2
unstructured==0.14.8
unstructured-client==0.23.7
urllib3==2.2.2
wrapt==1.16.0
yarl==1.9.4

aiohttp==3.9.5

aiosignal==1.3.1

annotated-types==0.7.0

anyio==4.4.0

async-timeout==4.0.3

attrs==23.2.0

backoff==2.2.1

beautifulsoup4==4.12.3

certifi==2024.6.2

chardet==5.2.0

charset-normalizer==3.3.2

click==8.1.7

dataclasses-json==0.6.7

deepdiff==7.0.1

emoji==2.12.1

exceptiongroup==1.2.1

filetype==1.2.0

frozenlist==1.4.1

greenlet==3.0.3

h11==0.14.0

httpcore==1.0.5

httpx==0.27.0

idna==3.7

joblib==1.4.2

jsonpatch==1.33

jsonpath-python==1.0.6

jsonpointer==3.0.0

langchain==0.2.6

langchain-community==0.2.6

langchain-core==0.2.10

langchain-text-splitters==0.2.2

langdetect==1.0.9

langsmith==0.1.82

lxml==5.2.2

marshmallow==3.21.3

multidict==6.0.5

mypy-extensions==1.0.0

nest-asyncio==1.6.0

nltk==3.8.1

numpy==1.26.4

ordered-set==4.1.0

orjson==3.10.5

packaging==24.1

pydantic==2.7.4

pydantic_core==2.18.4

pypdf==4.2.0

python-dateutil==2.9.0.post0

python-iso639==2024.4.27

python-magic==0.4.27

PyYAML==6.0.1

rapidfuzz==3.9.3

regex==2024.5.15

requests==2.32.3

requests-toolbelt==1.0.0

six==1.16.0

sniffio==1.3.1

soupsieve==2.5

SQLAlchemy==2.0.31

tabulate==0.9.0

tenacity==8.4.2

tqdm==4.66.4

typing-inspect==0.9.0

typing_extensions==4.12.2

unstructured==0.14.8

unstructured-client==0.23.7

urllib3==2.2.2

wrapt==1.16.0

yarl==1.9.4

※ pip install langchain-community

[PYTHON/LANGCHAIN] DirectoryLoader 클래스 : 생성자에서 loader_kwargs 인자를 사용해 파일 인코딩 자동 감지하기

■ DirectoryLoader 클래스의 생성자에서 loader_kwargs 인자를 사용해 파일 인코딩을 자동 감지하는 방법을 보여준다. ▶ main.py


from langchain_community.document_loaders import TextLoader
from langchain_community.document_loaders import DirectoryLoader

directoryLoader = DirectoryLoader(".", glob = "*.txt", loader_cls = TextLoader, loader_kwargs = {"autodetect_encoding" : True})

documentList = directoryLoader.load()

for document in documentList:
    print(document)
    print()

"""
page_content='테스트 문자열입니다.' metadata={'source': 'test-non-utf8.txt'}

page_content='테스트 문자열입니다.' metadata={'source': 'text-utf8.txt'}
"""

from langchain_community.document_loaders import TextLoader

from langchain_community.document_loaders import DirectoryLoader

directoryLoader = DirectoryLoader(".", glob = "*.txt", loader_cls = TextLoader, loader_kwargs = {"autodetect_encoding" : True})

documentList = directoryLoader.load()

for document in documentList:

print(document)

print()

"""

page_content='테스트 문자열입니다.' metadata={'source': 'test-non-utf8.txt'}

page_content='테스트 문자열입니다.' metadata={'source': 'text-utf8.txt'}

"""

▶ requirements.txt


aiohttp==3.9.5
aiosignal==1.3.1
annotated-types==0.7.0
async-timeout==4.0.3
attrs==23.2.0
certifi==2024.6.2
chardet==5.2.0
charset-normalizer==3.3.2
dataclasses-json==0.6.7
frozenlist==1.4.1
greenlet==3.0.3
idna==3.7
jsonpatch==1.33
jsonpointer==3.0.0
langchain==0.2.6
langchain-community==0.2.6
langchain-core==0.2.10
langchain-text-splitters==0.2.2
langsmith==0.1.82
marshmallow==3.21.3
multidict==6.0.5
mypy-extensions==1.0.0
numpy==1.26.4
orjson==3.10.5
packaging==24.1
pydantic==2.7.4
pydantic_core==2.18.4
PyYAML==6.0.1
requests==2.32.3
SQLAlchemy==2.0.31
tenacity==8.4.2
typing-inspect==0.9.0
typing_extensions==4.12.2
urllib3==2.2.2
yarl==1.9.4

aiohttp==3.9.5

aiosignal==1.3.1

annotated-types==0.7.0

async-timeout==4.0.3

attrs==23.2.0

certifi==2024.6.2

chardet==5.2.0

charset-normalizer==3.3.2

dataclasses-json==0.6.7

frozenlist==1.4.1

greenlet==3.0.3

idna==3.7

jsonpatch==1.33

jsonpointer==3.0.0

langchain==0.2.6

langchain-community==0.2.6

langchain-core==0.2.10

langchain-text-splitters==0.2.2

langsmith==0.1.82

marshmallow==3.21.3

multidict==6.0.5

mypy-extensions==1.0.0

numpy==1.26.4

orjson==3.10.5

packaging==24.1

pydantic==2.7.4

pydantic_core==2.18.4

PyYAML==6.0.1

requests==2.32.3

SQLAlchemy==2.0.31

tenacity==8.4.2

typing-inspect==0.9.0

typing_extensions==4.12.2

urllib3==2.2.2

yarl==1.9.4

※ pip install

[PYTHON/LANGCHAIN] DirectoryLoader 클래스 : 생성자에서 silent_errors 인자를 사용해 파일 인코딩 에러 발생시키기

■ DirectoryLoader 클래스의 생성자에서 silent_errors 인자를 사용해 파일 인코딩 에러를 발생시키는 방법을 보여준다. ※ silent_errors 인자의 디폴트 값은 True이다. ▶ main.py


from langchain_community.document_loaders import TextLoader
from langchain_community.document_loaders import DirectoryLoader

directoryLoader = DirectoryLoader(".", glob = "*.txt", loader_cls = TextLoader, silent_errors = False)

documentList = directoryLoader.load()

for document in documentList:
    print(document)
    print()

"""
Error loading file test-non-utf8.txt
Traceback (most recent call last):
  File "/home/king/testproject/env/lib/python3.10/site-packages/langchain_community/document_loaders/text.py", line 43, in lazy_load
    text = f.read()
  File "/usr/lib/python3.10/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc5 in position 0: invalid continuation byte

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/king/testproject/main.py", line 6, in <module>
    documentList = directoryLoader.load()
  File "/home/king/testproject/env/lib/python3.10/site-packages/langchain_community/document_loaders/directory.py", line 117, in load
    return list(self.lazy_load())
  File "/home/king/testproject/env/lib/python3.10/site-packages/langchain_community/document_loaders/directory.py", line 195, in lazy_load
    yield from self._lazy_load_file(i, p, pbar)
  File "/home/king/testproject/env/lib/python3.10/site-packages/langchain_community/document_loaders/directory.py", line 233, in _lazy_load_file
    raise e
  File "/home/king/testproject/env/lib/python3.10/site-packages/langchain_community/document_loaders/directory.py", line 223, in _lazy_load_file
    for subdoc in loader.lazy_load():
  File "/home/king/testproject/env/lib/python3.10/site-packages/langchain_community/document_loaders/text.py", line 56, in lazy_load
    raise RuntimeError(f"Error loading {self.file_path}") from e
RuntimeError: Error loading test-non-utf8.txt
"""

from langchain_community.document_loaders import TextLoader

from langchain_community.document_loaders import DirectoryLoader

directoryLoader = DirectoryLoader(".", glob = "*.txt", loader_cls = TextLoader, silent_errors = False)

documentList = directoryLoader.load()

for document in documentList:

print(document)

print()

"""

Error loading file test-non-utf8.txt

Traceback (most recent call last):

File "/home/king/testproject/env/lib/python3.10/site-packages/langchain_community/document_loaders/text.py", line 43, in lazy_load

text = f.read()

File "/usr/lib/python3.10/codecs.py", line 322, in decode

(result, consumed) = self._buffer_decode(data, self.errors, final)

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc5 in position 0: invalid continuation byte

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

File "/home/king/testproject/main.py", line 6, in <module>

documentList = directoryLoader.load()

File "/home/king/testproject/env/lib/python3.10/site-packages/langchain_community/document_loaders/directory.py", line 117, in load

return list(self.lazy_load())

File "/home/king/testproject/env/lib/python3.10/site-packages/langchain_community/document_loaders/directory.py", line 195, in lazy_load

yield from self._lazy_load_file(i, p, pbar)

File "/home/king/testproject/env/lib/python3.10/site-packages/langchain_community/document_loaders/directory.py", line 233, in _lazy_load_file

raise e

File "/home/king/testproject/env/lib/python3.10/site-packages/langchain_community/document_loaders/directory.py", line 223, in _lazy_load_file

for subdoc in loader.lazy_load():

File "/home/king/testproject/env/lib/python3.10/site-packages/langchain_community/document_loaders/text.py", line 56, in lazy_load

raise RuntimeError(f"Error loading {self.file_path}") from e

RuntimeError: Error loading test-non-utf8.txt

"""

[PYTHON/LANGCHAIN] DirectoryLoader 클래스 : 생성자에서 loader_cls 인자를 PythonLoader 로더 클래스 설정하기

■ DirectoryLoader 클래스의 생성자에서 loader_cls 인자를 PythonLoader 로더 클래스로 설정하는 방법을 보여준다. ▶ main.py


from langchain_community.document_loaders import PythonLoader
from langchain_community.document_loaders import DirectoryLoader

directoryLoader = DirectoryLoader(".", glob = "*.py", loader_cls = PythonLoader)

documentList = directoryLoader.load()

for document in documentList:
    print(document)
    print()

"""
<class 'langchain_community.document_loaders.directory.DirectoryLoader'>
page_content='from langchain_community.document_loaders import PythonLoader\nfrom langchain_community.document_loaders import DirectoryLoader\n\ndirectoryLoader = DirectoryLoader(".", glob="*.py", loader_cls = PythonLoader)\n\nprint(type(directoryLoader))\n\ndocumentList = directoryLoader.load()\n\nfor document in documentList:\n    print(document)\n    print()' metadata={'source': 'main.py'}
"""

from langchain_community.document_loaders import PythonLoader

from langchain_community.document_loaders import DirectoryLoader

directoryLoader = DirectoryLoader(".", glob = "*.py", loader_cls = PythonLoader)

documentList = directoryLoader.load()

for document in documentList:

print(document)

print()

"""

page_content='from langchain_community.document_loaders import PythonLoader\nfrom langchain_community.document_loaders import DirectoryLoader\n\ndirectoryLoader = DirectoryLoader(".", glob="*.py", loader_cls = PythonLoader)\n\nprint(type(directoryLoader))\n\ndocumentList = directoryLoader.load()\n\nfor document in documentList:\n print(document)\n print()' metadata={'source': 'main.py'}

"""

▶ requirements.txt


aiohttp==3.9.5
aiosignal==1.3.1
annotated-types==0.7.0
async-timeout==4.0.3
attrs==23.2.0
certifi==2024.6.2
charset-normalizer==3.3.2
dataclasses-json==0.6.7
frozenlist==1.4.1
greenlet==3.0.3
idna==3.7
jsonpatch==1.33
jsonpointer==3.0.0
langchain==0.2.6
langchain-community==0.2.6
langchain-core==0.2.10
langchain-text-splitters==0.2.2
langsmith==0.1.82
marshmallow==3.21.3
multidict==6.0.5
mypy-extensions==1.0.0
numpy==1.26.4
orjson==3.10.5
packaging==24.1
pydantic==2.7.4
pydantic_core==2.18.4
PyYAML==6.0.1
requests==2.32.3
SQLAlchemy==2.0.31
tenacity==8.4.2
typing-inspect==0.9.0
typing_extensions==4.12.2
urllib3==2.2.2
yarl==1.9.4

aiohttp==3.9.5

aiosignal==1.3.1

annotated-types==0.7.0

async-timeout==4.0.3

attrs==23.2.0

certifi==2024.6.2

charset-normalizer==3.3.2

dataclasses-json==0.6.7

frozenlist==1.4.1

greenlet==3.0.3

idna==3.7

jsonpatch==1.33

jsonpointer==3.0.0

langchain==0.2.6

langchain-community==0.2.6

langchain-core==0.2.10

langchain-text-splitters==0.2.2

langsmith==0.1.82

marshmallow==3.21.3

multidict==6.0.5

mypy-extensions==1.0.0

numpy==1.26.4

orjson==3.10.5

packaging==24.1

pydantic==2.7.4

pydantic_core==2.18.4

PyYAML==6.0.1

requests==2.32.3

SQLAlchemy==2.0.31

tenacity==8.4.2

typing-inspect==0.9.0

typing_extensions==4.12.2

urllib3==2.2.2

yarl==1.9.4

※ pip install langchain-community

[PYTHON/LANGCHAIN] DirectoryLoader 클래스 : 생성자에서 loader_cls 인자를 사용해 TextLoader 로더 클래스 설정하기

■ DirectoryLoader 클래스의 생성자에서 loader_cls 인자를 사용해 TextLoader 로더 클래스를 설정하는 방법을 보여준다. ▶ main.py


from langchain_community.document_loaders import TextLoader
from langchain_community.document_loaders import DirectoryLoader

directoryLoader = DirectoryLoader(".", glob = "*.md", loader_cls = TextLoader)

documentList = directoryLoader.load()

for document in documentList:
    print(document)
    print()

"""
page_content='# 제목 1단계\n## 제목 2단계  \n### 제목 3단계\n#### 제목 4단계\n##### 제목 5단계\n###### 제목 6단계' metadata={'source': 'header.md'}

page_content='1. 첫번째\n1. 두번째\n1. 세번째\n  \n+ 항목1\n    - 항목 2\n      * 항목 3\n        + 항목 4' metadata={'source': 'list.md'}
"""

from langchain_community.document_loaders import TextLoader

from langchain_community.document_loaders import DirectoryLoader

directoryLoader = DirectoryLoader(".", glob = "*.md", loader_cls = TextLoader)

documentList = directoryLoader.load()

for document in documentList:

print(document)

print()

"""

page_content='# 제목 1단계\n## 제목 2단계 \n### 제목 3단계\n#### 제목 4단계\n##### 제목 5단계\n###### 제목 6단계' metadata={'source': 'header.md'}

page_content='1. 첫번째\n1. 두번째\n1. 세번째\n \n+ 항목1\n - 항목 2\n * 항목 3\n + 항목 4' metadata={'source': 'list.md'}

"""

▶ requirements.txt


aiohttp==3.9.5
aiosignal==1.3.1
annotated-types==0.7.0
async-timeout==4.0.3
attrs==23.2.0
certifi==2024.6.2
charset-normalizer==3.3.2
dataclasses-json==0.6.7
frozenlist==1.4.1
greenlet==3.0.3
idna==3.7
jsonpatch==1.33
jsonpointer==3.0.0
langchain==0.2.6
langchain-community==0.2.6
langchain-core==0.2.10
langchain-text-splitters==0.2.2
langsmith==0.1.82
marshmallow==3.21.3
multidict==6.0.5
mypy-extensions==1.0.0
numpy==1.26.4
orjson==3.10.5
packaging==24.1
pydantic==2.7.4
pydantic_core==2.18.4
PyYAML==6.0.1
requests==2.32.3
SQLAlchemy==2.0.31
tenacity==8.4.2
typing-inspect==0.9.0
typing_extensions==4.12.2
urllib3==2.2.2
yarl==1.9.4

aiohttp==3.9.5

aiosignal==1.3.1

annotated-types==0.7.0

async-timeout==4.0.3

attrs==23.2.0

certifi==2024.6.2

charset-normalizer==3.3.2

dataclasses-json==0.6.7

frozenlist==1.4.1

greenlet==3.0.3

idna==3.7

jsonpatch==1.33

jsonpointer==3.0.0

langchain==0.2.6

langchain-community==0.2.6

langchain-core==0.2.10

langchain-text-splitters==0.2.2

langsmith==0.1.82

marshmallow==3.21.3

multidict==6.0.5

mypy-extensions==1.0.0

numpy==1.26.4

orjson==3.10.5

packaging==24.1

pydantic==2.7.4

pydantic_core==2.18.4

PyYAML==6.0.1

requests==2.32.3

SQLAlchemy==2.0.31

tenacity==8.4.2

typing-inspect==0.9.0

typing_extensions==4.12.2

urllib3==2.2.2

yarl==1.9.4

※ pip install

[PYTHON/LANGCHAIN] DirectoryLoader 클래스 : 생성자에서 use_multithreading 인자를 사용해 멀티스레딩 설정하기

■ DirectoryLoader 클래스의 생성자에서 use_multithreading 인자를 사용해 멀티스레딩을 설정하는 방법을 보여준다. ▶ main.py


from langchain_community.document_loaders import DirectoryLoader

directoryLoader = DirectoryLoader(".", glob = "*.md", use_multithreading = True)

documentList = directoryLoader.load()

for document in documentList:
    print(document)
    print()

"""
page_content='첫번째\n\n두번째\n\n세번째\n\n항목1\n\n항목 2\n항목 3\n항목 4' metadata={'source': 'list.md'}

page_content='제목 1단계\n\n제목 2단계\n\n제목 3단계\n\n제목 4단계\n\n제목 5단계\n\n제목 6단계' metadata={'source': 'header.md'}
"""

from langchain_community.document_loaders import DirectoryLoader

directoryLoader = DirectoryLoader(".", glob = "*.md", use_multithreading = True)

documentList = directoryLoader.load()

for document in documentList:

print(document)

print()

"""

page_content='첫번째\n\n두번째\n\n세번째\n\n항목1\n\n항목 2\n항목 3\n항목 4' metadata={'source': 'list.md'}

page_content='제목 1단계\n\n제목 2단계\n\n제목 3단계\n\n제목 4단계\n\n제목 5단계\n\n제목 6단계' metadata={'source': 'header.md'}

"""

▶ requirements.txt


aiohttp==3.9.5
aiosignal==1.3.1
annotated-types==0.7.0
anyio==4.4.0
async-timeout==4.0.3
attrs==23.2.0
backoff==2.2.1
beautifulsoup4==4.12.3
certifi==2024.6.2
chardet==5.2.0
charset-normalizer==3.3.2
click==8.1.7
dataclasses-json==0.6.7
deepdiff==7.0.1
emoji==2.12.1
exceptiongroup==1.2.1
filetype==1.2.0
frozenlist==1.4.1
greenlet==3.0.3
h11==0.14.0
httpcore==1.0.5
httpx==0.27.0
idna==3.7
joblib==1.4.2
jsonpatch==1.33
jsonpath-python==1.0.6
jsonpointer==3.0.0
langchain==0.2.6
langchain-community==0.2.6
langchain-core==0.2.10
langchain-text-splitters==0.2.2
langdetect==1.0.9
langsmith==0.1.82
lxml==5.2.2
Markdown==3.6
marshmallow==3.21.3
multidict==6.0.5
mypy-extensions==1.0.0
nest-asyncio==1.6.0
nltk==3.8.1
numpy==1.26.4
ordered-set==4.1.0
orjson==3.10.5
packaging==24.1
pydantic==2.7.4
pydantic_core==2.18.4
pypdf==4.2.0
python-dateutil==2.9.0.post0
python-iso639==2024.4.27
python-magic==0.4.27
PyYAML==6.0.1
rapidfuzz==3.9.3
regex==2024.5.15
requests==2.32.3
requests-toolbelt==1.0.0
six==1.16.0
sniffio==1.3.1
soupsieve==2.5
SQLAlchemy==2.0.31
tabulate==0.9.0
tenacity==8.4.2
tqdm==4.66.4
typing-inspect==0.9.0
typing_extensions==4.12.2
unstructured==0.14.8
unstructured-client==0.23.7
urllib3==2.2.2
wrapt==1.16.0
yarl==1.9.4

aiohttp==3.9.5

aiosignal==1.3.1

annotated-types==0.7.0

anyio==4.4.0

async-timeout==4.0.3

attrs==23.2.0

backoff==2.2.1

beautifulsoup4==4.12.3

certifi==2024.6.2

chardet==5.2.0

charset-normalizer==3.3.2

click==8.1.7

dataclasses-json==0.6.7

deepdiff==7.0.1

emoji==2.12.1

exceptiongroup==1.2.1

filetype==1.2.0

frozenlist==1.4.1

greenlet==3.0.3

h11==0.14.0

httpcore==1.0.5

httpx==0.27.0

idna==3.7

joblib==1.4.2

jsonpatch==1.33

jsonpath-python==1.0.6

jsonpointer==3.0.0

langchain==0.2.6

langchain-community==0.2.6

langchain-core==0.2.10

langchain-text-splitters==0.2.2

langdetect==1.0.9

langsmith==0.1.82

lxml==5.2.2

Markdown==3.6

marshmallow==3.21.3

multidict==6.0.5

mypy-extensions==1.0.0

nest-asyncio==1.6.0

nltk==3.8.1

numpy==1.26.4

ordered-set==4.1.0

orjson==3.10.5

packaging==24.1

pydantic==2.7.4

pydantic_core==2.18.4

pypdf==4.2.0

python-dateutil==2.9.0.post0

python-iso639==2024.4.27

python-magic==0.4.27

PyYAML==6.0.1

rapidfuzz==3.9.3

regex==2024.5.15

requests==2.32.3

requests-toolbelt==1.0.0

six==1.16.0

sniffio==1.3.1

soupsieve==2.5

SQLAlchemy==2.0.31

tabulate==0.9.0

tenacity==8.4.2

tqdm==4.66.4

typing-inspect==0.9.0

typing_extensions==4.12.2

unstructured==0.14.8

unstructured-client==0.23.7

urllib3==2.2.2

wrapt==1.16.0

yarl==1.9.4

※ pip install langchain-community unstructured[md]

[PYTHON/LANGCHAIN] DirectoryLoader 클래스 : 생성자에서 show_progress 인자를 사용해 문서 로드 진행 상태 표시하기

■ DirectoryLoader 클래스의 생성자에서 show_progress 인자를 사용해 문서 로드 진행 상태를 표시하는 방법을 보여준다. ▶ main.py


from langchain_community.document_loaders import DirectoryLoader

directoryLoader = DirectoryLoader(".", glob = "*.md", show_progress = True)

documentList = directoryLoader.load()

for document in documentList:
    print(document)
    print()

"""
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:01<00:00,  1.92it/s]
page_content='제목 1단계\n\n제목 2단계\n\n제목 3단계\n\n제목 4단계\n\n제목 5단계\n\n제목 6단계' metadata={'source': 'header.md'}

page_content='첫번째\n\n두번째\n\n세번째\n\n항목1\n\n항목 2\n항목 3\n항목 4' metadata={'source': 'list.md'}
"""

from langchain_community.document_loaders import DirectoryLoader

directoryLoader = DirectoryLoader(".", glob = "*.md", show_progress = True)

documentList = directoryLoader.load()

for document in documentList:

print(document)

print()

"""

100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:01<00:00, 1.92it/s]

page_content='제목 1단계\n\n제목 2단계\n\n제목 3단계\n\n제목 4단계\n\n제목 5단계\n\n제목 6단계' metadata={'source': 'header.md'}

page_content='첫번째\n\n두번째\n\n세번째\n\n항목1\n\n항목 2\n항목 3\n항목 4' metadata={'source': 'list.md'}

"""

▶ requirements.txt


aiohttp==3.9.5
aiosignal==1.3.1
annotated-types==0.7.0
anyio==4.4.0
async-timeout==4.0.3
attrs==23.2.0
backoff==2.2.1
beautifulsoup4==4.12.3
certifi==2024.6.2
chardet==5.2.0
charset-normalizer==3.3.2
click==8.1.7
dataclasses-json==0.6.7
deepdiff==7.0.1
emoji==2.12.1
exceptiongroup==1.2.1
filetype==1.2.0
frozenlist==1.4.1
greenlet==3.0.3
h11==0.14.0
httpcore==1.0.5
httpx==0.27.0
idna==3.7
joblib==1.4.2
jsonpatch==1.33
jsonpath-python==1.0.6
jsonpointer==3.0.0
langchain==0.2.6
langchain-community==0.2.6
langchain-core==0.2.10
langchain-text-splitters==0.2.2
langdetect==1.0.9
langsmith==0.1.82
lxml==5.2.2
Markdown==3.6
marshmallow==3.21.3
multidict==6.0.5
mypy-extensions==1.0.0
nest-asyncio==1.6.0
nltk==3.8.1
numpy==1.26.4
ordered-set==4.1.0
orjson==3.10.5
packaging==24.1
pydantic==2.7.4
pydantic_core==2.18.4
pypdf==4.2.0
python-dateutil==2.9.0.post0
python-iso639==2024.4.27
python-magic==0.4.27
PyYAML==6.0.1
rapidfuzz==3.9.3
regex==2024.5.15
requests==2.32.3
requests-toolbelt==1.0.0
six==1.16.0
sniffio==1.3.1
soupsieve==2.5
SQLAlchemy==2.0.31
tabulate==0.9.0
tenacity==8.4.2
tqdm==4.66.4
typing-inspect==0.9.0
typing_extensions==4.12.2
unstructured==0.14.8
unstructured-client==0.23.7
urllib3==2.2.2
wrapt==1.16.0
yarl==1.9.4

aiohttp==3.9.5

aiosignal==1.3.1

annotated-types==0.7.0

anyio==4.4.0

async-timeout==4.0.3

attrs==23.2.0

backoff==2.2.1

beautifulsoup4==4.12.3

certifi==2024.6.2

chardet==5.2.0

charset-normalizer==3.3.2

click==8.1.7

dataclasses-json==0.6.7

deepdiff==7.0.1

emoji==2.12.1

exceptiongroup==1.2.1

filetype==1.2.0

frozenlist==1.4.1

greenlet==3.0.3

h11==0.14.0

httpcore==1.0.5

httpx==0.27.0

idna==3.7

joblib==1.4.2

jsonpatch==1.33

jsonpath-python==1.0.6

jsonpointer==3.0.0

langchain==0.2.6

langchain-community==0.2.6

langchain-core==0.2.10

langchain-text-splitters==0.2.2

langdetect==1.0.9

langsmith==0.1.82

lxml==5.2.2

Markdown==3.6

marshmallow==3.21.3

multidict==6.0.5

mypy-extensions==1.0.0

nest-asyncio==1.6.0

nltk==3.8.1

numpy==1.26.4

ordered-set==4.1.0

orjson==3.10.5

packaging==24.1

pydantic==2.7.4

pydantic_core==2.18.4

pypdf==4.2.0

python-dateutil==2.9.0.post0

python-iso639==2024.4.27

python-magic==0.4.27

PyYAML==6.0.1

rapidfuzz==3.9.3

regex==2024.5.15

requests==2.32.3

requests-toolbelt==1.0.0

six==1.16.0

sniffio==1.3.1

soupsieve==2.5

SQLAlchemy==2.0.31

tabulate==0.9.0

tenacity==8.4.2

tqdm==4.66.4

typing-inspect==0.9.0

typing_extensions==4.12.2

unstructured==0.14.8

unstructured-client==0.23.7

urllib3==2.2.2

wrapt==1.16.0

yarl==1.9.4

※ pip

[PYTHON/LANGCHAIN] DirectoryLoader 클래스 : 생성자에서 glob 인자를 사용해 MD 파일 문서 로드하기

■ DirectoryLoader 클래스의 생성자에서 glob 인자를 사용해 MD 파일 문서를 로드하는 방법을 보여준다. ▶ main.py


from langchain_community.document_loaders import DirectoryLoader

directoryLoader = DirectoryLoader(".", glob = "*.md")

documentList = directoryLoader.load()

for document in documentList:
    print(document)
    print()

"""
page_content='제목 1단계\n\n제목 2단계\n\n제목 3단계\n\n제목 4단계\n\n제목 5단계\n\n제목 6단계' metadata={'source': 'header.md'}

page_content='첫번째\n\n두번째\n\n세번째\n\n항목1\n\n항목 2\n항목 3\n항목 4' metadata={'source': 'list.md'}
"""

from langchain_community.document_loaders import DirectoryLoader

directoryLoader = DirectoryLoader(".", glob = "*.md")

documentList = directoryLoader.load()

for document in documentList:

print(document)

print()

"""

page_content='제목 1단계\n\n제목 2단계\n\n제목 3단계\n\n제목 4단계\n\n제목 5단계\n\n제목 6단계' metadata={'source': 'header.md'}

page_content='첫번째\n\n두번째\n\n세번째\n\n항목1\n\n항목 2\n항목 3\n항목 4' metadata={'source': 'list.md'}

"""

▶ requirements.txt


aiohttp==3.9.5
aiosignal==1.3.1
annotated-types==0.7.0
anyio==4.4.0
async-timeout==4.0.3
attrs==23.2.0
backoff==2.2.1
beautifulsoup4==4.12.3
certifi==2024.6.2
chardet==5.2.0
charset-normalizer==3.3.2
click==8.1.7
dataclasses-json==0.6.7
deepdiff==7.0.1
emoji==2.12.1
exceptiongroup==1.2.1
filetype==1.2.0
frozenlist==1.4.1
greenlet==3.0.3
h11==0.14.0
httpcore==1.0.5
httpx==0.27.0
idna==3.7
joblib==1.4.2
jsonpatch==1.33
jsonpath-python==1.0.6
jsonpointer==3.0.0
langchain==0.2.6
langchain-community==0.2.6
langchain-core==0.2.10
langchain-text-splitters==0.2.2
langdetect==1.0.9
langsmith==0.1.82
lxml==5.2.2
Markdown==3.6
marshmallow==3.21.3
multidict==6.0.5
mypy-extensions==1.0.0
nest-asyncio==1.6.0
nltk==3.8.1
numpy==1.26.4
ordered-set==4.1.0
orjson==3.10.5
packaging==24.1
pydantic==2.7.4
pydantic_core==2.18.4
pypdf==4.2.0
python-dateutil==2.9.0.post0
python-iso639==2024.4.27
python-magic==0.4.27
PyYAML==6.0.1
rapidfuzz==3.9.3
regex==2024.5.15
requests==2.32.3
requests-toolbelt==1.0.0
six==1.16.0
sniffio==1.3.1
soupsieve==2.5
SQLAlchemy==2.0.31
tabulate==0.9.0
tenacity==8.4.2
tqdm==4.66.4
typing-inspect==0.9.0
typing_extensions==4.12.2
unstructured==0.14.8
unstructured-client==0.23.7
urllib3==2.2.2
wrapt==1.16.0
yarl==1.9.4

aiohttp==3.9.5

aiosignal==1.3.1

annotated-types==0.7.0

anyio==4.4.0

async-timeout==4.0.3

attrs==23.2.0

backoff==2.2.1

beautifulsoup4==4.12.3

certifi==2024.6.2

chardet==5.2.0

charset-normalizer==3.3.2

click==8.1.7

dataclasses-json==0.6.7

deepdiff==7.0.1

emoji==2.12.1

exceptiongroup==1.2.1

filetype==1.2.0

frozenlist==1.4.1

greenlet==3.0.3

h11==0.14.0

httpcore==1.0.5

httpx==0.27.0

idna==3.7

joblib==1.4.2

jsonpatch==1.33

jsonpath-python==1.0.6

jsonpointer==3.0.0

langchain==0.2.6

langchain-community==0.2.6

langchain-core==0.2.10

langchain-text-splitters==0.2.2

langdetect==1.0.9

langsmith==0.1.82

lxml==5.2.2

Markdown==3.6

marshmallow==3.21.3

multidict==6.0.5

mypy-extensions==1.0.0

nest-asyncio==1.6.0

nltk==3.8.1

numpy==1.26.4

ordered-set==4.1.0

orjson==3.10.5

packaging==24.1

pydantic==2.7.4

pydantic_core==2.18.4

pypdf==4.2.0

python-dateutil==2.9.0.post0

python-iso639==2024.4.27

python-magic==0.4.27

PyYAML==6.0.1

rapidfuzz==3.9.3

regex==2024.5.15

requests==2.32.3

requests-toolbelt==1.0.0

six==1.16.0

sniffio==1.3.1

soupsieve==2.5

SQLAlchemy==2.0.31

tabulate==0.9.0

tenacity==8.4.2

tqdm==4.66.4

typing-inspect==0.9.0

typing_extensions==4.12.2

unstructured==0.14.8

unstructured-client==0.23.7

urllib3==2.2.2

wrapt==1.16.0

yarl==1.9.4

※ pip install

[PYTHON/COMMON] NamedTemporaryFile 함수 : delete 인자를 False로 설정한 경우 처리하기

■ NamedTemporaryFile 함수에서 delete 인자를 False로 설정한 경우 처리하는 방법을 보여준다. ※ NamedTemporaryFile 함수에서 delete 인자를 False로 설정한 경우 with 구문을 벗어나도

[PYTHON/COMMON] _TemporaryFileWrapper 클래스 : flush 메소드를 사용해 버퍼 비우기

■ _TemporaryFileWrapper 클래스의 flush 메소드를 사용해 버퍼를 비우는 방법을 보여준다. ※ with 구문을 벗어날 때 임시 파일이 삭제되기 때문에 flush 메소드를 사용할

[PYTHON/COMMON] _TemporaryFileWrapper 클래스 : write 메소드를 사용해 임시 파일에 데이터 쓰기

■ _TemporaryFileWrapper 클래스의 write 메소드를 사용해 임시 파일에 데이터를 쓰는 방법을 보여준다. ▶ 예제 코드 (PY)


import tempfile

sourceString = """
"Team"     , "Payroll (millions)", "Wins"
"Nationals", 81.34               , 98
"Reds"     , 82.20               , 97
"Yankees"  , 197.96              , 95
"Giants"   , 117.62              , 94
""".strip()

with tempfile.NamedTemporaryFile(delete = True, mode = "w+") as temporaryFileWrapper:
    temporaryFileWrapper.write(sourceString)

import tempfile

sourceString = """

"Team" , "Payroll (millions)", "Wins"

"Nationals", 81.34 , 98

"Reds" , 82.20 , 97

"Yankees" , 197.96 , 95

"Giants" , 117.62 , 94

""".strip()

with tempfile.NamedTemporaryFile(delete = True, mode = "w+") as temporaryFileWrapper:

temporaryFileWrapper.write(sourceString)

[PYTHON/COMMON] NamedTemporaryFile 함수 : 임시 파일 생성하기

■ NamedTemporaryFile 함수를 사용해 임시 파일을 생성한다. ※ NamedTemporaryFile 함수에서 delete 인자가 True이면 with문을 벗어날 때 임시 파일이 삭제된다. ※ delete 인자를

[PYTHON/LANGCHAIN] CSVLoader 클래스 : CSV 문자열을 문서로 로드하기

■ CSVLoader 클래스에서 CSV 문자열을 문서로 로드하는 방법을 보여준다. ▶ main.py


import tempfile
import os

from langchain_community.document_loaders.csv_loader import CSVLoader

sourceString = """
"Team"     , "Payroll (millions)", "Wins"
"Nationals", 81.34               , 98
"Reds"     , 82.20               , 97
"Yankees"  , 197.96              , 95
"Giants"   , 117.62              , 94
""".strip()

with tempfile.NamedTemporaryFile(delete = False, mode = "w+") as temporaryFileWrapper:
    temporaryFileWrapper.write(sourceString)
    temporaryFileWrapper.flush()

csvLoader = CSVLoader(file_path = temporaryFileWrapper.name)

documentList = csvLoader.load()

for document in documentList:
    print(document)

os.remove(temporaryFileWrapper.file.name)

"""
page_content='Team: Nationals\n"Payroll (millions)": 81.34\n"Wins": 98' metadata={'source': '/tmp/tmpk46bgw3c', 'row': 0}
page_content='Team: Reds\n"Payroll (millions)": 82.20\n"Wins": 97' metadata={'source': '/tmp/tmpk46bgw3c', 'row': 1}
page_content='Team: Yankees\n"Payroll (millions)": 197.96\n"Wins": 95' metadata={'source': '/tmp/tmpk46bgw3c', 'row': 2}
page_content='Team: Giants\n"Payroll (millions)": 117.62\n"Wins": 94' metadata={'source': '/tmp/tmpk46bgw3c', 'row': 3}
"""

import tempfile

import os

from langchain_community.document_loaders.csv_loader import CSVLoader

sourceString = """

"Team" , "Payroll (millions)", "Wins"

"Nationals", 81.34 , 98

"Reds" , 82.20 , 97

"Yankees" , 197.96 , 95

"Giants" , 117.62 , 94

""".strip()

with tempfile.NamedTemporaryFile(delete = False, mode = "w+") as temporaryFileWrapper:

temporaryFileWrapper.write(sourceString)

temporaryFileWrapper.flush()

csvLoader = CSVLoader(file_path = temporaryFileWrapper.name)

documentList = csvLoader.load()

for document in documentList:

print(document)

os.remove(temporaryFileWrapper.file.name)

"""

page_content='Team: Nationals\n"Payroll (millions)": 81.34\n"Wins": 98' metadata={'source': '/tmp/tmpk46bgw3c', 'row': 0}

page_content='Team: Reds\n"Payroll (millions)": 82.20\n"Wins": 97' metadata={'source': '/tmp/tmpk46bgw3c', 'row': 1}

page_content='Team: Yankees\n"Payroll (millions)": 197.96\n"Wins": 95' metadata={'source': '/tmp/tmpk46bgw3c', 'row': 2}

page_content='Team: Giants\n"Payroll (millions)": 117.62\n"Wins": 94' metadata={'source': '/tmp/tmpk46bgw3c', 'row': 3}

"""

▶ requirements.txt


aiohttp==3.9.5
aiosignal==1.3.1
annotated-types==0.7.0
async-timeout==4.0.3
attrs==23.2.0
certifi==2024.6.2
charset-normalizer==3.3.2
dataclasses-json==0.6.7
frozenlist==1.4.1
greenlet==3.0.3
idna==3.7
jsonpatch==1.33
jsonpointer==3.0.0
langchain==0.2.6
langchain-community==0.2.6
langchain-core==0.2.10
langchain-text-splitters==0.2.2
langsmith==0.1.82
marshmallow==3.21.3
multidict==6.0.5
mypy-extensions==1.0.0
numpy==1.26.4
orjson==3.10.5
packaging==24.1
pydantic==2.7.4
pydantic_core==2.18.4
PyYAML==6.0.1
requests==2.32.3
SQLAlchemy==2.0.31
tenacity==8.4.2
typing-inspect==0.9.0
typing_extensions==4.12.2
urllib3==2.2.2
yarl==1.9.4

aiohttp==3.9.5

aiosignal==1.3.1

annotated-types==0.7.0

async-timeout==4.0.3

attrs==23.2.0

certifi==2024.6.2

charset-normalizer==3.3.2

dataclasses-json==0.6.7

frozenlist==1.4.1

greenlet==3.0.3

idna==3.7

jsonpatch==1.33

jsonpointer==3.0.0

langchain==0.2.6

langchain-community==0.2.6

langchain-core==0.2.10

langchain-text-splitters==0.2.2

langsmith==0.1.82

marshmallow==3.21.3

multidict==6.0.5

mypy-extensions==1.0.0

numpy==1.26.4

orjson==3.10.5

packaging==24.1

pydantic==2.7.4

pydantic_core==2.18.4

PyYAML==6.0.1

requests==2.32.3

SQLAlchemy==2.0.31

tenacity==8.4.2

typing-inspect==0.9.0

typing_extensions==4.12.2

urllib3==2.2.2

yarl==1.9.4

※ pip install langchain-community 명령을 실행했다.

[PYTHON/LANGCHAIN] CSVLoader 클래스 : 생성자에서 source_column 인자를 사용해 CSV 파일 문서 로드하기

■ CSVLoader 클래스의 생성자에서 source_column 인자를 사용해 CSV 파일 문서를 로드하는 방법을 보여준다. ▶ main.py


from langchain_community.document_loaders.csv_loader import CSVLoader

csvLoader = CSVLoader(file_path = "source.csv", source_column = "성명")

documentList = csvLoader.load()

for document in documentList:
    print(document)

"""
page_content='ID: ID001\n성명: 홍길동\n점수: 90' metadata={'source': '홍길동', 'row': 0}
page_content='ID: ID002\n성명: 김철수\n점수: 100' metadata={'source': '김철수', 'row': 1}
page_content='ID: ID003\n성명: 이영희\n점수: 80' metadata={'source': '이영희', 'row': 2}
"""

from langchain_community.document_loaders.csv_loader import CSVLoader

csvLoader = CSVLoader(file_path = "source.csv", source_column = "성명")

documentList = csvLoader.load()

for document in documentList:

print(document)

"""

page_content='ID: ID001\n성명: 홍길동\n점수: 90' metadata={'source': '홍길동', 'row': 0}

page_content='ID: ID002\n성명: 김철수\n점수: 100' metadata={'source': '김철수', 'row': 1}

page_content='ID: ID003\n성명: 이영희\n점수: 80' metadata={'source': '이영희', 'row': 2}

"""

▶ requirements.txt


aiohttp==3.9.5
aiosignal==1.3.1
annotated-types==0.7.0
async-timeout==4.0.3
attrs==23.2.0
certifi==2024.6.2
charset-normalizer==3.3.2
dataclasses-json==0.6.7
frozenlist==1.4.1
greenlet==3.0.3
idna==3.7
jsonpatch==1.33
jsonpointer==3.0.0
langchain==0.2.6
langchain-community==0.2.6
langchain-core==0.2.10
langchain-text-splitters==0.2.2
langsmith==0.1.82
marshmallow==3.21.3
multidict==6.0.5
mypy-extensions==1.0.0
numpy==1.26.4
orjson==3.10.5
packaging==24.1
pydantic==2.7.4
pydantic_core==2.18.4
PyYAML==6.0.1
requests==2.32.3
SQLAlchemy==2.0.31
tenacity==8.4.2
typing-inspect==0.9.0
typing_extensions==4.12.2
urllib3==2.2.2
yarl==1.9.4

aiohttp==3.9.5

aiosignal==1.3.1

annotated-types==0.7.0

async-timeout==4.0.3

attrs==23.2.0

certifi==2024.6.2

charset-normalizer==3.3.2

dataclasses-json==0.6.7

frozenlist==1.4.1

greenlet==3.0.3

idna==3.7

jsonpatch==1.33

jsonpointer==3.0.0

langchain==0.2.6

langchain-community==0.2.6

langchain-core==0.2.10

langchain-text-splitters==0.2.2

langsmith==0.1.82

marshmallow==3.21.3

multidict==6.0.5

mypy-extensions==1.0.0

numpy==1.26.4

orjson==3.10.5

packaging==24.1

pydantic==2.7.4

pydantic_core==2.18.4

PyYAML==6.0.1

requests==2.32.3

SQLAlchemy==2.0.31

tenacity==8.4.2

typing-inspect==0.9.0

typing_extensions==4.12.2

urllib3==2.2.2

yarl==1.9.4

※ pip install

[PYTHON/LANGCHAIN] CSVLoader 클래스 : 생성자에서 csv_args 인자를 사용해 CSV 파일 문서 로드하기

■ CSVLoader 클래스의 생성자에서 csv_args 인자를 사용해 CSV 파일 문서를 로드하는 방법을 보여준다. ▶ main.py


from langchain_community.document_loaders.csv_loader import CSVLoader

csvLoader = CSVLoader(
    file_path = "source.csv",
    csv_args  = {
        "delimiter"  : ",",
        "quotechar"  : '"',
        "fieldnames" : ["ID", "성명", "점수"] # CSV 파일에 헤더 라인이 없는 경우 설정한다.
    }
)

documentList = csvLoader.load()

for document in documentList:
    print(document)

"""
page_content='ID: ID001\n성명: 홍길동\n점수: 90' metadata={'source': './source.csv', 'row': 0}
page_content='ID: ID002\n성명: 김철수\n점수: 100' metadata={'source': './source.csv', 'row': 1}
page_content='ID: ID003\n성명: 이영희\n점수: 80' metadata={'source': './source.csv', 'row': 2}
"""

from langchain_community.document_loaders.csv_loader import CSVLoader

csvLoader = CSVLoader(

file_path = "source.csv",

csv_args = {

"delimiter" : ",",

"quotechar" : '"',

"fieldnames" : ["ID", "성명", "점수"] # CSV 파일에 헤더 라인이 없는 경우 설정한다.

}

)

documentList = csvLoader.load()

for document in documentList:

print(document)

"""

page_content='ID: ID001\n성명: 홍길동\n점수: 90' metadata={'source': './source.csv', 'row': 0}

page_content='ID: ID002\n성명: 김철수\n점수: 100' metadata={'source': './source.csv', 'row': 1}

page_content='ID: ID003\n성명: 이영희\n점수: 80' metadata={'source': './source.csv', 'row': 2}

"""

▶ requirements.txt


aiohttp==3.9.5
aiosignal==1.3.1
annotated-types==0.7.0
async-timeout==4.0.3
attrs==23.2.0
certifi==2024.6.2
charset-normalizer==3.3.2
dataclasses-json==0.6.7
frozenlist==1.4.1
greenlet==3.0.3
idna==3.7
jsonpatch==1.33
jsonpointer==3.0.0
langchain==0.2.6
langchain-community==0.2.6
langchain-core==0.2.10
langchain-text-splitters==0.2.2
langsmith==0.1.82
marshmallow==3.21.3
multidict==6.0.5
mypy-extensions==1.0.0
numpy==1.26.4
orjson==3.10.5
packaging==24.1
pydantic==2.7.4
pydantic_core==2.18.4
PyYAML==6.0.1
requests==2.32.3
SQLAlchemy==2.0.31
tenacity==8.4.2
typing-inspect==0.9.0
typing_extensions==4.12.2
urllib3==2.2.2
yarl==1.9.4

aiohttp==3.9.5

aiosignal==1.3.1

annotated-types==0.7.0

async-timeout==4.0.3

attrs==23.2.0

certifi==2024.6.2

charset-normalizer==3.3.2

dataclasses-json==0.6.7

frozenlist==1.4.1

greenlet==3.0.3

idna==3.7

jsonpatch==1.33

jsonpointer==3.0.0

langchain==0.2.6

langchain-community==0.2.6

langchain-core==0.2.10

langchain-text-splitters==0.2.2

langsmith==0.1.82

marshmallow==3.21.3

multidict==6.0.5

mypy-extensions==1.0.0

numpy==1.26.4

orjson==3.10.5

packaging==24.1

pydantic==2.7.4

pydantic_core==2.18.4

PyYAML==6.0.1

requests==2.32.3

SQLAlchemy==2.0.31

tenacity==8.4.2

typing-inspect==0.9.0

typing_extensions==4.12.2

urllib3==2.2.2

yarl==1.9.4

※ pip install

[PYTHON/LANGCHAIN] CSVLoader 클래스 : load 메소드를 사용해 CSV 파일 문서 로드하기

■ CSVLoader 클래스의 load 메소드를 사용해 CSV 파일 문서를 로드하는 방법을 보여준다. ▶ main.py


from langchain_community.document_loaders.csv_loader import CSVLoader

csvLoader = CSVLoader(file_path = "source.csv")

documentList = csvLoader.load()

for document in documentList:
    print(document)

"""
page_content='ID: ID001\n성명: 홍길동\n점수: 90' metadata={'source': './source.csv', 'row': 0}
page_content='ID: ID002\n성명: 김철수\n점수: 100' metadata={'source': './source.csv', 'row': 1}
page_content='ID: ID003\n성명: 이영희\n점수: 80' metadata={'source': './source.csv', 'row': 2}
"""

from langchain_community.document_loaders.csv_loader import CSVLoader

csvLoader = CSVLoader(file_path = "source.csv")

documentList = csvLoader.load()

for document in documentList:

print(document)

"""

page_content='ID: ID001\n성명: 홍길동\n점수: 90' metadata={'source': './source.csv', 'row': 0}

page_content='ID: ID002\n성명: 김철수\n점수: 100' metadata={'source': './source.csv', 'row': 1}

page_content='ID: ID003\n성명: 이영희\n점수: 80' metadata={'source': './source.csv', 'row': 2}

"""

▶ requirements.txt


aiohttp==3.9.5
aiosignal==1.3.1
annotated-types==0.7.0
async-timeout==4.0.3
attrs==23.2.0
certifi==2024.6.2
charset-normalizer==3.3.2
dataclasses-json==0.6.7
frozenlist==1.4.1
greenlet==3.0.3
idna==3.7
jsonpatch==1.33
jsonpointer==3.0.0
langchain==0.2.6
langchain-community==0.2.6
langchain-core==0.2.10
langchain-text-splitters==0.2.2
langsmith==0.1.82
marshmallow==3.21.3
multidict==6.0.5
mypy-extensions==1.0.0
numpy==1.26.4
orjson==3.10.5
packaging==24.1
pydantic==2.7.4
pydantic_core==2.18.4
PyYAML==6.0.1
requests==2.32.3
SQLAlchemy==2.0.31
tenacity==8.4.2
typing-inspect==0.9.0
typing_extensions==4.12.2
urllib3==2.2.2
yarl==1.9.4

aiohttp==3.9.5

aiosignal==1.3.1

annotated-types==0.7.0

async-timeout==4.0.3

attrs==23.2.0

certifi==2024.6.2

charset-normalizer==3.3.2

dataclasses-json==0.6.7

frozenlist==1.4.1

greenlet==3.0.3

idna==3.7

jsonpatch==1.33

jsonpointer==3.0.0

langchain==0.2.6

langchain-community==0.2.6

langchain-core==0.2.10

langchain-text-splitters==0.2.2

langsmith==0.1.82

marshmallow==3.21.3

multidict==6.0.5

mypy-extensions==1.0.0

numpy==1.26.4

orjson==3.10.5

packaging==24.1

pydantic==2.7.4

pydantic_core==2.18.4

PyYAML==6.0.1

requests==2.32.3

SQLAlchemy==2.0.31

tenacity==8.4.2

typing-inspect==0.9.0

typing_extensions==4.12.2

urllib3==2.2.2

yarl==1.9.4

※ pip install langchain-community

[PYTHON/COMMON] BufferedReader 클래스 : tell 메소드를 사용해 파일의 현재 위치 구하기

■ BufferedReader 클래스의 tell 메소드를 사용해 파일의 현재 위치를 구하는 방법을 보여준다. ▶ 예제 코드 (PY)


with open("example.bin", "rb") as bufferedReader:
    bufferedReader.seek(10)
    position = bufferedReader.tell()
    print(position)

with open("example.bin", "rb") as bufferedReader:

bufferedReader.seek(10)

position = bufferedReader.tell()

print(position)

[PYTHON/COMMON] BufferedReader 클래스 : seek 메소드를 사용해 파일 포인터 이동시키기

■ BufferedReader 클래스의 seek 메소드를 사용해 파일 포인터를 이동시키는 방법을 보여준다. • seek 메소드의 시그니처는 다음과 같다 : seek(offset, whence) • whence

[PYTHON/COMMON] BufferedReader 클래스 : read 메소드를 사용해 이진 파일 읽기

■ BufferedReader 클래스의 read 메소드를 사용해 이진 파일을 읽는 방법을 보여준다. ▶ 예제 코드 (PY)


with open("example.bin", "rb") as bufferedReader:
    readBytes = bufferedReader.read(3)
    print(readBytes)

"""
b'\x01\x02\x03
"""

with open("example.bin", "rb") as bufferedReader:

readBytes = bufferedReader.read(3)

print(readBytes)

"""

b'\x01\x02\x03

"""

example.zip

[PYTHON/COMMON] open 함수 : 이진 파일을 읽기 모드로 열기

■ open 함수를 사용해 이진 파일을 읽기 모드로 여는 방법을 보여준다. ▶ 예제 코드 (PY)


bufferedReader = open("example.bin", "rb")

bufferedReader = open("example.bin", "rb")

example.zip

[PYTHON/COMMON] BufferedWriter 클래스 : write 메소드를 사용해 이진 파일 쓰기

■ BufferedWriter 클래스의 write 메소드를 사용해 이진 파일을 쓰는 방법을 보여준다. ▶ 예제 코드 (PY)


with open("example.bin", "wb") as bufferedWriter:
    bufferedWriter.write(b"\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f")

with open("example.bin", "wb") as bufferedWriter:

bufferedWriter.write(b"\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f")

example.zip

[PYTHON/COMMON] open 함수 : 이진 파일을 쓰기 모드로 열기

■ open 함수를 사용해 이진 파일을 쓰기 모드로 여는 방법을 보여준다. ▶ 예제 코드 (PY)


bufferedWriter = open("example.bin", "wb")

bufferedWriter = open("example.bin", "wb")

[OS/UBUNTU] chmod 명령 : 파일에 실행 권한 설정하기

■ chmod 명령을 사용해 파일에 실행 권한을 설정하는 방법을 보여준다. 1. CTRL + ALT + T 키를 눌러서 [터미널]을 실행한다. 2. [터미널]에서