AIを利用した検索エンジン「Txtai」ライブラリをインストールする

AIを利用した検索エンジン「Txtai」ライブラリのインストールについて解説しています。

Txtaiは、テキストセクション上のインデクシング(索引)に基づいて構築された、AIを搭載した検索エンジンです。テキストのセクションと、検索バーに入力されたクエリとの間で類似性検索を行うことができます。対話式のマシンを構築するのにも使用できます。

■Python

今回のPythonのバージョンは、「3.6.8」を使用しています。(Windows10)(pythonランチャーでの確認)

■Txtaiをインストールする

Txtaiをインストールを行いますが、今回はpipを経由してインストールを行うので、まずWindowsのコマンドプロンプトを起動します。

pip install txtai

起動後、上記のコマンドを入力し、Enterキーを押します。

なお、今回は、pythonランチャーを使用しており、Python Version 3.6.8にインストールを行うために、pipを使う場合にはコマンドでの切り替えを行います。

py -3.6 -m pip install txtai

切り替えるために、上記のコマンドを入力し、Enterキーを押します。

Defaulting to user installation because normal site-packages is not writeable
Collecting txtai
  Using cached txtai-3.0.0-py3-none-any.whl (57 kB)
Collecting regex>=2020.5.14
  Downloading regex-2021.4.4-cp36-cp36m-win_amd64.whl (269 kB)
     |████████████████████████████████| 269 kB 1.1 MB/s
Collecting tqdm>=4.46.0
  Downloading tqdm-4.60.0-py2.py3-none-any.whl (75 kB)
     |████████████████████████████████| 75 kB 873 kB/s
Collecting annoy>=1.16.3
  Downloading annoy-1.17.0.tar.gz (646 kB)
     |████████████████████████████████| 646 kB 2.2 MB/s
Collecting soundfile>=0.10.3.post1
  Using cached SoundFile-0.10.3.post1-py2.py3.cp26.cp27.cp32.cp33.cp34.cp35.cp36.pp27.pp32.pp33-none-win_amd64.whl (689 kB)
Collecting sentencepiece>=0.1.91
  Using cached sentencepiece-0.1.95-cp36-cp36m-win_amd64.whl (1.2 MB)
Collecting requests>=2.24.0
  Downloading requests-2.25.1-py2.py3-none-any.whl (61 kB)
     |████████████████████████████████| 61 kB 1.9 MB/s
Collecting transformers>=4.4.2
  Downloading transformers-4.6.0-py3-none-any.whl (2.3 MB)
     |████████████████████████████████| 2.3 MB 2.2 MB/s
Collecting sentence-transformers>=1.0.3
  Downloading sentence-transformers-1.1.1.tar.gz (81 kB)
     |████████████████████████████████| 81 kB 2.6 MB/s
Collecting uvicorn>=0.12.1
  Downloading uvicorn-0.13.4-py3-none-any.whl (46 kB)
     |████████████████████████████████| 46 kB 464 kB/s
Collecting scikit-learn>=0.23.1
  Downloading scikit_learn-0.24.2-cp36-cp36m-win_amd64.whl (6.8 MB)
     |████████████████████████████████| 6.8 MB 3.3 MB/s
Collecting nltk>=3.5
  Downloading nltk-3.6.2-py3-none-any.whl (1.5 MB)
     |████████████████████████████████| 1.5 MB 6.8 MB/s
Collecting pymagnitude-lite>=0.1.43
  Downloading pymagnitude_lite-0.1.143-py3-none-any.whl (34 kB)
Collecting fastapi>=0.61.1
  Downloading fastapi-0.65.1-py3-none-any.whl (50 kB)
     |████████████████████████████████| 50 kB 1.6 MB/s
Collecting fasttext>=0.9.2
  Using cached fasttext-0.9.2.tar.gz (68 kB)
Collecting tika>=1.24
  Downloading tika-1.24.tar.gz (28 kB)
Collecting hnswlib>=0.5.0
  Downloading hnswlib-0.5.1.tar.gz (29 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting torch>=1.4.0
  Downloading torch-1.8.1-cp36-cp36m-win_amd64.whl (190.5 MB)
     |████████████████████████████████| 190.5 MB 6.8 MB/s
Requirement already satisfied: pillow>=7.2.0 in c:\users\user\appdata\roaming\python\python36\site-packages (from txtai) (8.2.0)
Requirement already satisfied: numpy>=1.18.4 in c:\users\user\appdata\roaming\python\python36\site-packages (from txtai) (1.19.5)
Collecting PyYAML>=5.3
  Downloading PyYAML-5.4.1-cp36-cp36m-win_amd64.whl (209 kB)
     |████████████████████████████████| 209 kB 3.3 MB/s
Collecting apache-libcloud>=3.3.1
  Using cached apache_libcloud-3.3.1-py2.py3-none-any.whl (3.7 MB)
Collecting starlette==0.14.2
  Downloading starlette-0.14.2-py3-none-any.whl (60 kB)
     |████████████████████████████████| 60 kB 1.8 MB/s
Collecting pydantic!=1.7,!=1.7.1,!=1.7.2,!=1.7.3,!=1.8,!=1.8.1,<2.0.0,>=1.6.2
  Downloading pydantic-1.8.2-cp36-cp36m-win_amd64.whl (1.9 MB)
     |████████████████████████████████| 1.9 MB 6.4 MB/s
Collecting pybind11>=2.2
  Using cached pybind11-2.6.2-py2.py3-none-any.whl (191 kB)
Requirement already satisfied: setuptools>=0.7.0 in c:\users\user\appdata\roaming\python\python36\site-packages (from fasttext>=0.9.2->txtai) (56.2.0)
Collecting click
  Downloading click-8.0.0-py3-none-any.whl (96 kB)
     |████████████████████████████████| 96 kB 1.3 MB/s
Collecting joblib
  Downloading joblib-1.0.1-py3-none-any.whl (303 kB)
     |████████████████████████████████| 303 kB 3.3 MB/s
Requirement already satisfied: typing-extensions>=3.7.4.3 in c:\users\user\appdata\roaming\python\python36\site-packages (from pydantic!=1.7,!=1.7.1,!=1.7.2,!=1.7.3,!=1.8,!=1.8.1,<2.0.0,>=1.6.2->fastapi>=0.61.1->txtai) (3.7.4.3)
Collecting dataclasses>=0.6
  Downloading dataclasses-0.8-py3-none-any.whl (19 kB)
Collecting fasteners>=0.14.1
  Downloading fasteners-0.16-py2.py3-none-any.whl (28 kB)
Collecting lz4>=1.0.0
  Downloading lz4-3.1.3-cp36-cp36m-win_amd64.whl (192 kB)
     |████████████████████████████████| 192 kB 1.7 MB/s
Collecting xxhash>=1.0.1
  Downloading xxhash-2.0.2-cp36-cp36m-win_amd64.whl (35 kB)
Requirement already satisfied: six in c:\users\user\appdata\roaming\python\python36\site-packages (from fasteners>=0.14.1->pymagnitude-lite>=0.1.43->txtai) (1.15.0)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\user\appdata\roaming\python\python36\site-packages (from requests>=2.24.0->txtai) (2020.12.5)
Requirement already satisfied: idna<3,>=2.5 in c:\users\user\appdata\roaming\python\python36\site-packages (from requests>=2.24.0->txtai) (2.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\users\user\appdata\roaming\python\python36\site-packages (from requests>=2.24.0->txtai) (1.24.3)
Requirement already satisfied: chardet<5,>=3.0.2 in c:\users\user\appdata\roaming\python\python36\site-packages (from requests>=2.24.0->txtai) (3.0.4)
Collecting threadpoolctl>=2.0.0
  Downloading threadpoolctl-2.1.0-py3-none-any.whl (12 kB)
Collecting scipy>=0.19.1
  Downloading scipy-1.5.4-cp36-cp36m-win_amd64.whl (31.2 MB)
     |████████████████████████████████| 31.2 MB 3.2 MB/s
Collecting torchvision
  Downloading torchvision-0.9.1-cp36-cp36m-win_amd64.whl (852 kB)
     |████████████████████████████████| 852 kB 3.3 MB/s
Collecting cffi>=1.0
  Downloading cffi-1.14.5-cp36-cp36m-win_amd64.whl (178 kB)
     |████████████████████████████████| 178 kB 3.3 MB/s
Collecting pycparser
  Using cached pycparser-2.20-py2.py3-none-any.whl (112 kB)
Requirement already satisfied: importlib-metadata in c:\users\user\appdata\roaming\python\python36\site-packages (from transformers>=4.4.2->txtai) (4.0.1)
Collecting tokenizers<0.11,>=0.10.1
  Downloading tokenizers-0.10.2-cp36-cp36m-win_amd64.whl (2.0 MB)
     |████████████████████████████████| 2.0 MB 3.3 MB/s
Collecting huggingface-hub==0.0.8
  Downloading huggingface_hub-0.0.8-py3-none-any.whl (34 kB)
Collecting filelock
  Downloading filelock-3.0.12-py3-none-any.whl (7.6 kB)
Collecting sacremoses
  Downloading sacremoses-0.0.45-py3-none-any.whl (895 kB)
     |████████████████████████████████| 895 kB 2.2 MB/s
Collecting packaging
  Downloading packaging-20.9-py2.py3-none-any.whl (40 kB)
     |████████████████████████████████| 40 kB 644 kB/s
Collecting h11>=0.8
  Using cached h11-0.12.0-py3-none-any.whl (54 kB)
Collecting click
  Using cached click-7.1.2-py2.py3-none-any.whl (82 kB)
Requirement already satisfied: zipp>=0.5 in c:\users\user\appdata\roaming\python\python36\site-packages (from importlib-metadata->transformers>=4.4.2->txtai) (3.4.1)
Collecting pyparsing>=2.0.2
  Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Building wheels for collected packages: annoy, fasttext, hnswlib, sentence-transformers, tika
  Building wheel for annoy (setup.py) ... done
  Created wheel for annoy: filename=annoy-1.17.0-cp36-cp36m-win_amd64.whl size=52696 sha256=06b85f350bfe2a833d4ef2b2be8dbad94e82cd316902115df1e6974138128f10
  Stored in directory: c:\users\user\appdata\local\pip\cache\wheels\1d\90\43\49cf1e7f7aaaebab491d3447e5d4063fdf7407173a3455f8c2
  Building wheel for fasttext (setup.py) ... done
  Created wheel for fasttext: filename=fasttext-0.9.2-cp36-cp36m-win_amd64.whl size=235374 sha256=e9e3ce02fb9242664a86bfeeb64a3fe903014d34ae858232cddc988f6d0a60a7
  Stored in directory: c:\users\user\appdata\local\pip\cache\wheels\c3\5c\d0\4a725c6ee7df3267d818d3bc9d89bb173b94832f2b9eca6368
  Building wheel for hnswlib (PEP 517) ... done
  Created wheel for hnswlib: filename=hnswlib-0.5.1-cp36-cp36m-win_amd64.whl size=121606 sha256=3b43d8cf849a3f78fa148168226c504973aabb2e1c533746499f8de009205644
  Stored in directory: c:\users\user\appdata\local\pip\cache\wheels\a4\4c\2a\9b6a94e03a1827a2b54493fd5cf4c3c4c351d9e4ed3a132ba4
  Building wheel for sentence-transformers (setup.py) ... done
  Created wheel for sentence-transformers: filename=sentence_transformers-1.1.1-py3-none-any.whl size=123336 sha256=c8f0f9aa560b8d3259256077f7033f13c53ed8f7249751e611f6e4b61a929a10
  Stored in directory: c:\users\user\appdata\local\pip\cache\wheels\9e\62\5f\757335634733a2087b8217daa078dd25c53218403a5e4bf4a4
  Building wheel for tika (setup.py) ... done
  Created wheel for tika: filename=tika-1.24-py3-none-any.whl size=32884 sha256=ebc6e91b7e7c54e7f2267c62731644fcc73efc8c020ac5dd5c8c7b8cc1c346ed
  Stored in directory: c:\users\user\appdata\local\pip\cache\wheels\69\87\43\94e20227488089d3444bef415b094ec04b33b4f82e334cc8bd
Successfully built annoy fasttext hnswlib sentence-transformers tika
Installing collected packages: tqdm, requests, regex, pyparsing, joblib, filelock, dataclasses, click, torch, tokenizers, threadpoolctl, scipy, sacremoses, pycparser, packaging, huggingface-hub, xxhash, transformers, torchvision, starlette, sentencepiece, scikit-learn, pydantic, pybind11, nltk, lz4, h11, fasteners, cffi, uvicorn, tika, soundfile, sentence-transformers, PyYAML, pymagnitude-lite, hnswlib, fasttext, fastapi, apache-libcloud, annoy, txtai
  Attempting uninstall: tqdm
    Found existing installation: tqdm 4.31.1
    Uninstalling tqdm-4.31.1:
      Successfully uninstalled tqdm-4.31.1
  WARNING: The script tqdm.exe is installed in 'C:\Users\user\AppData\Roaming\Python\Python36\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  Attempting uninstall: requests
    Found existing installation: requests 2.21.0
    Uninstalling requests-2.21.0:
      Successfully uninstalled requests-2.21.0
  Attempting uninstall: regex
    Found existing installation: regex 2017.4.5
    Uninstalling regex-2017.4.5:
      Successfully uninstalled regex-2017.4.5
  WARNING: The scripts convert-caffe2-to-onnx.exe and convert-onnx-to-caffe2.exe are installed in 'C:\Users\user\AppData\Roaming\Python\Python36\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script sacremoses.exe is installed in 'C:\Users\user\AppData\Roaming\Python\Python36\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script huggingface-cli.exe is installed in 'C:\Users\user\AppData\Roaming\Python\Python36\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script transformers-cli.exe is installed in 'C:\Users\user\AppData\Roaming\Python\Python36\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script pybind11-config.exe is installed in 'C:\Users\user\AppData\Roaming\Python\Python36\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script nltk.exe is installed in 'C:\Users\user\AppData\Roaming\Python\Python36\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script uvicorn.exe is installed in 'C:\Users\user\AppData\Roaming\Python\Python36\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script tika-python.exe is installed in 'C:\Users\user\AppData\Roaming\Python\Python36\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed PyYAML-5.4.1 annoy-1.17.0 apache-libcloud-3.3.1 cffi-1.14.5 click-7.1.2 dataclasses-0.8 fastapi-0.65.1 fasteners-0.16 fasttext-0.9.2 filelock-3.0.12 h11-0.12.0 hnswlib-0.5.1 huggingface-hub-0.0.8 joblib-1.0.1 lz4-3.1.3 nltk-3.6.2 packaging-20.9 pybind11-2.6.2 pycparser-2.20 pydantic-1.8.2 pymagnitude-lite-0.1.143 pyparsing-2.4.7 regex-2021.4.4 requests-2.25.1 sacremoses-0.0.45 scikit-learn-0.24.2 scipy-1.5.4 sentence-transformers-1.1.1 sentencepiece-0.1.95 soundfile-0.10.3.post1 starlette-0.14.2 threadpoolctl-2.1.0 tika-1.24 tokenizers-0.10.2 torch-1.8.1 torchvision-0.9.1 tqdm-4.60.0 transformers-4.6.0 txtai-3.0.0 uvicorn-0.13.4 xxhash-2.0.2

Enterキーを押すと、インストールが開始され、「Successfully installed」と表示されました。エラーは表示されなかったので、正常にインストールが完了する。WARNING(警告)が表示されているが、エラーではない。

Command "python setup.py egg_info" failed with error code 1 in C:\Users\user\AppData\Local\Temp\pip-install-omub1cv9\fasttext\

なお、上記のエラーが発生する場合があります。

pip install --upgrade pip setuptools
py -3.6 -m pip install --upgrade pip setuptools --user

このようなエラーが発生する場合は上記のコマンドを入力し、Enterキーを押すと、問題が解決できます。

コメント

タイトルとURLをコピーしました