【Python】大きな表形式のデータセットを視覚化して探索する「vaex」のインストール

大きな表形式のデータセットを視覚化して探索する「vaex」のインストールについて解説しています。

「vaex（https://github.com/vaexio/vaex）」は、大規模な表形式データを可視化し、探索するための遅延 Out-of-Core DataFrameのための高性能なPythonライブラリです。なお、「Out-of-Core 」は、ディスクからチャンク（ひとまとまりのデータの塊）を読み込むことで、利用可能なRAMよりも大きなデータを操作できるようにする一連の手法である。

■Python

今回のPythonのバージョンは、「3.8.5」を使用しています。（Windows10）（pythonランチャーでの確認）

■vaexをインストールする

vaexをインストールを行いますが、今回はpipを経由してインストールを行うので、まずWindowsのコマンドプロンプトを起動します。

pip install vaex

起動後、上記のコマンドを入力し、Enterキーを押します。

なお、今回は、pythonランチャーを使用しており、Python Version 3.8.5にインストールを行うために、バージョンの切り替えを行います。

py -3.8 -m pip install vaex

切り替えるために、上記のコマンドを入力し、Enterキーを押します。

Defaulting to user installation because normal site-packages is not writeable
Collecting vaex
Downloading vaex-4.16.0-py3-none-any.whl (4.7 kB)
Collecting vaex-core<4.17,>=4.16.0
Downloading vaex_core-4.16.1-cp38-cp38-win_amd64.whl (2.0 MB)
---------------------------------------- 2.0/2.0 MB 2.5 MB/s eta 0:00:00
Collecting vaex-astro<0.10,>=0.9.3
Downloading vaex_astro-0.9.3-py3-none-any.whl (20 kB)
Collecting vaex-viz<0.6,>=0.5.4
Downloading vaex_viz-0.5.4-py3-none-any.whl (19 kB)
Collecting vaex-jupyter<0.9,>=0.8.1
Downloading vaex_jupyter-0.8.1-py3-none-any.whl (43 kB)
---------------------------------------- 43.2/43.2 kB 2.2 MB/s eta 0:00:00
Collecting vaex-ml<0.19,>=0.18.1
Downloading vaex_ml-0.18.1-py3-none-any.whl (58 kB)
---------------------------------------- 58.2/58.2 kB 3.0 MB/s eta 0:00:00
Collecting vaex-hdf5<0.15,>=0.13.0
Downloading vaex_hdf5-0.14.1-py3-none-any.whl (16 kB)
Collecting vaex-server<0.9,>=0.8.1
Using cached vaex_server-0.8.1-py3-none-any.whl (23 kB)
Collecting astropy
Downloading astropy-5.2.1-cp38-cp38-win_amd64.whl (6.6 MB)
---------------------------------------- 6.6/6.6 MB 3.9 MB/s eta 0:00:00
Collecting frozendict!=2.2.0
Downloading frozendict-2.3.5-cp38-cp38-win_amd64.whl (35 kB)
Collecting tabulate>=0.8.3
Downloading tabulate-0.9.0-py3-none-any.whl (35 kB)
Collecting pyarrow>=5.0.0
Using cached pyarrow-11.0.0-cp38-cp38-win_amd64.whl (20.6 MB)
Collecting pandas
Using cached pandas-1.5.3-cp38-cp38-win_amd64.whl (11.0 MB)
Collecting nest-asyncio>=1.3.3
Using cached nest_asyncio-1.5.6-py3-none-any.whl (5.2 kB)
Collecting aplus
Using cached aplus-0.11.0-py3-none-any.whl
Collecting progressbar2
Using cached progressbar2-4.2.0-py2.py3-none-any.whl (27 kB)
Collecting rich
Using cached rich-13.3.1-py3-none-any.whl (239 kB)
Collecting future>=0.15.2
Downloading future-0.18.3.tar.gz (840 kB)
---------------------------------------- 840.9/840.9 kB 4.1 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
Requirement already satisfied: requests in c:\users\user_\appdata\roaming\python\python38\site-packages (from vaex-core<4.17,>=4.16.0->vaex) (2.28.2)
Collecting blake3
Downloading blake3-0.3.3-cp38-none-win_amd64.whl (207 kB)
---------------------------------------- 207.7/207.7 kB 6.4 MB/s eta 0:00:00
Collecting six
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting pydantic>=1.8.0
Downloading pydantic-1.10.5-cp38-cp38-win_amd64.whl (2.2 MB)
---------------------------------------- 2.2/2.2 MB 5.4 MB/s eta 0:00:00
Collecting cloudpickle
Using cached cloudpickle-2.2.1-py3-none-any.whl (25 kB)
Collecting pyyaml
Using cached PyYAML-6.0-cp38-cp38-win_amd64.whl (155 kB)
Collecting dask!=2022.4.0
Downloading dask-2023.3.0-py3-none-any.whl (1.2 MB)
---------------------------------------- 1.2/1.2 MB 6.7 MB/s eta 0:00:00
Collecting filelock
Using cached filelock-3.9.0-py3-none-any.whl (9.7 kB)
Collecting numpy>=1.16
Using cached numpy-1.24.2-cp38-cp38-win_amd64.whl (14.9 MB)
Collecting h5py>=2.9
Downloading h5py-3.8.0-cp38-cp38-win_amd64.whl (2.7 MB)
---------------------------------------- 2.7/2.7 MB 5.8 MB/s eta 0:00:00
Collecting bqplot>=0.10.1
Using cached bqplot-0.12.36-py2.py3-none-any.whl (1.2 MB)
Collecting ipyvolume>=0.4
Downloading ipyvolume-0.6.1-py3-none-any.whl (1.6 MB)
---------------------------------------- 1.6/1.6 MB 3.5 MB/s eta 0:00:00
Collecting xarray
Downloading xarray-2023.1.0-py3-none-any.whl (973 kB)
---------------------------------------- 973.1/973.1 kB 6.2 MB/s eta 0:00:00
Collecting ipyleaflet
Downloading ipyleaflet-0.17.2-py3-none-any.whl (3.7 MB)
---------------------------------------- 3.7/3.7 MB 4.9 MB/s eta 0:00:00
Collecting ipyvuetify<2,>=1.2.2
Using cached ipyvuetify-1.8.4-py2.py3-none-any.whl (11.7 MB)
Collecting ipympl
Downloading ipympl-0.9.3-py2.py3-none-any.whl (511 kB)
---------------------------------------- 511.6/511.6 kB 6.4 MB/s eta 0:00:00
Collecting traitlets
Downloading traitlets-5.9.0-py3-none-any.whl (117 kB)
---------------------------------------- 117.4/117.4 kB 6.7 MB/s eta 0:00:00
Collecting jinja2
Using cached Jinja2-3.1.2-py3-none-any.whl (133 kB)
Collecting numba
Downloading numba-0.56.4-cp38-cp38-win_amd64.whl (2.5 MB)
---------------------------------------- 2.5/2.5 MB 5.6 MB/s eta 0:00:00
Collecting cachetools
Using cached cachetools-5.3.0-py3-none-any.whl (9.3 kB)
Collecting tornado>4.1
Using cached tornado-6.2-cp37-abi3-win_amd64.whl (425 kB)
Collecting fastapi
Downloading fastapi-0.92.0-py3-none-any.whl (56 kB)
---------------------------------------- 56.2/56.2 kB 3.1 MB/s eta 0:00:00
Collecting uvicorn[standard]
Using cached uvicorn-0.20.0-py3-none-any.whl (56 kB)
Collecting pillow
Using cached Pillow-9.4.0-cp38-cp38-win_amd64.whl (2.5 MB)
Collecting matplotlib>=1.3.1
Downloading matplotlib-3.7.0-cp38-cp38-win_amd64.whl (7.7 MB)
---------------------------------------- 7.7/7.7 MB 5.2 MB/s eta 0:00:00
Collecting ipywidgets<9,>=7.5.0
Downloading ipywidgets-8.0.4-py3-none-any.whl (137 kB)
---------------------------------------- 137.8/137.8 kB 4.1 MB/s eta 0:00:00
Collecting traittypes>=0.0.6
Using cached traittypes-0.2.1-py2.py3-none-any.whl (8.6 kB)
Collecting partd>=1.2.0
Using cached partd-1.3.0-py3-none-any.whl (18 kB)
Collecting toolz>=0.8.2
Using cached toolz-0.12.0-py3-none-any.whl (55 kB)
Requirement already satisfied: packaging>=20.0 in c:\users\user_\appdata\roaming\python\python38\site-packages (from dask!=2022.4.0->vaex-core<4.17,>=4.16.0->vaex) (23.0)
Collecting fsspec>=0.6.0
Downloading fsspec-2023.1.0-py3-none-any.whl (143 kB)
---------------------------------------- 143.0/143.0 kB 8.9 MB/s eta 0:00:00
Requirement already satisfied: click>=7.0 in c:\users\user_\appdata\roaming\python\python38\site-packages (from dask!=2022.4.0->vaex-core<4.17,>=4.16.0->vaex) (8.1.3)
Collecting ipyvue>=1.7.0
Downloading ipyvue-1.9.0-py2.py3-none-any.whl (2.7 MB)
---------------------------------------- 2.7/2.7 MB 7.1 MB/s eta 0:00:00
Collecting pythreejs>=2.4.0
Downloading pythreejs-2.4.2-py3-none-any.whl (3.4 MB)
---------------------------------------- 3.4/3.4 MB 5.7 MB/s eta 0:00:00
Collecting ipywebrtc
Using cached ipywebrtc-0.6.0-py2.py3-none-any.whl (260 kB)
Collecting cycler>=0.10
Using cached cycler-0.11.0-py3-none-any.whl (6.4 kB)
Collecting contourpy>=1.0.1
Using cached contourpy-1.0.7-cp38-cp38-win_amd64.whl (162 kB)
Collecting fonttools>=4.22.0
Using cached fonttools-4.38.0-py3-none-any.whl (965 kB)
Collecting importlib-resources>=3.2.0
Downloading importlib_resources-5.12.0-py3-none-any.whl (36 kB)
Collecting python-dateutil>=2.7
Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting pyparsing>=2.3.1
Using cached pyparsing-3.0.9-py3-none-any.whl (98 kB)
Collecting kiwisolver>=1.0.1
Using cached kiwisolver-1.4.4-cp38-cp38-win_amd64.whl (55 kB)
Collecting pytz>=2020.1
Using cached pytz-2022.7.1-py2.py3-none-any.whl (499 kB)
Collecting typing-extensions>=4.2.0
Using cached typing_extensions-4.5.0-py3-none-any.whl (27 kB)
Collecting pyerfa>=2.0
Using cached pyerfa-2.0.0.1-cp38-cp38-win_amd64.whl (366 kB)
Collecting starlette<0.26.0,>=0.25.0
Downloading starlette-0.25.0-py3-none-any.whl (66 kB)
---------------------------------------- 66.4/66.4 kB 3.5 MB/s eta 0:00:00
Collecting xyzservices>=2021.8.1
Downloading xyzservices-2023.2.0-py3-none-any.whl (55 kB)
---------------------------------------- 55.4/55.4 kB ? eta 0:00:00
Collecting branca>=0.5.0
Downloading branca-0.6.0-py3-none-any.whl (24 kB)
Collecting ipython-genutils
Using cached ipython_genutils-0.2.0-py2.py3-none-any.whl (26 kB)
Collecting ipython<9
Downloading ipython-8.11.0-py3-none-any.whl (793 kB)
---------------------------------------- 793.3/793.3 kB 7.2 MB/s eta 0:00:00
Collecting MarkupSafe>=2.0
Using cached MarkupSafe-2.1.2-cp38-cp38-win_amd64.whl (16 kB)
Collecting numpy>=1.16
Downloading numpy-1.23.5-cp38-cp38-win_amd64.whl (14.7 MB)
---------------------------------------- 14.7/14.7 MB 5.2 MB/s eta 0:00:00
Requirement already satisfied: setuptools in c:\users\user_\appdata\roaming\python\python38\site-packages (from numba->vaex-ml<0.19,>=0.18.1->vaex) (59.8.0)
Collecting importlib-metadata
Using cached importlib_metadata-6.0.0-py3-none-any.whl (21 kB)
Collecting llvmlite<0.40,>=0.39.0dev0
Using cached llvmlite-0.39.1-cp38-cp38-win_amd64.whl (23.2 MB)
Collecting python-utils>=3.0.0
Downloading python_utils-3.5.2-py2.py3-none-any.whl (24 kB)
Requirement already satisfied: charset-normalizer<4,>=2 in c:\users\user_\appdata\roaming\python\python38\site-packages (from requests->vaex-core<4.17,>=4.16.0->vaex) (3.0.1)
Requirement already satisfied: idna<4,>=2.5 in c:\users\user_\appdata\roaming\python\python38\site-packages (from requests->vaex-core<4.17,>=4.16.0->vaex) (3.4)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\user_\appdata\roaming\python\python38\site-packages (from requests->vaex-core<4.17,>=4.16.0->vaex) (2022.12.7)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\users\user_\appdata\roaming\python\python38\site-packages (from requests->vaex-core<4.17,>=4.16.0->vaex) (1.26.14)
Collecting pygments<3.0.0,>=2.14.0
Using cached Pygments-2.14.0-py3-none-any.whl (1.1 MB)
Collecting markdown-it-py<3.0.0,>=2.1.0
Using cached markdown_it_py-2.2.0-py3-none-any.whl (84 kB)
Requirement already satisfied: h11>=0.8 in c:\users\user_\appdata\roaming\python\python38\site-packages (from uvicorn[standard]->vaex-server<0.9,>=0.8.1->vaex) (0.14.0)
Requirement already satisfied: python-dotenv>=0.13 in c:\users\user_\appdata\roaming\python\python38\site-packages (from uvicorn[standard]->vaex-server<0.9,>=0.8.1->vaex) (1.0.0)
Collecting websockets>=10.4
Using cached websockets-10.4-cp38-cp38-win_amd64.whl (101 kB)
Requirement already satisfied: colorama>=0.4 in c:\users\user_\appdata\roaming\python\python38\site-packages (from uvicorn[standard]->vaex-server<0.9,>=0.8.1->vaex) (0.4.6)
Collecting watchfiles>=0.13
Downloading watchfiles-0.18.1-cp37-abi3-win_amd64.whl (264 kB)
---------------------------------------- 264.0/264.0 kB 5.4 MB/s eta 0:00:00
Collecting httptools>=0.5.0
Downloading httptools-0.5.0-cp38-cp38-win_amd64.whl (144 kB)
---------------------------------------- 144.9/144.9 kB 9.0 MB/s eta 0:00:00
Collecting zipp>=3.1.0
Using cached zipp-3.15.0-py3-none-any.whl (6.8 kB)
Collecting jedi>=0.16
Using cached jedi-0.18.2-py2.py3-none-any.whl (1.6 MB)
Collecting pickleshare
Using cached pickleshare-0.7.5-py2.py3-none-any.whl (6.9 kB)
Collecting matplotlib-inline
Using cached matplotlib_inline-0.1.6-py3-none-any.whl (9.4 kB)
Collecting backcall
Using cached backcall-0.2.0-py2.py3-none-any.whl (11 kB)
Collecting decorator
Using cached decorator-5.1.1-py3-none-any.whl (9.1 kB)
Collecting stack-data
Using cached stack_data-0.6.2-py3-none-any.whl (24 kB)
Collecting prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30
Downloading prompt_toolkit-3.0.38-py3-none-any.whl (385 kB)
---------------------------------------- 385.8/385.8 kB 4.8 MB/s eta 0:00:00
Collecting widgetsnbextension~=4.0
Downloading widgetsnbextension-4.0.5-py3-none-any.whl (2.0 MB)
---------------------------------------- 2.0/2.0 MB 6.2 MB/s eta 0:00:00
Collecting ipykernel>=4.5.1
Downloading ipykernel-6.21.2-py3-none-any.whl (149 kB)
---------------------------------------- 149.7/149.7 kB ? eta 0:00:00
Collecting jupyterlab-widgets~=3.0
Downloading jupyterlab_widgets-3.0.5-py3-none-any.whl (384 kB)
---------------------------------------- 384.3/384.3 kB 8.0 MB/s eta 0:00:00
Collecting mdurl~=0.1
Using cached mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Collecting locket
Using cached locket-1.0.0-py2.py3-none-any.whl (4.4 kB)
Collecting ipydatawidgets>=1.1.1
Downloading ipydatawidgets-4.3.3-py2.py3-none-any.whl (271 kB)
---------------------------------------- 271.7/271.7 kB 5.6 MB/s eta 0:00:00
Collecting anyio<5,>=3.4.0
Using cached anyio-3.6.2-py3-none-any.whl (80 kB)
Requirement already satisfied: sniffio>=1.1 in c:\users\user_\appdata\roaming\python\python38\site-packages (from anyio<5,>=3.4.0->starlette<0.26.0,>=0.25.0->fastapi->vaex-server<0.9,>=0.8.1->vaex) (1.3.0)
Collecting jupyter-core!=5.0.*,>=4.12
Downloading jupyter_core-5.2.0-py3-none-any.whl (94 kB)
---------------------------------------- 94.3/94.3 kB 5.3 MB/s eta 0:00:00
Collecting comm>=0.1.1
Downloading comm-0.1.2-py3-none-any.whl (6.5 kB)
Collecting debugpy>=1.6.5
Downloading debugpy-1.6.6-cp38-cp38-win_amd64.whl (4.9 MB)
---------------------------------------- 4.9/4.9 MB 6.1 MB/s eta 0:00:00
Collecting jupyter-client>=6.1.12
Downloading jupyter_client-8.0.3-py3-none-any.whl (102 kB)
---------------------------------------- 102.7/102.7 kB 3.0 MB/s eta 0:00:00
Collecting pyzmq>=20
Using cached pyzmq-25.0.0-cp38-cp38-win_amd64.whl (975 kB)
Collecting psutil
Using cached psutil-5.9.4-cp36-abi3-win_amd64.whl (252 kB)
Collecting parso<0.9.0,>=0.8.0
Using cached parso-0.8.3-py2.py3-none-any.whl (100 kB)
Collecting wcwidth
Downloading wcwidth-0.2.6-py2.py3-none-any.whl (29 kB)
Collecting executing>=1.2.0
Using cached executing-1.2.0-py2.py3-none-any.whl (24 kB)
Collecting asttokens>=2.1.0
Using cached asttokens-2.2.1-py2.py3-none-any.whl (26 kB)
Collecting pure-eval
Using cached pure_eval-0.2.2-py3-none-any.whl (11 kB)
Collecting pywin32>=1.0
Using cached pywin32-305-cp38-cp38-win_amd64.whl (12.3 MB)
Collecting platformdirs>=2.5
Downloading platformdirs-3.0.0-py3-none-any.whl (14 kB)
Building wheels for collected packages: future
Building wheel for future (setup.py) ... done
Created wheel for future: filename=future-0.18.3-py3-none-any.whl size=492037 sha256=4c96d3cde4be8827ccbf836ffc572edb4aa5a164f25697a6be800b7d2978f701
Stored in directory: c:\users\user_\appdata\local\pip\cache\wheels\a0\0b\ee\e6994fadb42c1354dcccb139b0bf2795271bddfe6253ccdf11
Successfully built future
Installing collected packages: wcwidth, pywin32, pytz, pure-eval, pickleshare, ipywebrtc, ipython-genutils, executing, blake3, backcall, aplus, zipp, xyzservices, widgetsnbextension, websockets, typing-extensions, traitlets, tornado, toolz, tabulate, six, pyzmq, pyyaml, python-utils, pyparsing, pygments, psutil, prompt-toolkit, platformdirs, pillow, parso, numpy, nest-asyncio, mdurl, MarkupSafe, locket, llvmlite, kiwisolver, jupyterlab-widgets, httptools, future, fsspec, frozendict, fonttools, filelock, decorator, debugpy, cycler, cloudpickle, cachetools, anyio, watchfiles, uvicorn, traittypes, starlette, python-dateutil, pyerfa, pydantic, pyarrow, progressbar2, partd, matplotlib-inline, markdown-it-py, jupyter-core, jinja2, jedi, importlib-resources, importlib-metadata, h5py, contourpy, comm, asttokens, stack-data, rich, pandas, numba, matplotlib, jupyter-client, fastapi, dask, branca, astropy, xarray, vaex-core, ipython, vaex-viz, vaex-server, vaex-ml, vaex-hdf5, vaex-astro, ipykernel, ipywidgets, ipyvue, ipympl, ipyleaflet, ipydatawidgets, bqplot, pythreejs, ipyvuetify, ipyvolume, vaex-jupyter, vaex
Successfully installed MarkupSafe-2.1.2 anyio-3.6.2 aplus-0.11.0 astropy-5.2.1 asttokens-2.2.1 backcall-0.2.0 blake3-0.3.3 bqplot-0.12.36 branca-0.6.0 cachetools-5.3.0 cloudpickle-2.2.1 comm-0.1.2 contourpy-1.0.7 cycler-0.11.0 dask-2023.3.0 debugpy-1.6.6 decorator-5.1.1 executing-1.2.0 fastapi-0.92.0 filelock-3.9.0 fonttools-4.38.0 frozendict-2.3.5 fsspec-2023.1.0 future-0.18.3 h5py-3.8.0 httptools-0.5.0 importlib-metadata-6.0.0 importlib-resources-5.12.0 ipydatawidgets-4.3.3 ipykernel-6.21.2 ipyleaflet-0.17.2 ipympl-0.9.3 ipython-8.11.0 ipython-genutils-0.2.0 ipyvolume-0.6.1 ipyvue-1.9.0 ipyvuetify-1.8.4 ipywebrtc-0.6.0 ipywidgets-8.0.4 jedi-0.18.2 jinja2-3.1.2 jupyter-client-8.0.3 jupyter-core-5.2.0 jupyterlab-widgets-3.0.5 kiwisolver-1.4.4 llvmlite-0.39.1 locket-1.0.0 markdown-it-py-2.2.0 matplotlib-3.7.0 matplotlib-inline-0.1.6 mdurl-0.1.2 nest-asyncio-1.5.6 numba-0.56.4 numpy-1.23.5 pandas-1.5.3 parso-0.8.3 partd-1.3.0 pickleshare-0.7.5 pillow-9.4.0 platformdirs-3.0.0 progressbar2-4.2.0 prompt-toolkit-3.0.38 psutil-5.9.4 pure-eval-0.2.2 pyarrow-11.0.0 pydantic-1.10.5 pyerfa-2.0.0.1 pygments-2.14.0 pyparsing-3.0.9 python-dateutil-2.8.2 python-utils-3.5.2 pythreejs-2.4.2 pytz-2022.7.1 pywin32-305 pyyaml-6.0 pyzmq-25.0.0 rich-13.3.1 six-1.16.0 stack-data-0.6.2 starlette-0.25.0 tabulate-0.9.0 toolz-0.12.0 tornado-6.2 traitlets-5.9.0 traittypes-0.2.1 typing-extensions-4.5.0 uvicorn-0.20.0 vaex-4.16.0 vaex-astro-0.9.3 vaex-core-4.16.1 vaex-hdf5-0.14.1 vaex-jupyter-0.8.1 vaex-ml-0.18.1 vaex-server-0.8.1 vaex-viz-0.5.4 watchfiles-0.18.1 wcwidth-0.2.6 websockets-10.4 widgetsnbextension-4.0.5 xarray-2023.1.0 xyzservices-2023.2.0 zipp-3.15.0

Enterキーを押すと、インストールが開始され、上記のように「Successfully installed」と表示されます。これが表示されれば、vaexが正常にインストールされたことになります。

なお、今回はvaexのバージョン4.16.0をインストールしました。