【Python】Scrapyダウンロードハンドラー 「scrapy-playwright」のインストール

【Python】Scrapyダウンロードハンドラー 「scrapy-playwright」のインストールについて解説しています。

「scrapy-playwright（https://github.com/scrapy-plugins/scrapy-playwright）」は、playwright-pythonを使用してリクエストを実行するScrapyダウンロードハンドラー 。JavaScriptを必要とするページを処理するに使用できる。playwright-pythonは、Chromium、Firefox、WebKitブラウザを単一のAPIで自動化するためのPythonライブラリ。

■Python

今回のPythonのバージョンは、「3.8.5」を使用しています。（Windows10）（pythonランチャーでの確認）

■scrapy-playwrightをインストールする

scrapy-playwrightをインストールを行いますが、今回はpipを経由してインストールを行うので、まずWindowsのコマンドプロンプトを起動します。

pip install scrapy-playwright

起動後、上記のコマンドを入力し、Enterキーを押します。

なお、今回は、pythonランチャーを使用しており、Python Version 3.8.5にインストールを行うために、バージョンの切り替えを行います。

py -3.8 -m pip install scrapy-playwright

切り替えるために、上記のコマンドを入力し、Enterキーを押します。

Defaulting to user installation because normal site-packages is not writeable
Collecting scrapy-playwright
Downloading scrapy_playwright-0.0.17-py3-none-any.whl (16 kB)
Requirement already satisfied: playwright>=1.15 in c:\users\user_\appdata\roaming\python\python38\site-packages (from scrapy-playwright) (1.17.2)
Requirement already satisfied: scrapy!=2.4.0,>=2.0 in c:\users\user_\appdata\roaming\python\python38\site-packages (from scrapy-playwright) (2.6.1)
Requirement already satisfied: pyee>=8.0.1 in c:\users\user_\appdata\roaming\python\python38\site-packages (from playwright>=1.15->scrapy-playwright) (8.2.2)
Requirement already satisfied: greenlet>=1.0.0 in c:\users\user_\appdata\roaming\python\python38\site-packages (from playwright>=1.15->scrapy-playwright) (1.1.2)
Requirement already satisfied: typing-extensions in c:\users\user_\appdata\roaming\python\python38\site-packages (from playwright>=1.15->scrapy-playwright) (4.0.0)
Requirement already satisfied: websockets>=8.1 in c:\users\user_\appdata\roaming\python\python38\site-packages (from playwright>=1.15->scrapy-playwright) (10.2)
Requirement already satisfied: PyDispatcher>=2.0.5 in c:\users\user_\appdata\roaming\python\python38\site-packages (from scrapy!=2.4.0,>=2.0->scrapy-playwright) (2.0.5)
Requirement already satisfied: itemadapter>=0.1.0 in c:\users\user_\appdata\roaming\python\python38\site-packages (from scrapy!=2.4.0,>=2.0->scrapy-playwright) (0.4.0)
Requirement already satisfied: service-identity>=16.0.0 in c:\users\user_\appdata\roaming\python\python38\site-packages (from scrapy!=2.4.0,>=2.0->scrapy-playwright) (21.1.0)
Requirement already satisfied: Twisted>=17.9.0 in c:\users\user_\appdata\roaming\python\python38\site-packages (from scrapy!=2.4.0,>=2.0->scrapy-playwright) (22.2.0)
Requirement already satisfied: protego>=0.1.15 in c:\users\user_\appdata\roaming\python\python38\site-packages (from scrapy!=2.4.0,>=2.0->scrapy-playwright) (0.2.1)
Requirement already satisfied: zope.interface>=4.1.3 in c:\users\user_\appdata\roaming\python\python38\site-packages (from scrapy!=2.4.0,>=2.0->scrapy-playwright) (5.4.0)
Requirement already satisfied: parsel>=1.5.0 in c:\users\user_\appdata\roaming\python\python38\site-packages (from scrapy!=2.4.0,>=2.0->scrapy-playwright) (1.6.0)
Requirement already satisfied: cryptography>=2.0 in c:\users\user_\appdata\roaming\python\python38\site-packages (from scrapy!=2.4.0,>=2.0->scrapy-playwright) (3.3.2)
Requirement already satisfied: tldextract in c:\users\user_\appdata\roaming\python\python38\site-packages (from scrapy!=2.4.0,>=2.0->scrapy-playwright) (3.1.2)
Requirement already satisfied: lxml>=3.5.0 in c:\users\user_\appdata\roaming\python\python38\site-packages (from scrapy!=2.4.0,>=2.0->scrapy-playwright) (4.8.0)
Requirement already satisfied: queuelib>=1.4.2 in c:\users\user_\appdata\roaming\python\python38\site-packages (from scrapy!=2.4.0,>=2.0->scrapy-playwright) (1.6.2)
Requirement already satisfied: itemloaders>=1.0.1 in c:\users\user_\appdata\roaming\python\python38\site-packages (from scrapy!=2.4.0,>=2.0->scrapy-playwright) (1.0.4)
Requirement already satisfied: cssselect>=0.9.1 in c:\users\user_\appdata\roaming\python\python38\site-packages (from scrapy!=2.4.0,>=2.0->scrapy-playwright) (1.1.0)
Requirement already satisfied: setuptools in c:\users\user_\appdata\roaming\python\python38\site-packages (from scrapy!=2.4.0,>=2.0->scrapy-playwright) (62.3.3)
Requirement already satisfied: w3lib>=1.17.0 in c:\users\user_\appdata\roaming\python\python38\site-packages (from scrapy!=2.4.0,>=2.0->scrapy-playwright) (1.22.0)
Requirement already satisfied: pyOpenSSL>=16.2.0 in c:\users\user_\appdata\roaming\python\python38\site-packages (from scrapy!=2.4.0,>=2.0->scrapy-playwright) (21.0.0)
Requirement already satisfied: six>=1.4.1 in c:\users\user_\appdata\roaming\python\python38\site-packages (from cryptography>=2.0->scrapy!=2.4.0,>=2.0->scrapy-playwright) (1.12.0)
Requirement already satisfied: cffi>=1.12 in c:\users\user_\appdata\roaming\python\python38\site-packages (from cryptography>=2.0->scrapy!=2.4.0,>=2.0->scrapy-playwright) (1.15.0)
Requirement already satisfied: jmespath>=0.9.5 in c:\users\user_\appdata\roaming\python\python38\site-packages (from itemloaders>=1.0.1->scrapy!=2.4.0,>=2.0->scrapy-playwright) (0.10.0)
Requirement already satisfied: attrs>=19.1.0 in c:\users\user_\appdata\roaming\python\python38\site-packages (from service-identity>=16.0.0->scrapy!=2.4.0,>=2.0->scrapy-playwright) (21.4.0)
Requirement already satisfied: pyasn1-modules in c:\users\user_\appdata\roaming\python\python38\site-packages (from service-identity>=16.0.0->scrapy!=2.4.0,>=2.0->scrapy-playwright) (0.2.8)
Requirement already satisfied: pyasn1 in c:\users\user_\appdata\roaming\python\python38\site-packages (from service-identity>=16.0.0->scrapy!=2.4.0,>=2.0->scrapy-playwright) (0.4.8)
Requirement already satisfied: constantly>=15.1 in c:\users\user_\appdata\roaming\python\python38\site-packages (from Twisted>=17.9.0->scrapy!=2.4.0,>=2.0->scrapy-playwright) (15.1.0)
Requirement already satisfied: incremental>=21.3.0 in c:\users\user_\appdata\roaming\python\python38\site-packages (from Twisted>=17.9.0->scrapy!=2.4.0,>=2.0->scrapy-playwright) (21.3.0)
Requirement already satisfied: hyperlink>=17.1.1 in c:\users\user_\appdata\roaming\python\python38\site-packages (from Twisted>=17.9.0->scrapy!=2.4.0,>=2.0->scrapy-playwright) (21.0.0)
Requirement already satisfied: twisted-iocpsupport<2,>=1.0.2 in c:\users\user_\appdata\roaming\python\python38\site-packages (from Twisted>=17.9.0->scrapy!=2.4.0,>=2.0->scrapy-playwright) (1.0.2)
Requirement already satisfied: Automat>=0.8.0 in c:\users\user_\appdata\roaming\python\python38\site-packages (from Twisted>=17.9.0->scrapy!=2.4.0,>=2.0->scrapy-playwright) (20.2.0)
Requirement already satisfied: requests>=2.1.0 in c:\users\user_\appdata\roaming\python\python38\site-packages (from tldextract->scrapy!=2.4.0,>=2.0->scrapy-playwright) (2.26.0)
Requirement already satisfied: filelock>=3.0.8 in c:\users\user_\appdata\roaming\python\python38\site-packages (from tldextract->scrapy!=2.4.0,>=2.0->scrapy-playwright) (3.7.0)
Requirement already satisfied: requests-file>=1.4 in c:\users\user_\appdata\roaming\python\python38\site-packages (from tldextract->scrapy!=2.4.0,>=2.0->scrapy-playwright) (1.5.1)
Requirement already satisfied: idna in c:\users\user_\appdata\roaming\python\python38\site-packages (from tldextract->scrapy!=2.4.0,>=2.0->scrapy-playwright) (3.2)
Requirement already satisfied: pycparser in c:\users\user_\appdata\roaming\python\python38\site-packages (from cffi>=1.12->cryptography>=2.0->scrapy!=2.4.0,>=2.0->scrapy-playwright) (2.21)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\user_\appdata\roaming\python\python38\site-packages (from requests>=2.1.0->tldextract->scrapy!=2.4.0,>=2.0->scrapy-playwright) (2021.5.30)
Requirement already satisfied: charset-normalizer~=2.0.0 in c:\users\user_\appdata\roaming\python\python38\site-packages (from requests>=2.1.0->tldextract->scrapy!=2.4.0,>=2.0->scrapy-playwright) (2.0.2)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\users\user_\appdata\roaming\python\python38\site-packages (from requests>=2.1.0->tldextract->scrapy!=2.4.0,>=2.0->scrapy-playwright) (1.26.6)
Installing collected packages: scrapy-playwright
Successfully installed scrapy-playwright-0.0.17

Enterキーを押すと、インストールが開始され、上記のように「Successfully installed」と表示されます。これが表示されれば、scrapy-playwrightが正常にインストールされたことになります。

なお、今回はscrapy-playwrightのバージョン0.0.17をインストールしました。