Ubuntu16.04 安装最新 Scrapy 完全攻略

[email protected]:~$ sudo apt-get install python-pip
[sudo] password for jhczz: 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following package was automatically installed and is no longer required:
  snap-confine
Use 'sudo apt autoremove' to remove it.
The following additional packages will be installed:
  libexpat1-dev libpython-all-dev libpython-dev libpython2.7-dev python-all
  python-all-dev python-dev python-pip-whl python-pkg-resources
  python-setuptools python-wheel python2.7-dev
Suggested packages:
  python-setuptools-doc
The following NEW packages will be installed:
  libexpat1-dev libpython-all-dev libpython-dev libpython2.7-dev python-all
  python-all-dev python-dev python-pip python-pip-whl python-pkg-resources
  python-setuptools python-wheel python2.7-dev
0 upgraded, 13 newly installed, 0 to remove and 3 not upgraded.
Need to get 29.8 MB of archives.
After this operation, 45.1 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://hk.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libexpat1-dev amd64 2.1.0-7ubuntu0.16.04.3 [115 kB]
Get:2 http://hk.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libpython2.7-dev amd64 2.7.12-1ubuntu0~16.04.1 [27.8 MB]
30% [2 libpython2.7-dev 10.5 MB/27.8 MB 38%]                                    Get:3 http://hk.archive.ubuntu.com/ubuntu xenial/main amd64 libpython-dev amd64 2.7.11-1 [7,728 B]
Get:4 http://hk.archive.ubuntu.com/ubuntu xenial/main amd64 libpython-all-dev amd64 2.7.11-1 [992 B]
Get:5 http://hk.archive.ubuntu.com/ubuntu xenial/main amd64 python-all amd64 2.7.11-1 [978 B]
Get:6 http://hk.archive.ubuntu.com/ubuntu xenial-updates/main amd64 python2.7-dev amd64 2.7.12-1ubuntu0~16.04.1 [276 kB]
Get:7 http://hk.archive.ubuntu.com/ubuntu xenial/main amd64 python-dev amd64 2.7.11-1 [1,160 B]
Get:8 http://hk.archive.ubuntu.com/ubuntu xenial/main amd64 python-all-dev amd64 2.7.11-1 [1,000 B]
Get:9 http://hk.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 python-pip-whl all 8.1.1-2ubuntu0.4 [1,110 kB]
Get:10 http://hk.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 python-pip all 8.1.1-2ubuntu0.4 [144 kB]
Get:11 http://hk.archive.ubuntu.com/ubuntu xenial/main amd64 python-pkg-resources all 20.7.0-1 [108 kB]
Get:12 http://hk.archive.ubuntu.com/ubuntu xenial/main amd64 python-setuptools all 20.7.0-1 [169 kB]
Get:13 http://hk.archive.ubuntu.com/ubuntu xenial/universe amd64 python-wheel all 0.29.0-1 [48.0 kB]
Fetched 29.8 MB in 19s (1,526 kB/s)                                            
Selecting previously unselected package libexpat1-dev:amd64.
(Reading database ... 210250 files and directories currently installed.)
Preparing to unpack .../libexpat1-dev_2.1.0-7ubuntu0.16.04.3_amd64.deb ...
Unpacking libexpat1-dev:amd64 (2.1.0-7ubuntu0.16.04.3) ...
Selecting previously unselected package libpython2.7-dev:amd64.
Preparing to unpack .../libpython2.7-dev_2.7.12-1ubuntu0~16.04.1_amd64.deb ...
Unpacking libpython2.7-dev:amd64 (2.7.12-1ubuntu0~16.04.1) ...
Selecting previously unselected package libpython-dev:amd64.
Preparing to unpack .../libpython-dev_2.7.11-1_amd64.deb ...
Unpacking libpython-dev:amd64 (2.7.11-1) ...
Selecting previously unselected package libpython-all-dev:amd64.
Preparing to unpack .../libpython-all-dev_2.7.11-1_amd64.deb ...
Unpacking libpython-all-dev:amd64 (2.7.11-1) ...
Selecting previously unselected package python-all.
Preparing to unpack .../python-all_2.7.11-1_amd64.deb ...
Unpacking python-all (2.7.11-1) ...
Selecting previously unselected package python2.7-dev.
Preparing to unpack .../python2.7-dev_2.7.12-1ubuntu0~16.04.1_amd64.deb ...
Unpacking python2.7-dev (2.7.12-1ubuntu0~16.04.1) ...
Selecting previously unselected package python-dev.
Preparing to unpack .../python-dev_2.7.11-1_amd64.deb ...
Unpacking python-dev (2.7.11-1) ...
Selecting previously unselected package python-all-dev.
Preparing to unpack .../python-all-dev_2.7.11-1_amd64.deb ...
Unpacking python-all-dev (2.7.11-1) ...
Selecting previously unselected package python-pip-whl.
Preparing to unpack .../python-pip-whl_8.1.1-2ubuntu0.4_all.deb ...
Unpacking python-pip-whl (8.1.1-2ubuntu0.4) ...
Selecting previously unselected package python-pip.
Preparing to unpack .../python-pip_8.1.1-2ubuntu0.4_all.deb ...
Unpacking python-pip (8.1.1-2ubuntu0.4) ...
Selecting previously unselected package python-pkg-resources.
Preparing to unpack .../python-pkg-resources_20.7.0-1_all.deb ...
Unpacking python-pkg-resources (20.7.0-1) ...
Selecting previously unselected package python-setuptools.
Preparing to unpack .../python-setuptools_20.7.0-1_all.deb ...
Unpacking python-setuptools (20.7.0-1) ...
Selecting previously unselected package python-wheel.
Preparing to unpack .../python-wheel_0.29.0-1_all.deb ...
Unpacking python-wheel (0.29.0-1) ...
Processing triggers for doc-base (0.10.7) ...
Processing 1 added doc-base file...
Processing triggers for man-db (2.7.5-1) ...
Setting up libexpat1-dev:amd64 (2.1.0-7ubuntu0.16.04.3) ...
Setting up libpython2.7-dev:amd64 (2.7.12-1ubuntu0~16.04.1) ...
Setting up libpython-dev:amd64 (2.7.11-1) ...
Setting up libpython-all-dev:amd64 (2.7.11-1) ...
Setting up python-all (2.7.11-1) ...
Setting up python2.7-dev (2.7.12-1ubuntu0~16.04.1) ...
Setting up python-dev (2.7.11-1) ...
Setting up python-all-dev (2.7.11-1) ...
Setting up python-pip-whl (8.1.1-2ubuntu0.4) ...
Setting up python-pip (8.1.1-2ubuntu0.4) ...
Setting up python-pkg-resources (20.7.0-1) ...
Setting up python-setuptools (20.7.0-1) ...
Setting up python-wheel (0.29.0-1) ...
[email protected]:~$ sudo apt-get install python-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
python-dev is already the newest version (2.7.11-1).
python-dev set to manually installed.
The following package was automatically installed and is no longer required:
  snap-confine
Use 'sudo apt autoremove' to remove it.
0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.
[email protected]:~$ sudo apt-get install libevent-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following package was automatically installed and is no longer required:
  snap-confine
Use 'sudo apt autoremove' to remove it.
The following additional packages will be installed:
  libevent-core-2.0-5 libevent-extra-2.0-5 libevent-openssl-2.0-5
  libevent-pthreads-2.0-5
The following NEW packages will be installed:
  libevent-core-2.0-5 libevent-dev libevent-extra-2.0-5 libevent-openssl-2.0-5
  libevent-pthreads-2.0-5
0 upgraded, 5 newly installed, 0 to remove and 3 not upgraded.
Need to get 349 kB of archives.
After this operation, 1,846 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://hk.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libevent-core-2.0-5 amd64 2.0.21-stable-2ubuntu0.16.04.1 [70.6 kB]
Get:2 http://hk.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libevent-extra-2.0-5 amd64 2.0.21-stable-2ubuntu0.16.04.1 [51.1 kB]
Get:3 http://hk.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libevent-pthreads-2.0-5 amd64 2.0.21-stable-2ubuntu0.16.04.1 [5,020 B]
Get:4 http://hk.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libevent-openssl-2.0-5 amd64 2.0.21-stable-2ubuntu0.16.04.1 [10.6 kB]
Get:5 http://hk.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libevent-dev amd64 2.0.21-stable-2ubuntu0.16.04.1 [211 kB]
Fetched 349 kB in 1s (202 kB/s)       
Selecting previously unselected package libevent-core-2.0-5:amd64.
(Reading database ... 210689 files and directories currently installed.)
Preparing to unpack .../libevent-core-2.0-5_2.0.21-stable-2ubuntu0.16.04.1_amd64.deb ...
Unpacking libevent-core-2.0-5:amd64 (2.0.21-stable-2ubuntu0.16.04.1) ...
Selecting previously unselected package libevent-extra-2.0-5:amd64.
Preparing to unpack .../libevent-extra-2.0-5_2.0.21-stable-2ubuntu0.16.04.1_amd64.deb ...
Unpacking libevent-extra-2.0-5:amd64 (2.0.21-stable-2ubuntu0.16.04.1) ...
Selecting previously unselected package libevent-pthreads-2.0-5:amd64.
Preparing to unpack .../libevent-pthreads-2.0-5_2.0.21-stable-2ubuntu0.16.04.1_amd64.deb ...
Unpacking libevent-pthreads-2.0-5:amd64 (2.0.21-stable-2ubuntu0.16.04.1) ...
Selecting previously unselected package libevent-openssl-2.0-5:amd64.
Preparing to unpack .../libevent-openssl-2.0-5_2.0.21-stable-2ubuntu0.16.04.1_amd64.deb ...
Unpacking libevent-openssl-2.0-5:amd64 (2.0.21-stable-2ubuntu0.16.04.1) ...
Selecting previously unselected package libevent-dev.
Preparing to unpack .../libevent-dev_2.0.21-stable-2ubuntu0.16.04.1_amd64.deb ...
Unpacking libevent-dev (2.0.21-stable-2ubuntu0.16.04.1) ...
Processing triggers for libc-bin (2.23-0ubuntu9) ...
Setting up libevent-core-2.0-5:amd64 (2.0.21-stable-2ubuntu0.16.04.1) ...
Setting up libevent-extra-2.0-5:amd64 (2.0.21-stable-2ubuntu0.16.04.1) ...
Setting up libevent-pthreads-2.0-5:amd64 (2.0.21-stable-2ubuntu0.16.04.1) ...
Setting up libevent-openssl-2.0-5:amd64 (2.0.21-stable-2ubuntu0.16.04.1) ...
Setting up libevent-dev (2.0.21-stable-2ubuntu0.16.04.1) ...
Processing triggers for libc-bin (2.23-0ubuntu9) ...
[email protected]:~$ sudo apt-get install libssl-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following package was automatically installed and is no longer required:
  snap-confine
Use 'sudo apt autoremove' to remove it.
The following additional packages will be installed:
  libssl-doc zlib1g-dev
The following NEW packages will be installed:
  libssl-dev libssl-doc zlib1g-dev
0 upgraded, 3 newly installed, 0 to remove and 3 not upgraded.
Need to get 2,590 kB of archives.
After this operation, 10.5 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://hk.archive.ubuntu.com/ubuntu xenial-updates/main amd64 zlib1g-dev amd64 1:1.2.8.dfsg-2ubuntu4.1 [168 kB]
Get:2 http://hk.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libssl-dev amd64 1.0.2g-1ubuntu4.8 [1,345 kB]
Get:3 http://hk.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libssl-doc all 1.0.2g-1ubuntu4.8 [1,078 kB]
Fetched 2,590 kB in 2s (1,070 kB/s)   
Selecting previously unselected package zlib1g-dev:amd64.
(Reading database ... 210767 files and directories currently installed.)
Preparing to unpack .../zlib1g-dev_1%3a1.2.8.dfsg-2ubuntu4.1_amd64.deb ...
Unpacking zlib1g-dev:amd64 (1:1.2.8.dfsg-2ubuntu4.1) ...
Selecting previously unselected package libssl-dev:amd64.
Preparing to unpack .../libssl-dev_1.0.2g-1ubuntu4.8_amd64.deb ...
Unpacking libssl-dev:amd64 (1.0.2g-1ubuntu4.8) ...
Selecting previously unselected package libssl-doc.
Preparing to unpack .../libssl-doc_1.0.2g-1ubuntu4.8_all.deb ...
Unpacking libssl-doc (1.0.2g-1ubuntu4.8) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up zlib1g-dev:amd64 (1:1.2.8.dfsg-2ubuntu4.1) ...
Setting up libssl-dev:amd64 (1.0.2g-1ubuntu4.8) ...
Setting up libssl-doc (1.0.2g-1ubuntu4.8) ...
[email protected]:~$ sudo pip install scrapy
The directory '/home/jhczz/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/jhczz/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting scrapy
  Downloading Scrapy-1.4.0-py2.py3-none-any.whl (248kB)
    100% |████████████████████████████████| 256kB 285kB/s 
Collecting queuelib (from scrapy)
  Downloading queuelib-1.4.2-py2.py3-none-any.whl
Collecting cssselect>=0.9 (from scrapy)
  Downloading cssselect-1.0.1-py2.py3-none-any.whl
Collecting six>=1.5.2 (from scrapy)
  Downloading six-1.10.0-py2.py3-none-any.whl
Collecting parsel>=1.1 (from scrapy)
  Downloading parsel-1.2.0-py2.py3-none-any.whl
Collecting service-identity (from scrapy)
  Downloading service_identity-17.0.0-py2.py3-none-any.whl
Collecting pyOpenSSL (from scrapy)
  Downloading pyOpenSSL-17.2.0-py2.py3-none-any.whl (52kB)
    100% |████████████████████████████████| 61kB 358kB/s 
Collecting w3lib>=1.17.0 (from scrapy)
  Downloading w3lib-1.18.0-py2.py3-none-any.whl
Collecting lxml (from scrapy)
  Downloading lxml-3.8.0-cp27-cp27mu-manylinux1_x86_64.whl (6.8MB)
    100% |████████████████████████████████| 6.8MB 172kB/s 
Collecting Twisted>=13.1.0 (from scrapy)
  Downloading Twisted-17.5.0.tar.bz2 (3.0MB)
    100% |████████████████████████████████| 3.0MB 392kB/s 
Collecting PyDispatcher>=2.0.5 (from scrapy)
  Downloading PyDispatcher-2.0.5.tar.gz
Collecting pyasn1 (from service-identity->scrapy)
  Downloading pyasn1-0.3.3-py2.py3-none-any.whl (63kB)
    100% |████████████████████████████████| 71kB 2.0MB/s 
Collecting pyasn1-modules (from service-identity->scrapy)
  Downloading pyasn1_modules-0.1.1-py2.py3-none-any.whl (60kB)
    100% |████████████████████████████████| 61kB 2.4MB/s 
Collecting attrs (from service-identity->scrapy)
  Downloading attrs-17.2.0-py2.py3-none-any.whl
Collecting cryptography>=1.9 (from pyOpenSSL->scrapy)
  Downloading cryptography-2.0.3-cp27-cp27mu-manylinux1_x86_64.whl (2.2MB)
    100% |████████████████████████████████| 2.2MB 537kB/s 
Collecting zope.interface>=3.6.0 (from Twisted>=13.1.0->scrapy)
  Downloading zope.interface-4.4.2-cp27-cp27mu-manylinux1_x86_64.whl (170kB)
    100% |████████████████████████████████| 174kB 2.2MB/s 
Collecting constantly>=15.1 (from Twisted>=13.1.0->scrapy)
  Downloading constantly-15.1.0-py2.py3-none-any.whl
Collecting incremental>=16.10.1 (from Twisted>=13.1.0->scrapy)
  Downloading incremental-17.5.0-py2.py3-none-any.whl
Collecting Automat>=0.3.0 (from Twisted>=13.1.0->scrapy)
  Downloading Automat-0.6.0-py2.py3-none-any.whl
Collecting hyperlink>=17.1.1 (from Twisted>=13.1.0->scrapy)
  Downloading hyperlink-17.3.1-py2.py3-none-any.whl (73kB)
    100% |████████████████████████████████| 81kB 3.0MB/s 
Collecting ipaddress (from cryptography>=1.9->pyOpenSSL->scrapy)
  Downloading ipaddress-1.0.18-py2-none-any.whl
Collecting idna>=2.1 (from cryptography>=1.9->pyOpenSSL->scrapy)
  Downloading idna-2.6-py2.py3-none-any.whl (56kB)
    100% |████████████████████████████████| 61kB 3.0MB/s 
Collecting asn1crypto>=0.21.0 (from cryptography>=1.9->pyOpenSSL->scrapy)
  Downloading asn1crypto-0.22.0-py2.py3-none-any.whl (97kB)
    100% |████████████████████████████████| 102kB 2.8MB/s 
Collecting enum34 (from cryptography>=1.9->pyOpenSSL->scrapy)
  Downloading enum34-1.1.6-py2-none-any.whl
Collecting cffi>=1.7 (from cryptography>=1.9->pyOpenSSL->scrapy)
  Downloading cffi-1.10.0-cp27-cp27mu-manylinux1_x86_64.whl (392kB)
    100% |████████████████████████████████| 399kB 1.7MB/s 
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from zope.interface>=3.6.0->Twisted>=13.1.0->scrapy)
Collecting pycparser (from cffi>=1.7->cryptography>=1.9->pyOpenSSL->scrapy)
  Downloading pycparser-2.18.tar.gz (245kB)
    100% |████████████████████████████████| 256kB 2.1MB/s 
Installing collected packages: queuelib, cssselect, six, w3lib, lxml, parsel, pyasn1, ipaddress, idna, asn1crypto, enum34, pycparser, cffi, cryptography, pyOpenSSL, pyasn1-modules, attrs, service-identity, zope.interface, constantly, incremental, Automat, hyperlink, Twisted, PyDispatcher, scrapy
  Running setup.py install for pycparser ... done
  Running setup.py install for Twisted ... done
  Running setup.py install for PyDispatcher ... done
Successfully installed Automat-0.6.0 PyDispatcher-2.0.5 Twisted-17.5.0 asn1crypto-0.22.0 attrs-17.2.0 cffi-1.10.0 constantly-15.1.0 cryptography-2.0.3 cssselect-1.0.1 enum34-1.1.6 hyperlink-17.3.1 idna-2.6 incremental-17.5.0 ipaddress-1.0.18 lxml-3.8.0 parsel-1.2.0 pyOpenSSL-17.2.0 pyasn1-0.3.3 pyasn1-modules-0.1.1 pycparser-2.18 queuelib-1.4.2 scrapy-1.4.0 service-identity-17.0.0 six-1.10.0 w3lib-1.18.0 zope.interface-4.4.2
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
[email protected]:~$ scrapy version
Scrapy 1.4.0
[email protected]:~$