欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页  >  IT编程

Python爬虫5-利用usergent伪装访问方式

程序员文章站 2024-01-26 22:41:52
GitHub代码练习地址:https://github.com/Neo-ML/PythonPractice/blob/master/SpiderPrac08_useragent.py ......

github代码练习地址:https://github.com/neo-ml/pythonpractice/blob/master/spiderprac08_useragent.py


useragent
useragent: 用户代理,简称ua, 属于heads的一部分,服务器通过ua来判断访问者身份
常见的ua值如下,使用的时候可以直接复制粘贴,也可以用浏览器访问的时候抓包:
1.android

mozilla/5.0 (linux; android 4.1.1; nexus 7 build/jro03d) applewebkit/535.19 (khtml, like gecko) chrome/18.0.1025.166 safari/535.19
mozilla/5.0 (linux; u; android 4.0.4; en-gb; gt-i9300 build/imm76d) applewebkit/534.30 (khtml, like gecko) version/4.0 mobile safari/534.30
mozilla/5.0 (linux; u; android 2.2; en-gb; gt-p1000 build/froyo) applewebkit/533.1 (khtml, like gecko) version/4.0 mobile safari/533.1

2.firefox

mozilla/5.0 (windows nt 6.2; wow64; rv:21.0) gecko/20100101 firefox/21.0
mozilla/5.0 (android; mobile; rv:14.0) gecko/14.0 firefox/14.0

3.google chrome

mozilla/5.0 (windows nt 6.2; wow64) applewebkit/537.36 (khtml, like gecko) chrome/27.0.1453.94 safari/537.36
mozilla/5.0 (linux; android 4.0.4; galaxy nexus build/imm76b) applewebkit/535.19 (khtml, like gecko) chrome/18.0.1025.133 mobile safari/535.19

4.ios

mozilla/5.0 (ipad; cpu os 5_0 like mac os x) applewebkit/534.46 (khtml, like gecko) version/5.1 mobile/9a334 safari/7534.48.3
mozilla/5.0 (ipod; u; cpu like mac os x; en) applewebkit/420.1 (khtml, like gecko) version/3.0 mobile/3a101a safari/419.3

设置ua可以通过两种方式:
heads
add_header