欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页  >  科技

一个简单的爬虫 头部构造

程序员文章站 2022-12-11 14:31:02
# -*- coding:utf-8 -*-import urllib.requestimport randomua_headers = { 'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:57.0) Gecko/20100101 Firefox/57.0',}head_list=[ "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv2.0.1) Gecko....
# -*- coding:utf-8 -*-
import urllib.request
import random
ua_headers = {
    'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:57.0) Gecko/20100101 Firefox/57.0',
}
head_list=[
        "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv2.0.1) Gecko/20100101 Firefox/4.0.1",
        "Mozilla/5.0 (Windows NT 6.1; rv2.0.1) Gecko/20100101 Firefox/4.0.1",
        "Opera/9.80 (Macintosh; Intel Mac OS X 10.6.8; U; en) Presto/2.8.131 Version/11.11",
        "Opera/9.80 (Windows NT 6.1; U; en) Presto/2.8.131 Version/11.11",
        "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_0) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11"
]
user_agent=random.choice(head_list)


url = r'https://www.baidu.com//'
request=urllib.request.Request(url)
request.add_header('User-Agent',user_agent)

print(request.get_header('User-agent'))

print(request.headers,request.type,request.data)

response=urllib.request.urlopen(request)

html=response.read()
print(html.decode('gb2312'))

 

本文地址:https://blog.csdn.net/dudu3332/article/details/109933427

相关标签: python 爬虫