芝麻HTTP:设置Selenium+Chrome代理
程序员文章站
2022-04-16 10:26:06
微博登录限制了错误次数···加上Cookie大批账号被封需要从Cookie池中 剔除被封的账号··· 需要使用代理··· 无赖百度了大半天都是特么的啥玩意儿???结果换成了 Google手到擒来 分分钟解决(那么问题来了?百度除了卖假药还会干啥?) Selenium+Chrome认证代理不能通过op ......
微博登录限制了错误次数···加上Cookie大批账号被封需要从Cookie池中 剔除被封的账号··· 需要使用代理··· 无赖百度了大半天都是特么的啥玩意儿???结果换成了 Google手到擒来 分分钟解决(那么问题来了?百度除了卖假药还会干啥?)
Selenium+Chrome认证代理不能通过options处理。只能换个方法使用扩展解决
原文地址:https://*.com/questions/29983106/how-can-i-set-proxy-with-authentication-in-selenium-chrome-web-driver-using-pyth#answer-30953780 (Stack Overflow 这是个好地方啊)
# -*- coding: utf-8 -*- # @Time : 2017/11/15 9:50 # @Author : 哎哟卧槽 # @Site : # @File : pubilc.py # @Software: PyCharm import string import zipfile def create_proxyauth_extension(proxy_host, proxy_port, proxy_username, proxy_password, scheme='http', plugin_path=None): """代理认证插件 args: proxy_host (str): 你的代理地址或者域名(str类型) proxy_port (int): 代理端口号(int类型) proxy_username (str):用户名(字符串) proxy_password (str): 密码 (字符串) kwargs: scheme (str): 代理方式 默认http plugin_path (str): 扩展的绝对路径 return str -> plugin_path """ if plugin_path is None: plugin_path = 'vimm_chrome_proxyauth_plugin.zip' manifest_json = """ { "version": "1.0.0", "manifest_version": 2, "name": "Chrome Proxy", "permissions": [ "proxy", "tabs", "unlimitedStorage", "storage", "<all_urls>", "webRequest", "webRequestBlocking" ], "background": { "scripts": ["background.js"] }, "minimum_chrome_version":"22.0.0" } """ background_js = string.Template( """ var config = { mode: "fixed_servers", rules: { singleProxy: { scheme: "${scheme}", host: "${host}", port: parseInt(${port}) }, bypassList: ["foobar.com"] } }; chrome.proxy.settings.set({value: config, scope: "regular"}, function() {}); function callbackFn(details) { return { authCredentials: { username: "${username}", password: "${password}" } }; } chrome.webRequest.onAuthRequired.addListener( callbackFn, {urls: ["<all_urls>"]}, ['blocking'] ); """ ).substitute( host=proxy_host, port=proxy_port, username=proxy_username, password=proxy_password, scheme=scheme, ) with zipfile.ZipFile(plugin_path, 'w') as zp: zp.writestr("manifest.json", manifest_json) zp.writestr("background.js", background_js) return plugin_path
使用方法:
from selenium import webdriver from common.pubilc import create_proxyauth_extension proxyauth_plugin_path = create_proxyauth_extension( proxy_host="XXXXX.com", proxy_port=9020, proxy_username="XXXXXXX", proxy_password="XXXXXXX" ) co = webdriver.ChromeOptions() # co.add_argument("--start-maximized") co.add_extension(proxyauth_plugin_path) driver = webdriver.Chrome(executable_path="C:\chromedriver.exe", chrome_options=co) driver.get("http://ip138.com/") print(driver.page_source)
无认证代理:
options = webdriver.ChromeOptions() options.add_argument('--proxy-server=http://ip:port') driver = webdriver.Chrome(executable_path="C:\chromedriver.exe", chrome_options=0ptions) driver.get("http://ip138.com/") print(driver.page_source)