欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页  >  IT编程

python爬虫原理讲解

程序员文章站 2022-07-23 18:14:58
原理很简单,利用python下载我更新的txt题库,然后转化为字典,最后对应答案输出。 # -*- coding: utf-8 -*- """...

原理很简单,利用python下载我更新的txt题库,然后转化为字典,最后对应答案输出。

# -*- coding: utf-8 -*-
"""
Created on Tue Jan  9 12:05:34 2018

@author: admin
"""
info = []
from bs4 import BeautifulSoup
import requests
import os
print('========Welcome to use QduSafeExamHelper========')
print('|                                              |')
print('|             Made with python3.6              |')
print('|                                              |')
print('=========== Verison: 1.0.6 by ARCHer ===========\n')
print('请将考试页面保存(Ctrl+s)在当前程序同一目录下并按照下方面的说明操作!')
print('1.请确认保存的网页名为.html ')
print('2.html文件与本程序在同一目录')
print('3.确认前两项无误后按回车确认')
input()
result = {}
update_url = 'https://kaoshidaan.oss-cn-qingdao.aliyuncs.com/result.txt'
r = requests.get(update_url)
with open("result.txt",'wb') as f:
    f.write(r.content)
file = open( 'result.txt' , 'r' )
lines = file.readlines()
file.close()
for line in lines:
    result_data = line.split('----', 1 )
    result[result_data[0]] = result_data[1].strip()
#print(result)
with open( '.html',encoding = 'utf-8') as wb_data:
    soup = BeautifulSoup( wb_data , 'lxml' )
    numbers = soup.select('dd > a')
    questions = soup.select( 'dd > a' )
for question,number in zip( questions, numbers ):
    data = {
            'question':question.get('href'),
            'number':number.get_text(),
            'answer':result.get(question.get('href'))
            }
    info.append(data)
os.remove('result.txt')
for i in info:
    print('题目序号:',i['number'] ,'题目答案:',i['answer'] )
input('Finished')