ASP.NET抓取网页内容的实现方法

本文实例讲述了asp.net抓取网页内容的实现方法。分享给大家供大家参考。具体实现方法如下：

一、asp.net 使用httpwebrequest抓取网页内容

/// <summary>方法一：比较推荐 
/// 用httpwebrequest取得网页源码 
/// 对于带bom的网页很有效，不管是什么编码都能正确识别 
/// </summary> 
/// <param name="url">网页地址" </param> 
/// <returns>返回网页源文件</returns> 
public static string gethtmlsource2(string url) 
{ 
    //处理内容 
    string html = ""; 
    httpwebrequest request = (httpwebrequest)webrequest.create(url); 
    request.accept = "*/*"; //接受任意文件 
    request.useragent = "mozilla/4.0 (compatible; msie 6.0; windows nt 5.2; .net clr 1.1.4322)"; //  
    request.allowautoredirect = true;//是否允许302 
    //request.cookiecontainer = new cookiecontainer();//cookie容器， 
    request.referer = url; //当前页面的引用 
    httpwebresponse response = (httpwebresponse)request.getresponse(); 
    stream stream = response.getresponsestream(); 
    streamreader reader = new streamreader(stream, encoding.default); 
    html = reader.readtoend(); 
    stream.close(); 
    return html; 
}

二、asp.net 使用 webresponse 抓取网页内容

public static string gethttpdata2(string url) 
{ 
    string sexception = null; 
    string srslt = null; 
    webresponse owebrps = null; 
    webrequest owebrqst = webrequest.create(url); 
    owebrqst.timeout = 50000; 
    try 
    { 
        owebrps = owebrqst.getresponse(); 
    } 
    catch (webexception e) 
    { 
        sexception = e.message.tostring(); 
    } 
    catch (exception e) 
    { 
        sexception = e.tostring(); 
    } 
    finally 
    { 
        if (owebrps != null) 
        { 
            streamreader ostreamrd = new streamreader(owebrps.getresponsestream(), encoding.getencoding("utf-8")); 
            srslt = ostreamrd.readtoend(); 
            ostreamrd.close(); 
            owebrps.close(); 
        } 
    } 
    return srslt; 
}

希望本文所述对大家的c#程序设计有所帮助。