欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页  >  IT编程

C#网页信息采集方法汇总

程序员文章站 2023-12-15 17:38:58
本文实例总结了三种常用的c#网页信息采集方法。分享给大家供大家参考。具体实现方法如下: 一、通过httpwebresponse 来获取 复制代码 代码如下:public...

本文实例总结了三种常用的c#网页信息采集方法。分享给大家供大家参考。具体实现方法如下:

一、通过httpwebresponse 来获取

复制代码 代码如下:
public static string checkteamsiteurl(string url) 

        string response = ""; 
        httpwebresponse httpresponse = null; 
 
        //assert: user have access to url  
        try 
        { 
            httpwebrequest httprequest = (httpwebrequest)webrequest.create(url); 
            httprequest.headers.set("pragma", "no-cache"); 
 
                // request.headers.set("keepalive", "true"); 
 
                httprequest.cookiecontainer = new cookiecontainer(); 
 
 
 
                httprequest.referer = url; 
 
                httprequest.useragent = "mozilla/4.0 (compatible; msie 6.0; windows nt 5.0; .net clr 1.1.4322; .net clr 2.0.50727)"; 
 
               
 
            httprequest.credentials = system.net.credentialcache.defaultcredentials; 
            httpresponse = (httpwebresponse)httprequest.getresponse(); 
             
        } 
        catch (exception ex) 
        { 
            throw new applicationexception("http 403 access denied, url: " + url, ex); 
        } 
 
        //if here, the url is correct and the user has access  
        try 
        { 
            string strencod = httpresponse.contenttype; 
            streamreader stream; 
            if (strencod.tolower().indexof("utf") != -1) 
            { 
                stream = new streamreader(httpresponse.getresponsestream(), system.text.encoding.utf8); 
            } 
            else 
            { 
                stream = new streamreader(httpresponse.getresponsestream(), system.text.encoding.default); 
            } 
            
            char[] buff = new char[4000]; 
            stream.readblock(buff,0,4000); 
            response = new string(buff); 
            stream.close(); 
            httpresponse.close(); 
        } 
        catch (exception ex) 
        { 
            throw new applicationexception("http 404 page not found, url: " + url, ex); 
        } 
        return response; 
}

 
二、通过 webresponse 来获取

复制代码 代码如下:
public static string getpage(string url) 
{
        webresponse result = null; 
        string resultstring = ""; 
        try 
        { 
            webrequest req = webrequest.create(url); 
            req.timeout = 30000; 
            result = req.getresponse(); 
            stream receivestream = result.getresponsestream(); 
 
            //read the stream into a string 
            //streamreader sr = new streamreader(receivestream, system.text.encoding.utf8); 
            string strencod = result.contenttype; 
            streamreader sr; 
            if (strencod.tolower().indexof("utf") != -1) 
            { 
                sr = new streamreader(receivestream, system.text.encoding.utf8); 
            } 
            else 
            { 
                sr = new streamreader(receivestream, system.text.encoding.default); 
            } 
            resultstring = sr.readtoend(); 
            js.alert(resultstring); 
            //console.writeline(resultstring); 
        } 
        catch 
        { 
            throw new exception(); 
        } 
        finally 
        { 
            if (result != null) 
            { 
                result.close(); 
            } 
        } 
        return resultstring; 
}

 
三、通过webclient来获取

复制代码 代码如下:
public string get(int length) 

        try 
        { 
            getencodeing(); 
            webclient wb = new webclient(); 
            stream response = wb.openread(url); 
            streamreader reader = new streamreader(response, this.encoding, true, 256000); 
            char[] a = new char[length]; 
            int i  = reader.read(a,0,length); 
            reader.close(); 
            return new string(a); 
        } 
        catch (exception e) 
        { 
            return e.message; 
            //return null; 
        } 

private void getencodeing() 
{
        switch (this.encode) 
        { 
            case "utf-8": encoding = encoding.utf8; break; 
            case "gb2312": encoding = encoding.getencoding("gb2312"); break; 
            case "ascii": encoding = encoding.ascii; break; 
            default: encoding = encoding.getencoding(encode); break; 
        } 
}

希望本文所述对大家的c#程序设计有所帮助。

上一篇:

下一篇: