awstats 简单日志分析

 1、下载awstats （确保perl 语言已经安装）

wget http://jaist.dl.sourceforge.net/project/awstats/AWStats/7.2/awstats-7.2.tar.gz
tar xzf awstats-7.2.tar.gz
mv awstats-7.2  /usr/local/awststs
chown root:root -R /usr/local/awststs
chmod a+x /usr/local/awststs/tools/

运行配置向导
1、/usr/local/awstats/tools/awstats_configure.p2、根据提示选择 none

3、提示是否配置 选择Y

4、填写分析的域名 例如；www.zhzhao.com

5、输入配置文件存储的路径 ，直接回车存储到 /etc/awstats/awststs.www.zhzhao.com.conf 目录中

6、回车确定结束；

7、修改配置文件
　 

　 sed -i 's#LogFile="/var/log/httpd/mylog.log"#LogFile="/mydata/log_statstcal/access.log"#g' /etc/awstats/awststs.www.zhzhao.com.conf

      sed -i  's#DirData="/var/lib/awstats"#DirData="/mydata/logdata/"#g  /etc/awstats/awststs.www.zhzhao.com.conf

　　#LoadPlugin="decodeutfkeys"  去掉#注释
　　
　　logfile 定义读取log的路径 

　　dirdata 生成的分析数据存储路径


8、awstats 配置日志格式
配置分析 apache 日志时候 logformat 选择1 、在配置文件中 /etc/awstats/awststs.www.zhzhao.com.conf 配置 ；如果是IIS 则选择log_format=2

分析nginx 时候，自定义logformat 格式；
对应自身nginx log进行配置： 
我的分析nginx log 的logformat 格式为 把LogFormat =1 #LogFormat =1 注释掉 

LogFormat="%host %other %time1 %methodurl %code %bytesd %refererquot %uaquot %other "

awstats 日志格式解释如下

#   %host             Client hostname or IP address (or Sender host for mail log)
#   %host_r           Receiver hostname or IP address (for mail log)
#   %lognamequot      Authenticated login/user with format: "john"
#   %logname          Authenticated login/user with format: john
#   %time1            Date and time with format: [dd/mon/yyyy:hh:mm:ss +0000] or [dd/mon/yyyy:hh:mm:ss]
#   %time2            Date and time with format: yyyy-mm-dd hh:mm:ss
#   %time3            Date and time with format: Mon dd hh:mm:ss or Mon dd hh:mm:ss yyyy
#   %time4            Date and time with unix timestamp format: dddddddddd
#   %time5            Date and time with format iso: yyyy-mm-ddThh:mm:ss
#   %methodurl        Method and URL with format: "GET /index.html HTTP/x.x"
#   %methodurlnoprot  Method and URL with format: "GET /index.html"
#   %method           Method with format: GET
#   %url              URL only with format: /index.html
#   %query            Query string (used by URLWithQuery option)
#   %code             Return code status (with format for web log: 999)
#   %bytesd           Size of document in bytes
#   %refererquot      Referer page with format: "http://from.com/from.htm"
#   %referer          Referer page with format: http://from.com/from.htm
#   %uabracket        User agent with format: [Mozilla/4.0 (compatible, ...)]
#   %uaquot           User agent with format: "Mozilla/4.0 (compatible, ...)"
#   %ua               User agent with format: Mozilla/4.0_(compatible...)
#   %gzipin           mod_gzip compression input bytes: In:XXX
#   %gzipout          mod_gzip compression output bytes & ratio: Out:YYY:ZZpct.
#   %gzipratio        mod_gzip compression ratio: ZZpct.
#   %deflateratio     mod_deflate compression ratio with format: (ZZ)
#   %email            EMail sender (for mail log)
#   %email_r          EMail receiver (for mail log)
#   %virtualname      Web sever virtual hostname. Use this tag when same log
#                     contains data of several virtual web servers. AWStats
#                     will discard records not in SiteDomain nor HostAliases
#   %cluster          If log file is provided from several computers (merged by
#                     logresolvemerge.pl), use this to define cluster id field.
#   %extraX           Another field that you plan to use for building a
#                     personalized report with ExtraSection feature (See later).
#   If your log format has some fields not included in this list, use:
#   %other            Means another not used field
#   %otherquot        Means another not used double quoted field
#
# Examples for Apache combined logs (following two examples are equivalent):
# LogFormat = 1
# LogFormat = "%host %other %logname %time1 %methodurl %code %bytesd %refererquot %uaquot"%other

9、nginx 配置文件中log记录格式：

　　log_format main  　 '$http_x_forwarded_for $remote_user [$time_local] "$request" '
　　　　　　　　　　　　'$status $body_bytes_sent "$http_referer" '
　　　　　　　　　　　　'"$http_user_agent" "$http_x_forwarded_for"';
 
　　access_log logs/access.log main;

10、解决页面中文乱码问题；如果是apache 作为web，则设置apache 的httpd.conf 编码为gb2312

　　AddDefaultCharset GB2312

11、安装GeoIP 
下载 GeoIP.dat 上传到/usr/local/awstst/var/ 目录中

修改配置 文件

LoadPlugin="geoip GEOIP_STANDARD /usr/local/awstst/var/GeoIP.dat"
LoadPlugin="geoip_city_maxmind GEOIP_STANDARD/usr/local/awstst/var/GeoLiteCity.dat"

12、使用QQ纯真版IP

cd /usr/local/awstats/wwwroot/cgi-bin/plugins
# yum安装时目录为：/usr/share/awstats/wwwroot/cgi-bin/plugins ，没有则建立
wget http://www.haiyun.me/download/qqwry.pl
wget http://www.haiyun.me/download/qqhostinfo.pm
下载 qqwry.dat 文件，保存在百度网盘中；

上传 qqwry.dat 到 /usr/local/awstats/wwwroot/cgi-bin/plugins 目录中

#修改qqwry.pl内IP数据目录：
my $ipfile="${DIR}/plugins/qqwry.dat";

13、编写脚本，部署测试
　#！/bin/sh
　Awstats_Dir="/usr/local/awstats" #awstats 安装目录
　webstatic=/var/www/html/awstats #web 目录生成保存的数据

　　if [ ! -d $webstatic ];
　　then mkdir -p$webstatic
　　else
　　rm -rf $webstatic/*
　　echo 'del old webinfo sucess!'
　　fi
　　$Awstats_Dir/wwwroot/cgi-bin/awstats.pl -update -config=www.youliang.com

　　$Awstats_Dir/tools/awstats_buildstaticpages.pl -update -config=www.zhzhao.com -awstatsprog=$Awstats_Dir/wwwroot/cgi-bin/awstats.pl -lang=cn -dir=$webstatic