欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页

记录透传日志功能设计代码

程序员文章站 2024-03-18 21:46:40
...
设计思想:
  很简单请求到controll以后拦截请求,存到表里,表有两个,一个字典表table1,记录请求接口和请求实体的属性值和ext扩展表的列的对应关系。第二个扩展表table2,存真正的业务数据,so easy
 
使用方法:
  在controller上增加 注解@AutoLog 支持多个实体,类属性上增加@AutoLogField注解:
(为什么只支持controller因为是基于HandlerInterceptor做的,如果要支持service也可以,需增加MethodInterceptor)
 
 
原理:配置相关注解以后,第一次请求时会自动把相关字段存字典表,设置了缓存,减少请求次数,如果后续请求参数增加属性需要入库需要执行插入sql插入mysql
举例:RequestVo增加了一个字段  name,name上增加注解@AutoLogField,数据库需要执行:
select * from  table1 where service_name ='**.getResultByIdNo'; 到方法
insert table1(service_name,column_name,ext_column_name) value(**.getResultByIdNo","name","ext3");
ext3来自table2还未使用的字段。
 
优点:和业务逻辑代码解耦,提高开发效率,异步线程池入库不影响原来逻辑,
     日志参数可视化(相较于直接存json),扩展表可以增加索引查询(相较于直接存json),
     避免了json大字段,打印了log日志兜底
 
缺点:1.新增字段需要执行sql(也可以做成自动取校验,那样就要控制查表得频率,访问字典表得情况就会变多)
     2.考虑到mysql表的字段大小,目前支持20个扩展字段,根据反射的字段顺序截取
     3.针对map,list,业务对象属性目前还是保存的json
 
代码和表结构:
部分包路径屏蔽了
 
1.拦截器:

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import org.springframework.web.method.HandlerMethod;
import org.springframework.web.servlet.HandlerInterceptor;
import org.springframework.web.servlet.ModelAndView;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.lang.reflect.Method;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

/**
 * 自动日志拦截器
 *
 * @author yuJie
 * @since 2020/5/16
 */
@Component
public class AutoLogInterceptor implements HandlerInterceptor {
    private static final ExecutorService threadPool
            = new ThreadPoolExecutor(4, 10, 10000, TimeUnit.SECONDS, new ArrayBlockingQueue<>(1000));
    @Autowired
    private AutomaticLoggingService automaticLoggingService;

    /**
     * 预处理
     *
     * @param request
     * @param response
     * @param handler
     * @return
     * @throws Exception
     */
    @Override
    public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {

        return true;
    }


    @Override
    public void postHandle(HttpServletRequest request, HttpServletResponse response, Object handler, ModelAndView modelAndView) throws Exception {

    }

    @Override
    public void afterCompletion(HttpServletRequest request, HttpServletResponse response, Object handler, Exception ex) throws Exception {
        if (!(handler instanceof HandlerMethod)) {
            return;
        }
        HandlerMethod handlerMethod = (HandlerMethod) handler;
        Method method = handlerMethod.getMethod();
        AutoLog methodAnnotation = method.getAnnotation(AutoLog.class);
        if (methodAnnotation != null) {
            RequestLogVo requestLogVo = RequestLogVo.builder()
                    .reqParam(RequestHandleUtil.getReqParam(request))
                    .uri(request.getRequestURI())
                    .methodName(method.getName())
                    .requestService(handlerMethod.getBeanType())
                    .requestVO(methodAnnotation.value())
                    .build();

            threadPool.submit(() -> {
                automaticLoggingService.insertLog(requestLogVo);
            });
        }
    }

}

实体类:

ExtEntity(table2):
public static final int EXT_COLUMN_SIZE = 30;
    @TableId(type = IdType.AUTO)
    private Long id;
    private String serviceName;
    private String ext0;
    private String ext1;
    private String ext2;
    private String ext3;
    private String ext4;
    private String ext5;
    private String ext6;
    private String ext7;
    private String ext8;
    private String ext9;
    private String ext10;
    private String ext11;
    private String ext12;
    private String ext13;
    private String ext14;
    private String ext15;
    private String ext16;
    private String ext17;
    private String ext18;
    private String ext19;
    private String ext20;

 

2.字典表:

 @TableId(type = IdType.AUTO)
    private Long id;

    private String uri;
    /**
     * 唯一,service接口的全限定名
     */
    private String serviceName;

    private String columnName;

    private String extColumnName;

    public FieldDictEntity(String uri, String serviceName, String columnName, String extColumnName) {
        this.uri = uri;
        this.serviceName = serviceName;
        this.columnName = columnName;
        this.extColumnName = extColumnName;
    }

拦截请求后封装类:

@Data
@AllArgsConstructor
@NoArgsConstructor
@Builder
@ToString
public class RequestLogVo {

    private String uri;

    private String methodName;
    private Class<?> requestService;
    private Class<?>[] requestVO;
    private Map<String, String> reqParam;
}

两个注解类:

@Target({ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
public @interface AutoLog {
    Class<?>[] value();
}

@Target({ElementType.FIELD})
@Retention(RetentionPolicy.RUNTIME)
public @interface AutoLogField {
}
WebMvcConfigurationSupport配置:
@Override
public void addInterceptors(InterceptorRegistry registry) {
    registry.addInterceptor(AutoLogInterceptor);
    registry.addInterceptor(autoIdempotentInterceptor);
    super.addInterceptors(registry);
}
核心业务处理类:
AutomaticLoggingServiceImpl
package com.ddmc.das.utils.businesslog.service.impl;

import lombok.extern.slf4j.Slf4j;
import org.apache.commons.collections4.CollectionUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import org.springframework.util.Assert;

import java.lang.reflect.Field;
import java.util.ArrayList;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;

/**
 * @author yuJie
 * @since 2020/5/25
 */
@Slf4j
@Service
public class AutomaticLoggingServiceImpl implements AutomaticLoggingService {

    private static final String AUTO_LOG_PREX = "**:";
    @Autowired
    public ExtMapper extMapper;

    @Autowired
    public MasterLogMapper masterLogMapper;

    @Autowired
    public FieldDictServiceImpl fieldDictServiceImpl;
    @Autowired
    private RedisTemplate<String, Object> solarRedisTemplate;

    @Override
    @SysLogPoint
    @Transactional(rollbackFor = Exception.class)
    public void insertLog(RequestLogVo requestLogVo) {
        // TODO 增加缓存
         if (!solarRedisTemplate.hasKey(AUTO_LOG_PREX + getMethodName(requestLogVo))) {
        initLog(requestLogVo);
        }
        //入库
        insertExtLog(requestLogVo);
    }

    private void insertExtLog(RequestLogVo requestLogVo) {
        try {
            QueryWrapper<FieldDictEntity> Wrapper = new QueryWrapper();
            Wrapper.eq("service_name", getMethodName(requestLogVo));
            List<FieldDictEntity> fieldDictList = fieldDictServiceImpl.list(Wrapper);
            Assert.notEmpty(fieldDictList, "自定记录日志失败字典表为空:" + requestLogVo.getMethodName());
            Map<String, String> columnMap = fieldDictList.stream().collect(Collectors.toMap(FieldDictEntity::getColumnName, FieldDictEntity::getExtColumnName));
            ExtEntity extEntity = generateExtValue(columnMap, requestLogVo.getReqParam());
            extEntity.setServiceName(getMethodName(requestLogVo));
            extMapper.insert(extEntity);
        } catch (Exception e) {
            log.error("插入ExtLog失败:{}", e);
        }
    }

    /**
     * columnMap:{"columnName":"extColumnName"}
     * valueMap:{"columnName":value}
     * 设置ext表的扩展属性
     *
     * @param columnMap
     * @param valueMap
     * @return
     */
    private final ExtEntity generateExtValue(final Map<String, String> columnMap, final Map<String, String> valueMap) {
        try {
            Class clazz = ExtEntity.class;
            Field[] fields = clazz.getDeclaredFields();
            ExtEntity extEntity = (ExtEntity) clazz.newInstance();
            for (Map.Entry<String, String> entry : valueMap.entrySet()) {
                String extColumnName = columnMap.get(entry.getKey());
                for (Field f : fields) {
                    f.setAccessible(true);
                    if (f.getName().equals(extColumnName)) {
                        f.set(extEntity, entry.getValue());
                    }
                }
            }
            return extEntity;
        } catch (Exception e) {
            log.error("生成自动日志扩展表失败:{}", e);
        }
        return null;
    }

    /**
     * 根据访问接口的数据初始化日志主表和字典表
     *
     * @param requestLogVo
     */
    private void initLog(RequestLogVo requestLogVo) {
        try {
            QueryWrapper<FieldDictEntity> Wrapper = new QueryWrapper();
            Wrapper.eq("service_name", getMethodName(requestLogVo));
            List<FieldDictEntity> fieldDictList = fieldDictServiceImpl.list(Wrapper);
            if (CollectionUtils.isEmpty(fieldDictList)) {
                List<String> columns = new LinkedList<>();
                for (Class<?> cla : requestLogVo.getRequestVO()) {
                    generateField(cla, columns);
                }
                List<FieldDictEntity> dictEntityList = ConvertToFieldDictList(columns, requestLogVo);
                BusinessAssert.OBJECT_NOT_NULL.assertNotEmpty(dictEntityList);
                fieldDictServiceImpl.saveBatch(dictEntityList);
            }
            solarRedisTemplate.opsForValue().setIfAbsent(AUTO_LOG_PREX + getMethodName(requestLogVo), "exist");
        } catch (Exception e) {
            log.error("初始化日志字典表失败:{}", e);
        }
    }

    private final List<FieldDictEntity> ConvertToFieldDictList(List<String> columns, RequestLogVo requestLogVo) {
        int length = Math.min(columns.size(), ExtEntity.EXT_COLUMN_SIZE);
        List<FieldDictEntity> list = new ArrayList<>();
        for (int i = 0; i < length; i++) {
            list.add(new FieldDictEntity(requestLogVo.getUri(), getMethodName(requestLogVo), columns.get(i), "ext" + i));
        }
        return list;
    }

    private static String getMethodName( RequestLogVo requestLogVo){
        return requestLogVo.getRequestService().getName() + "." + requestLogVo.getMethodName();
    }

    /***
     * @param vo
     * @param columnMap
     */
    private void generateField(Class vo, List<String> columnMap) {
        Field[] fs = vo.getDeclaredFields();
        for (Field f : fs) {
            //PUBLIC: 1 PRIVATE: 2 PROTECTED: 4
            if (f.getModifiers() == 1 || f.getModifiers() == 2 || f.getModifiers() == 4) {
                AutoLogField autoLogField = (AutoLogField) f.getAnnotation(AutoLogField.class);
                if (autoLogField != null) {
                    columnMap.add(f.getName());
                }
            }
        }
    }
}
RequestHandleUtil:
package com.ddmc.utils.http;


import com.alibaba.fastjson.JSON;
import org.apache.commons.lang3.StringUtils;

import javax.servlet.http.HttpServletRequest;
import java.io.*;
import java.net.URLDecoder;
import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.Map;

/**
 * 类作用描述
 *
 * @author yuJie
 * @since 2020/5/18
 */
public class RequestHandleUtil {

    public static final String METHOD_POST = "POST";
    public static final String METHOD_GET = "GET";
    public static final String CONTENT_TYPE_JSON = "application/json";

    /**
     * 获取请求参数
     *
     * @param req
     * @return 请求参数格式key-value
     */
    public static String getReqParam(HttpServletRequest req, String key) {
        String method = req.getMethod();
        Map<String, String> reqMap = new HashMap<String, String>();
        if (METHOD_GET.equals(method)) {
            reqMap = doGet(req);
        } else if (METHOD_POST.equals(method)) {
            reqMap = doPost(req);
        } else {
            //其他请求方式暂不处理
            return "";
        }
        return reqMap.get(key);
    }

    /**
     * 获取请求参数
     *
     * @param req
     * @return 请求参数格式key-value
     */
    public static Map<String, String> getReqParam(HttpServletRequest req) {
        String method = req.getMethod();
        Map<String, String> reqMap = new HashMap<String, String>();
        if (METHOD_GET.equals(method)) {
            reqMap = doGet(req);
        } else if (METHOD_POST.equals(method)) {
            reqMap = doPost(req);
        } else {
            //其他请求方式暂不处理
            return new HashMap<>();
        }
        return reqMap;
    }

    private static Map<String, String> doGet(HttpServletRequest req) {
        String param = req.getQueryString();
        return getParamsToMap(param);
    }

    private static Map<String, String> doPost(HttpServletRequest request) {
        Map<String, String> map = new HashMap<>();
        if (request instanceof BodyReaderHttpServletRequestWrapper) {
            map = (Map) JSON.parse(((BodyReaderHttpServletRequestWrapper) request).getBodyStr());
        }
        return map;
    }

    public static Map<String, String> getParamsToMap(String params) {
        Map<String, String> map = new LinkedHashMap<>();
        if (StringUtils.isNotBlank(params)) {
            String[] array = params.split("&");
            for (String pair : array) {
                if ("=".equals(pair.trim())) {
                    continue;
                }
                String[] entity = pair.split("=");
                if (entity.length == 1) {
                    map.put(decode(entity[0]), null);
                } else {
                    map.put(decode(entity[0]), decode(entity[1]));
                }
            }
        }
        return map;
    }

    public static Map<String, String> jsonParamsToMap(String params) {
        Map<String, String> map = new LinkedHashMap<>();
        if (StringUtils.isNotBlank(params)) {
            String[] array = params.split(",");
            for (String pair : array) {
                String[] entity = pair.split(":");
                if (entity.length == 1) {
                    map.put(decode(entity[0]), null);
                } else {
                    map.put(decode(entity[0]), decode(entity[1]));
                }
            }
        }
        return map;
    }

    /**
     * 编码格式转换
     *
     * @param content
     * @return
     */
    public static String decode(String content) {
        try {
            return URLDecoder.decode(content, "UTF-8");
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        }
        return "";
    }
}

另外还需要解析post请求的body流

BodyReaderHttpServletRequestWrapper
package com.ddmc.utils.http;

import javax.servlet.ReadListener;
import javax.servlet.ServletInputStream;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletRequestWrapper;
import java.io.*;
import java.nio.charset.Charset;

/**
 * 类作用描述
 *
 * @author yuJie
 * @since 2020/5/18
 */
public class BodyReaderHttpServletRequestWrapper extends HttpServletRequestWrapper {
    private final byte[] body;
    private String bodyStr;

    public BodyReaderHttpServletRequestWrapper(HttpServletRequest request) throws IOException {
        super(request);
        String bodyString = getBodyString(request);
        body = bodyString.getBytes(Charset.forName("UTF-8"));
        bodyStr = bodyString;
    }

    public String getBodyStr() {
        return bodyStr;
    }

    @Override
    public ServletInputStream getInputStream() throws IOException {
        final ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(body);

        return new ServletInputStream() {
            @Override
            public int read() throws IOException {
                return byteArrayInputStream.read();
            }

            @Override
            public boolean isFinished() {
                return false;
            }

            @Override
            public boolean isReady() {
                return false;
            }

            @Override
            public void setReadListener(ReadListener readListener) {
            }
        };
    }


    public String getBodyString(HttpServletRequest request) throws IOException {
        StringBuilder sb = new StringBuilder();
        InputStream inputStream = null;
        BufferedReader reader = null;
        try {
            inputStream = request.getInputStream();
            reader = new BufferedReader(
                    new InputStreamReader(inputStream, Charset.forName("UTF-8")));

            char[] bodyCharBuffer = new char[1024];
            int len = 0;
            while ((len = reader.read(bodyCharBuffer)) != -1) {
                sb.append(new String(bodyCharBuffer, 0, len));
            }
        } catch (IOException e) {
            e.printStackTrace();
        } finally {
            if (inputStream != null) {
                try {
                    inputStream.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
            if (reader != null) {
                try {
                    reader.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
        return sb.toString();
    }
}

表:

CREATE TABLE `table1` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT COMMENT '主键',
  `tenant_id` bigint(20) NOT NULL DEFAULT '0' COMMENT '租户号',
  `uri` varchar(200) NOT NULL DEFAULT '' COMMENT '请求地址',
  `service_name` varchar(100) NOT NULL DEFAULT '' COMMENT '接口名',
  `column_name` varchar(20) DEFAULT NULL COMMENT '列名',
  `ext_column_name` varchar(20) DEFAULT NULL COMMENT '扩展表的列名',
  `create_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=21 DEFAULT CHARSET=utf8mb4 COMMENT='table1'

CREATE TABLE `table2` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT COMMENT '主键',
  `tenant_id` bigint(20) NOT NULL DEFAULT '0' COMMENT '租户号',
  `service_name` varchar(200) NOT NULL DEFAULT '' COMMENT '接口名',
`ext0` varchar(200) NOT NULL DEFAULT '' COMMENT '扩展字段',
  `ext1` varchar(200) NOT NULL DEFAULT '' COMMENT '扩展字段',
 `ext2` varchar(200) NOT NULL DEFAULT '' COMMENT '扩展字段',
 `ext3` varchar(200) NOT NULL DEFAULT '' COMMENT '扩展字段',
 `ext4` varchar(200) NOT NULL DEFAULT '' COMMENT '扩展字段',
 `ext5` varchar(200) NOT NULL DEFAULT '' COMMENT '扩展字段',
 `ext6` varchar(200) NOT NULL DEFAULT '' COMMENT '扩展字段',
 `ext7` varchar(200) NOT NULL DEFAULT '' COMMENT '扩展字段',
 `ext8` varchar(200) NOT NULL DEFAULT '' COMMENT '扩展字段',
 `ext9` varchar(200) NOT NULL DEFAULT '' COMMENT '扩展字段',
 `ext10` varchar(200) NOT NULL DEFAULT '' COMMENT '扩展字段',
 `ext11` varchar(200) NOT NULL DEFAULT '' COMMENT '扩展字段',
 `ext12` varchar(200) NOT NULL DEFAULT '' COMMENT '扩展字段',
 `ext13` varchar(200) NOT NULL DEFAULT '' COMMENT '扩展字段',
 `ext14` varchar(200) NOT NULL DEFAULT '' COMMENT '扩展字段',
 `ext15` varchar(200) NOT NULL DEFAULT '' COMMENT '扩展字段',
 `ext16` varchar(200) NOT NULL DEFAULT '' COMMENT '扩展字段',
 `ext17` varchar(200) NOT NULL DEFAULT '' COMMENT '扩展字段',
 `ext18` varchar(200) NOT NULL DEFAULT '' COMMENT '扩展字段',
 `ext19` varchar(200) NOT NULL DEFAULT '' COMMENT '扩展字段',
 `ext20` varchar(200) NOT NULL DEFAULT '' COMMENT '扩展字段',
  `create_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=21 DEFAULT CHARSET=utf8mb4 COMMENT='table2'