欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页  >  IT编程

Springboot线程池并发处理数据优化方式

程序员文章站 2022-03-09 23:01:09
目录第一步:首先配置线程基本参数第二步:让spring boot加载第三步:创建一个service接口第四步:编写现实类第五步:测试结果如下第一步:首先配置线程基本参数可以放在application....

第一步:首先配置线程基本参数

可以放在application.propertes文件种也可以放在自己新建的config/文件目录下,注意:但是需要使用@propertysource把配置文件进行加载。

# 异步线程配置
# 配置核心线程数
async.executor.thread.core_pool_size = 8
# 配置最大线程数
async.executor.thread.max_pool_size = 20
# 配置队列大小
async.executor.thread.queue_capacity = 99999
# 配置线程池中的线程的名称前缀
async.executor.thread.name.prefix = async-service-

第二步:让spring boot加载

用来定义如何创建一个threadpooltaskexecutor,要使用@configuration和@enableasync这两个注解,表示这是个配置类,并且是线程池的配置类

@slf4j
@enableasync
@configuration
public class rcexecutorconfig {
 
  @value("${async.executor.thread.core_pool_size}")
     private int corepoolsize;
     @value("${async.executor.thread.max_pool_size}")
     private int maxpoolsize;
     @value("${async.executor.thread.queue_capacity}")
     private int queuecapacity;
     @value("${async.executor.thread.name.prefix}")
     private string nameprefix;
    
     @bean(name = "asyncserviceexecutor")
     public executor asyncserviceexecutor() {
         log.info("start asyncserviceexecutor");
         threadpooltaskexecutor executor = new threadpooltaskexecutor();
         //threadpooltaskexecutor executor = new visiablethreadpooltaskexecutor();
         //配置核心线程数
         executor.setcorepoolsize(corepoolsize);
         //配置最大线程数
         executor.setmaxpoolsize(maxpoolsize);
         //配置队列大小
         executor.setqueuecapacity(queuecapacity);
         //配置线程池中的线程的名称前缀
         executor.setthreadnameprefix(nameprefix);
 
         // rejection-policy:当pool已经达到max size的时候,如何处理新任务
         // caller_runs:不在新线程中执行任务,而是有调用者所在的线程来执行
         executor.setrejectedexecutionhandler(new threadpoolexecutor.callerrunspolicy());
         //执行初始化
         executor.initialize();
         return executor;
     }
}

第三步:创建一个service接口

是异步线程的接口,便于测试

public interface asyncservice {
 
 /**
     *  执行异步任务
     *  可以根据需求,自己加参数拟定
     */
    void executeasync();
}

第四步:编写现实类

将service层的服务异步化,在executeasync()方法上增加注解@async("asyncserviceexecutor"),asyncserviceexecutor方法是前面rcexecutorconfig.java中的方法名,表明executeasync方法进入的线程池是asyncserviceexecutor方法创建的。

测试方面这里我加入了一个定时任务,使用的是corn表达式。(不懂得同学可以网上了解一下)

@slf4j
@service
public class asyncserviceimpl implements asyncservice {
 
 @override
 @scheduled(cron = " */1 * * * * ? ")
 @async("asyncserviceexecutor")
 public void executeasync() {
  log.info("start executeasync");
        system.out.println("异步线程执行批量插入等耗时任务");
        log.info("end executeasync");  
 }  
}

第五步:测试结果如下

10:32:15.004 [async-service-1] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:32:15.004 [async-service-1] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:32:16.003 [async-service-2] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:32:16.004 [async-service-2] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:32:17.001 [async-service-3] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:32:17.001 [async-service-3] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:32:18.002 [async-service-4] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:32:18.003 [async-service-4] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:32:19.002 [async-service-5] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:32:19.003 [async-service-5] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:32:20.001 [async-service-6] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:32:20.002 [async-service-6] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:32:21.002 [async-service-7] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:32:21.002 [async-service-7] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:32:22.004 [async-service-8] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:32:22.005 [async-service-8] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:32:23.001 [async-service-1] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:32:23.003 [async-service-1] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:32:24.003 [async-service-2] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:32:24.003 [async-service-2] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:32:25.001 [async-service-3] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:32:25.001 [async-service-3] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:32:26.002 [async-service-4] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:32:26.002 [async-service-4] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:32:27.002 [async-service-5] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:32:27.003 [async-service-5] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:32:28.001 [async-service-6] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:32:28.001 [async-service-6] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:32:29.001 [async-service-7] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:32:29.002 [async-service-7] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:32:30.001 [async-service-8] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:32:30.001 [async-service-8] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:32:31.001 [async-service-1] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:32:31.001 [async-service-1] info c.a.a.service.impl.asyncserviceimpl - end executeasync

还没完:

通过以上日志可以发现,[async-service-]是有多个线程的,显然已经在我们配置的线程池中执行了,表明每次请求都快速响应了,而耗时的操作都留给线程池中的线程去异步执行;

还有另一个问提就是,虽然已经用上了线程池,但是依然不清楚线程池当时的情况,有多少线程在执行,多少在队列中等待呢?于是这里我创建了一个threadpooltaskexecutor的子类,可以把每次提交线程的时候都会将当前线程池的运行状况打印出来

import lombok.extern.slf4j.slf4j;
 
@slf4j
public class visiablethreadpooltaskexecutor extends  threadpooltaskexecutor {
 
    /**
  * 
  */
 private static final long serialversionuid = -3518460523928455463l; 
 private void showthreadpoolinfo(string prefix) {
        threadpoolexecutor threadpoolexecutor = getthreadpoolexecutor(); 
        if (null == threadpoolexecutor) {
            return;
        }
 
        log.info("{}, {},taskcount [{}], completedtaskcount [{}], activecount [{}], queuesize [{}]",
                this.getthreadnameprefix(),
                prefix,
                threadpoolexecutor.gettaskcount(),
                threadpoolexecutor.getcompletedtaskcount(),
                threadpoolexecutor.getactivecount(),
                threadpoolexecutor.getqueue().size());
    }
 
    @override
    public void execute(runnable task) {
        showthreadpoolinfo("1. do execute");
        super.execute(task);
    }
 
    @override
    public void execute(runnable task, long starttimeout) {
        showthreadpoolinfo("2. do execute");
        super.execute(task, starttimeout);
    }
 
    @override
    public future<?> submit(runnable task) {
        showthreadpoolinfo("1. do submit");
        return super.submit(task);
    }
 
    @override
    public <t> future<t> submit(callable<t> task) {
        showthreadpoolinfo("2. do submit");
        return super.submit(task);
    }
 
    @override
    public listenablefuture<?> submitlistenable(runnable task) {
        showthreadpoolinfo("1. do submitlistenable");
        return super.submitlistenable(task);
    }
 
    @override
    public <t> listenablefuture<t> submitlistenable(callable<t> task) {
        showthreadpoolinfo("2. do submitlistenable");
        return super.submitlistenable(task);
    } 
}

其次:修改rcexecutorconfig.java的asyncserviceexecutor方法,

将threadpooltaskexecutor executor = new threadpooltaskexecutor()改为threadpooltaskexecutor executor = new visiablethreadpooltaskexecutor()

 @bean(name = "asyncserviceexecutor")
     public executor asyncserviceexecutor() {
         log.info("start asyncserviceexecutor");
         //threadpooltaskexecutor executor = new threadpooltaskexecutor();
         threadpooltaskexecutor executor = new visiablethreadpooltaskexecutor();
         //配置核心线程数
         executor.setcorepoolsize(corepoolsize);
         //配置最大线程数
         executor.setmaxpoolsize(maxpoolsize);
         //配置队列大小
         executor.setqueuecapacity(queuecapacity);
         //配置线程池中的线程的名称前缀
         executor.setthreadnameprefix(nameprefix); 
         // rejection-policy:当pool已经达到max size的时候,如何处理新任务
         // caller_runs:不在新线程中执行任务,而是有调用者所在的线程来执行
         executor.setrejectedexecutionhandler(new threadpoolexecutor.callerrunspolicy());
         //执行初始化
         executor.initialize();
         return executor;
     }

测试结果如下:

异步线程执行批量插入等耗时任务
10:41:35.003 [async-service-5] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:41:36.001 [sheduled-pool-1-thread-1] info c.a.a.e.t.visiablethreadpooltaskexecutor - async-service-, 2. do submit,taskcount [5], completedtaskcount [5], activecount [0], queuesize [0]
10:41:36.001 [async-service-6] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:41:36.002 [async-service-6] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:41:37.001 [sheduled-pool-1-thread-7] info c.a.a.e.t.visiablethreadpooltaskexecutor - async-service-, 2. do submit,taskcount [6], completedtaskcount [6], activecount [0], queuesize [0]
10:41:37.001 [async-service-7] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:41:37.002 [async-service-7] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:41:38.002 [sheduled-pool-1-thread-7] info c.a.a.e.t.visiablethreadpooltaskexecutor - async-service-, 2. do submit,taskcount [7], completedtaskcount [7], activecount [0], queuesize [0]
10:41:38.002 [async-service-8] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:41:38.002 [async-service-8] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:41:39.001 [sheduled-pool-1-thread-7] info c.a.a.e.t.visiablethreadpooltaskexecutor - async-service-, 2. do submit,taskcount [8], completedtaskcount [8], activecount [0], queuesize [0]
10:41:39.001 [async-service-1] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:41:39.002 [async-service-1] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:41:40.003 [sheduled-pool-1-thread-7] info c.a.a.e.t.visiablethreadpooltaskexecutor - async-service-, 2. do submit,taskcount [9], completedtaskcount [9], activecount [0], queuesize [0]
10:41:40.003 [async-service-2] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:41:40.003 [async-service-2] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:41:41.001 [sheduled-pool-1-thread-7] info c.a.a.e.t.visiablethreadpooltaskexecutor - async-service-, 2. do submit,taskcount [10], completedtaskcount [10], activecount [0], queuesize [0]
10:41:41.001 [async-service-3] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:41:41.001 [async-service-3] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:41:42.000 [sheduled-pool-1-thread-7] info c.a.a.e.t.visiablethreadpooltaskexecutor - async-service-, 2. do submit,taskcount [11], completedtaskcount [11], activecount [0], queuesize [0]
10:41:42.000 [async-service-4] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:41:42.000 [async-service-4] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:41:43.001 [sheduled-pool-1-thread-7] info c.a.a.e.t.visiablethreadpooltaskexecutor - async-service-, 2. do submit,taskcount [12], completedtaskcount [12], activecount [0], queuesize [0]
10:41:43.002 [async-service-5] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:41:43.003 [async-service-5] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:41:44.001 [sheduled-pool-1-thread-7] info c.a.a.e.t.visiablethreadpooltaskexecutor - async-service-, 2. do submit,taskcount [13], completedtaskcount [13], activecount [0], queuesize [0]
10:41:44.001 [async-service-6] info c.a.a.service.impl.asyncserviceimpl - start executeasync
异步线程执行批量插入等耗时任务
10:41:44.001 [async-service-6] info c.a.a.service.impl.asyncserviceimpl - end executeasync
10:41:45.000 [sheduled-pool-1-thread-7] info c.a.a.e.t.visiablethreadpooltaskexecutor - async-service-, 2. do submit,taskcount [14], completedtaskcount [14], activecount [0], queuesize [0]
10:41:45.001 [async-service-7] info c.a.a.service.impl.asyncserviceimpl - start executeasync

解释:这里意思提交了14个任务,处理了14个任务,对列中还剩0个任务

10:41:45.000 [sheduled-pool-1-thread-7] info c.a.a.e.t.visiablethreadpooltaskexecutor - async-service-, 2. do submit,taskcount [14], completedtaskcount [14], activecount [0], queuesize [0]

到此为止就ok了!

以上为个人经验,希望能给大家一个参考,也希望大家多多支持。