.Net性能测试框架Crank的使用方法

crank 是微软新出的一个性能测试框架，集成了多种基准测试工具，如bombardier、wrk等。

crank通过统一的配置，可以转换成不同基准测试工具命令进行测试。可参考bombardier job实现。

安装crank

运行如下两个命令分别安装crank的cli(controller)和agent。

dotnet tool update microsoft.crank.controller --version "0.2.0-*" --global
dotnet tool update microsoft.crank.agent--version "0.2.0-*" --global

需要.net sdk 5.0环境

安装完成后执行命令crank，会打印出如下可以配置的参数和介绍，或者github查看相关参数介绍。

ps c:\users\stack\desktop> crank
crank benchmarks controller

the crank controller orchestrates benchmark jobs on crank agents.

usage: crank [command] [options]

options:
  -?|-h|--help         show help information
  -c|--config          configuration file or url
  -s|--scenario        scenario to execute
  -j|--job             name of job to define
  --profile            profile name
  --script             execute a named script available in the configuration files. can be used multiple times.
  -j|--json            saves the results as json in the specified file.
  --csv                saves the results as csv in the specified file.
  --compare            an optional filename to compare the results to. can be used multiple times.
  --variable           variable
  --sql                connection string of the sql server database to store results in
  --table              table name of the sql database to store results in
  --session            a logical identifier to group related jobs.
  --description        a string describing the job.
  -p|--property        some custom key/value that will be added to the results, .e.g. --property arch=arm --property
                       os=linux

执行crank-agent，启动基准测试所需的代理服务。github查看相关参数介绍。

ps c:\users\stack\desktop> crank-agent
hosting environment: production
content root path: c:\users\stack\.dotnet\tools\.store\microsoft.crank.agent\0.2.0-alpha.21567.1\microsoft.crank.agent\0.2.0-alpha.21567.1\tools\net5.0\any\
now listening on: http://[::]:5010

创建crank配置文件

配置文件参考官方hello.benchmarks.yml示例

示例文件中引入了bombardier.yml，由于大环境对githubusercontent.com域名不太友好，可以考虑将bombardier.yml下载到本地，imports引入本地路径或者直接将文件内容加入到新建的配置文件。

imports:
  -https://raw.githubusercontent.com/dotnet/crank/main/src/microsoft.crank.jobs.bombardier/bombardier.yml

生产中使用推荐imports文件的方式，crank配置文件中做少量配置就可以完成基准测试，并且可以引入不同的microsoft.crank.jobs.xxx/xxx.yml,基于不同的基准测试工具进行测试。

其他测试工具配置文件都在https://github.com/dotnet/crank/blob/main/src/microsoft.crank.xxx下。

variables:
  headers:
    none: ''
    plaintext: '--header "accept: text/plain,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7" --header "connection: keep-alive"'
    html: '--header "accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" --header "connection: keep-alive"'
    json: '--header "accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7" --header "connection: keep-alive"'
    connectionclose: '--header "connection: close"'
  presetheaders: none

jobs:
  bombardier:
    source:
      repository: https://github.com/dotnet/crank.git
      branchorcommit: main
      project: src/microsoft.crank.jobs.bombardier/microsoft.crank.jobs.bombardier.csproj
      sourcekey: bombardier
      nobuild: true
    readystatetext: bombardier client
    waitforexit: true
    variables:
      connections: 256 #设置连接数
      warmup: 15 #设置测试预热次数
      duration: 15 #设置测试时间
      requests: 0 #设置测试请求实例数
      rate: 0 #设置每秒请求频率
      transport: fasthttp # | http1 | http2  设置使用golang的fasthttp库发送http请求
      serverscheme: http
      serveraddress: localhost
      serverport: 5000
      path: 
      bodyfile: # path or url for a file to use as the body content
      verb: # get when nothing is specified
      customheaders: [ ] # list of headers with the format: '<name1>: <value1>', e.g. [ 'content-type: application/json' ]
    arguments: "-c {{connections}} -w {{warmup}} -d {{duration}} -n {{requests}} --insecure -l {% if rate != 0 %} --rate {{ rate }} {% endif %} {% if transport %} --{{ transport}} {% endif %} {{headers[presetheaders]}} {% for h in customheaders %}{% assign s = h | split : ':' %}--header \"{{ s[0] }}: {{ s[1] | strip }}\" {% endfor %} {% if serveruri == blank or serveruri == empty %} {{serverscheme}}://{{serveraddress}}:{{serverport}}{{path}} {% else %} {{serveruri}}:{{serverport}}{{path}} {% endif %} {% if bodyfile != blank and bodyfile != empty %} -f {{bodyfile}} {% endif %}  {% if verb != blank and verb != empty %} -m {{verb}} {% endif %}"
    onconfigure: 
      # - job.timeout = number(job.variables.duration) + number(job.variables.warmup) + 10;

  server:
    source: #指定需要测试的项目，本文直接使用本地路径
      localfolder: .
      project: crank_demo.csproj
    readystatetext: application started.
    # source: 指定测试项目的远程仓库地址，并通过branchorcommit指定分支
    #   repository: https://github.com/dotnet/crank
    #   branchorcommit: main
    #   project: samples/hello/hello.csproj
    # readystatetext: application started.

scenarios: #配置基准测试场景
  crank_demo: #定义方案名，执行crank命令时指定该名称
    application: 
      job: server # 指定测试项目为上面定义的server
    load:
      job: bombardier # 指定测试工具bombardier
      variables:
        serverport: 5000 #配置http服务端口
        path: / #配置http服务地址

profiles:
  local:
    variables:
      serveraddress: localhost
    jobs: 
      application:
        endpoints: 
          - http://localhost:5010
      load:
        endpoints: 
          - http://localhost:5010

启动crank-agent

启动agent后执行crank，会有一个如下安装sdk的动作

[09:29:05.261] runtime: 6.0.0 (current)
[09:29:05.262] sdk: 6.0.100 (current)
[09:29:05.263] asp.net: 6.0.0 (current)
[09:29:05.265] creating custom global.json
[09:29:05.266] desktop: 6.0.0 (current)
[09:29:05.266] installing sdk '6.0.100' ...

所以启动agent时多指定一个dotnethome参数，避免重复安装sdk。

crank-agent --dotnethome 'c:\program files\dotnet'

启动crank

crank --config .\demo.benchmarks.yml --scenario crank_demo --profile local 

--scenario crank_demo: 指定定义的测试场景

--profile local :设置输出结果到本地，即控制台输出

可以通过参数指定结果输出到本地json文件（--output results.json ）或者数据库（--sql [connection-string] --table [table-name]）

结果输出

这里省略压测期间的日志输出，直接列出结果。

| application           |               |
| --------------------- | ------------- |
| cpu usage (%)         | 56            |
| cores usage (%)       | 447           |
| working set (mb)      | 140           |
| private memory (mb)   | 157           |
| build time (ms)       | 7,232         |
| start time (ms)       | 501           |
| published size (kb)   | 91,292        |
| .net core sdk version | 6.0.100       |
| asp.net core version  | 6.0.0+ae1a6cb |
| .net runtime version  | 6.0.0+4822e3c |


| load                  |                | 
| --------------------- | -------------- |
| cpu usage (%)         | 46             | 
| cores usage (%)       | 370            | 
| working set (mb)      | 29             |
| private memory (mb)   | 30             |
| build time (ms)       | 11,891         |
| start time (ms)       | 226            |
| published size (kb)   | 68,228         |
| .net core sdk version | 3.1.415        |
| asp.net core version  | 3.1.21+458d974 |
| .net runtime version  | 3.1.21+df8abc0 |
| first request (ms)    | 185            |
| requests              | 912,005        |
| bad responses         | 0              |
| mean latency (us)     | 4,207          |
| max latency (us)      | 138,999        |
| requests/sec          | 60,305         |
| requests/sec (max)    | 128,523        |

更多参考资料

crank readme

benchmarks crank为benchmarks重构版本

本文示例代码

以上就是本文的全部内容，希望对大家的学习有所帮助，也希望大家多多支持。