欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页  >  IT编程

大数据量下DataTable To List效率对比

程序员文章站 2022-04-08 21:17:24
使用反射和动态生成代码两种方式(Reflect和Emit) 反射将DataTable转为List方法 1 public static List ToListByReflect(this DataTable dt) where T : new() 2 { 3 List ts = ne ......

使用反射和动态生成代码两种方式(reflect和emit)

反射将datatable转为list方法

大数据量下DataTable To List效率对比
 1 public static list<t> tolistbyreflect<t>(this datatable dt) where t : new()
 2         {
 3             list<t> ts = new list<t>();
 4             string tempname = string.empty;
 5             foreach (datarow dr in dt.rows)
 6             {
 7                 t t = new t();
 8                 propertyinfo[] propertys = t.gettype().getproperties();
 9                 foreach (propertyinfo pi in propertys)
10                 {
11                     tempname = pi.name;
12                     if (dt.columns.contains(tempname))
13                     {
14                         object value = dr[tempname];
15                         if (value != dbnull.value)
16                         {
17                             pi.setvalue(t, value, null);
18                         }
19                     }
20                 }
21                 ts.add(t);
22             }
23             return ts;
24         }
view code

动态生成代码将datatable转为list方法

大数据量下DataTable To List效率对比
 1 public static list<t> tolistbyemit<t>(this datatable dt) where t : class, new()
 2         {
 3             list<t> list = new list<t>();
 4             if (dt == null || dt.rows.count == 0)
 5                 return list;
 6             datatableentitybuilder<t> eblist = datatableentitybuilder<t>.createbuilder(dt.rows[0]);
 7             foreach (datarow info in dt.rows)
 8                 list.add(eblist.build(info));
 9             dt.dispose();
10             dt = null;
11             return list;
12         }
13         public class datatableentitybuilder<entity>
14         {
15             private static readonly methodinfo getvaluemethod = typeof(datarow).getmethod("get_item", new type[] { typeof(int) });
16             private static readonly methodinfo isdbnullmethod = typeof(datarow).getmethod("isnull", new type[] { typeof(int) });
17             private delegate entity load(datarow datarecord);
18             private load handler;
19             private datatableentitybuilder() { }
20             public entity build(datarow datarecord)
21             {
22                 return handler(datarecord);
23             }
24             public static datatableentitybuilder<entity> createbuilder(datarow datarecord)
25             {
26                 datatableentitybuilder<entity> dynamicbuilder = new datatableentitybuilder<entity>();
27                 dynamicmethod method = new dynamicmethod("dynamiccreateentity", typeof(entity), new type[] { typeof(datarow) }, typeof(entity), true);
28                 ilgenerator generator = method.getilgenerator();
29                 localbuilder result = generator.declarelocal(typeof(entity));
30                 generator.emit(opcodes.newobj, typeof(entity).getconstructor(type.emptytypes));
31                 generator.emit(opcodes.stloc, result);
32                 for (int i = 0; i < datarecord.itemarray.length; i++)
33                 {
34                     propertyinfo propertyinfo = typeof(entity).getproperty(datarecord.table.columns[i].columnname);
35                     label endiflabel = generator.definelabel();
36                     if (propertyinfo != null && propertyinfo.getsetmethod() != null)
37                     {
38                         generator.emit(opcodes.ldarg_0);
39                         generator.emit(opcodes.ldc_i4, i);
40                         generator.emit(opcodes.callvirt, isdbnullmethod);
41                         generator.emit(opcodes.brtrue, endiflabel);
42                         generator.emit(opcodes.ldloc, result);
43                         generator.emit(opcodes.ldarg_0);
44                         generator.emit(opcodes.ldc_i4, i);
45                         generator.emit(opcodes.callvirt, getvaluemethod);
46                         generator.emit(opcodes.unbox_any, propertyinfo.propertytype);
47                         generator.emit(opcodes.callvirt, propertyinfo.getsetmethod());
48                         generator.marklabel(endiflabel);
49                     }
50                 }
51                 generator.emit(opcodes.ldloc, result);
52                 generator.emit(opcodes.ret);
53                 dynamicbuilder.handler = (load)method.createdelegate(typeof(load));
54                 return dynamicbuilder;
55             }
56         }
view code

然后写个控制台程序,对比一下两个方法的效率(测试类大概有40个属性)

大数据量下DataTable To List效率对比

电脑比较渣,使用emit方法转换100w条数据大概需要7秒,而反射则需要37秒。还测试了当数据量比较小时,reflect反而比较快。