Java中LocalCache本地缓存实现代码
前言
本次分享探讨java平台的本地缓存,是指占用jvm的heap区域来缓冲存储数据的缓存组件。
一、本地缓存应用场景
localcache有着极大的性能优势:
1. 单机情况下适当使用localcache会使应用的性能得到很大的提升。
2. 集群环境下对于敏感性要求不高的数据可以使用localcache,只配置简单的失效机制来保证数据的相对一致性。
哪些数据可以存储到本地缓存?
1.访问频繁的数据;
2.静态基础数据(长时间内不变的数据);
3.相对静态数据(短时间内不变的数据)。
二、java本地缓存标准
java缓存新标准(javax.cache),这个标准由jsr107所提出,已经被包含在java ee 7中。
特性:
1.原子操作,跟java.util.concurrentmap类似
2.从缓存中读取
3.写入缓存
4.缓存事件监听器
5.数据统计
6.包含所有隔离(ioslation)级别的事务
7.缓存注解(annotations)
8.保存定义key和值类型的泛型缓存
9.引用保存(只适用于堆缓存)和值保存定义
但目前应用不是很普遍。
三、java开源缓存框架
比较有名的本地缓存开源框架有:
1.ehcache
ehcache是一个纯java的在进程中的缓存,它具有以下特性:快速,简单,为hibernate2.1充当可插入的缓存,最小的依赖性,全面的文档和测试。
bug: 过期失效的缓存元素无法被gc掉,时间越长缓存越多,内存占用越大,导致内存泄漏的概率越大。
2.oscache
oscache有以下特点:缓存任何对象,你可以不受限制的缓存部分jsp页面或http请求,任何java对象都可以缓存。拥有全面的api--oscache api给你全面的程序来控制所有的oscache特性。永久缓存--缓存能随意的写入硬盘,因此允许昂贵的创建(expensive-to-create)数据来保持缓存,甚至能让应用重启。支持集群--集群缓存数据能被单个的进行参数配置,不需要修改代码。缓存记录的过期--你可以有最大限度的控制缓存对象的过期,包括可插入式的刷新策略(如果默认性能不需要时)。
3.jcache
java缓存新标准(javax.cache)
4.cache4j
cache4j是一个有简单api与实现快速的java对象缓存。它的特性包括:在内存中进行缓存,设计用于多线程环境,两种实现:同步与阻塞,多种缓存清除策略:lfu, lru, fifo,可使用强引用。
5.shiftone
shiftone java object cache是一个执行一系列严格的对象缓存策略的java lib,就像一个轻量级的配置缓存工作状态的框架。
6.whirlycache
whirlycache是一个快速的、可配置的、存在于内存中的对象的缓存。
四、localcache实现
1、localcache简介
localcache是一个精简版本地缓存组件,有以下特点:
1. 有容量上限maxcapacity;
2. 缓存达到容量上限时基于lru策略来移除缓存元素;
3. 缓存对象的生命周期(缓存失效时间)由调用方决定;
4. 缓存对象失效后,将会有定时清理线程来清理掉,不会导致内存泄漏。
5. 性能比ehcache稍强。
2、总体设计
localcache总体设计:
1. 缓存元素 cacheelement;
2. 缓存容器 lrulinkedhashmap;
3. 缓存接口 cache;
4. 缓存组件实现 localcache。
3、详细设计
1. cacheelement设计
/** * 缓存元素 * */ public class cacheelement { private object key; private object value; private long createtime; private long lifetime; private int hitcount; public cacheelement() { } public cacheelement(object key ,object value) { this.key = key; this.value = value; this.createtime = system.currenttimemillis(); } public object getkey() { return key; } public void setkey(object key) { this.key = key; } public object getvalue() { hitcount++; return value; } public void setvalue(object value) { this.value = value; } public long getcreatetime() { return createtime; } public void setcreatetime(long createtime) { this.createtime = createtime; } public int gethitcount() { return hitcount; } public void sethitcount(int hitcount) { this.hitcount = hitcount; } public long getlifetime() { return lifetime; } public void setlifetime(long lifetime) { this.lifetime = lifetime; } public boolean isexpired() { boolean isexpired = system.currenttimemillis() - getcreatetime() > getlifetime(); return isexpired; } /* * (non-javadoc) * @see java.lang.object#tostring() */ public string tostring() { stringbuffer sb = new stringbuffer(); sb.append("[ key=").append(key).append(", isexpired=").append(isexpired()) .append(", lifetime=").append(lifetime).append(", createtime=").append(createtime) .append(", hitcount=").append(hitcount) .append(", value=").append(value).append(" ]"); return sb.tostring(); } /* * (non-javadoc) * @see java.lang.object#hashcode() */ public final int hashcode(){ if(null == key){ return "".hashcode(); } return this.key.hashcode(); } /* * (non-javadoc) * @see java.lang.object#equals(java.lang.object) */ public final boolean equals(object object) { if ((object == null) || (!(object instanceof cacheelement))) { return false; } cacheelement element = (cacheelement) object; if ((this.key == null) || (element.getkey() == null)) { return false; } return this.key.equals(element.getkey()); } }
2. lrulinkedhashmap实现
import java.util.linkedhashmap; import java.util.set; import java.util.concurrent.locks.lock; import java.util.concurrent.locks.reentrantlock; /** * 实现 lru策略的 linkedhashmap * * @param <k> * @param <v> */ public class lrulinkedhashmap<k, v> extends linkedhashmap<k, v> { protected static final long serialversionuid = 2828675280716975892l; protected static final int default_max_entries = 100; protected final int initialcapacity; protected final int maxcapacity; protected boolean enableremoveeldestentry = true;//是否允许自动移除比较旧的元素(添加元素时) protected static final float default_load_factor = 0.8f; protected final lock lock = new reentrantlock(); public lrulinkedhashmap(int initialcapacity) { this(initialcapacity, default_max_entries); } public lrulinkedhashmap(int initialcapacity ,int maxcapacity) { //set accessorder=true, lru super(initialcapacity, default_load_factor, true); this.initialcapacity = initialcapacity; this.maxcapacity = maxcapacity; } /* * (non-javadoc) * @see java.util.linkedhashmap#removeeldestentry(java.util.map.entry) */ protected boolean removeeldestentry(java.util.map.entry<k, v> eldest) { return enableremoveeldestentry && ( size() > maxcapacity ); } /* * (non-javadoc) * @see java.util.linkedhashmap#get(java.lang.object) */ public v get(object key) { try { lock.lock(); return super.get(key); } finally { lock.unlock(); } } /* * (non-javadoc) * @see java.util.hashmap#put(java.lang.object, java.lang.object) */ public v put(k key, v value) { try { lock.lock(); return super.put(key, value); } finally { lock.unlock(); } } /* * (non-javadoc) * @see java.util.hashmap#remove(java.lang.object) */ public v remove(object key) { try { lock.lock(); return super.remove(key); } finally { lock.unlock(); } } /* * (non-javadoc) * @see java.util.linkedhashmap#clear() */ public void clear() { try { lock.lock(); super.clear(); } finally { lock.unlock(); } } /* * (non-javadoc) * @see java.util.hashmap#keyset() */ public set<k> keyset() { try { lock.lock(); return super.keyset(); } finally { lock.unlock(); } } public boolean isenableremoveeldestentry() { return enableremoveeldestentry; } public void setenableremoveeldestentry(boolean enableremoveeldestentry) { this.enableremoveeldestentry = enableremoveeldestentry; } public int getinitialcapacity() { return initialcapacity; } public int getmaxcapacity() { return maxcapacity; } }
3. cache接口设计
/** * 缓存接口 * */ public interface cache { /** * 获取缓存 * @param key * @return */ public <t> t getcache(object key); /** * 缓存对象 * @param key * @param value * @param millisecond 缓存生命周期(毫秒) */ public void putcache(object key, object value ,long millisecond); /** * 缓存容器中是否包含 key * @param key * @return */ public boolean containskey(object key); /** * 缓存列表大小 * @return */ public int getsize(); /** * 是否启用缓存 */ public boolean isenabled(); /** * 启用 或 停止 * @param enable */ public void setenabled(boolean enabled); /** * 移除所有缓存 */ public void invalidatecaches(); /** * 移除 指定key缓存 * @param key */ public void invalidatecache(object key); }
4. localcache实现
import java.util.date; import java.util.iterator; import java.util.random; import org.slf4j.logger; import org.slf4j.loggerfactory; /** * 本地缓存组件 */ public class localcache implements cache{ private logger logger = loggerfactory.getlogger(this.getclass()); private lrulinkedhashmap<object, cacheelement> cachemap; protected boolean initflag = false;//初始化标识 protected final long defaultlifetime = 5 * 60 * 1000;//5分钟 protected boolean warnlongerlifetime = false; protected final int default_initial_capacity = 100; protected final int default_max_capacity = 100000; protected int initialcapacity = default_initial_capacity;//初始化缓存容量 protected int maxcapacity = default_max_capacity;//最大缓存容量 protected int timeout = 20;//存取缓存操作响应超时时间(毫秒数) private boolean enabled = true; private thread gcthread = null; private string lastgcinfo = null;//最后一次gc清理信息{ size, removecount, time ,nowtime} private boolean loggcdetail = false;//记录gc清理细节 private boolean enablegc = true;//是否允许清理的缓存(添加元素时) private int gcmode = 0;//清理过期元素模式 { 0=迭代模式 ; 1=随机模式 } private int gcintervaltime = 2 * 60 * 1000;//间隔时间(分钟) private boolean iteratescanall = true;//是否迭代扫描全部 private float gcfactor = 0.5f;//清理百分比 private int maxiteratesize = default_max_capacity/2;//迭代模式下一次最大迭代数量 private volatile int iteratelastindex = 0;//最后迭代下标 private int maxrandomtimes = 100;//随机模式下最大随机次数 protected final static random random = new random(); private static localcache instance = new localcache(); public static localcache getinstance() { return instance; } private localcache(){ this.init(); } protected synchronized void init() { if(initflag){ logger.warn("init repeat."); return ; } this.initcache(); this.startgcdaemonthread(); initflag = true; if(logger.isinfoenabled()){ logger.info("init -- ok"); } } private void startgcdaemonthread(){ if(initflag){ return ; } this.maxiteratesize = maxcapacity /2; try{ this.gcthread = new thread() { public void run() { logger.info("[" + (thread.currentthread().getname()) + "]start..."); //sleep try { thread.sleep(getgcintervaltime() < 30000 ? 30000 : getgcintervaltime()); } catch (exception e) { e.printstacktrace(); } while( true ){ //gc gc(); //sleep try { thread.sleep(getgcintervaltime() < 30000 ? 30000 : getgcintervaltime()); } catch (exception e) { e.printstacktrace(); } } } }; this.gcthread.setname("localcache-gcthread"); this.gcthread.setdaemon(true); this.gcthread.start(); if(logger.isinfoenabled()){ logger.info("startgcdaemonthread -- ok"); } }catch(exception e){ logger.error("[localcache gc]daemonthread -- error: " + e.getmessage(), e); } } private void initcache(){ if(initflag){ return ; } initialcapacity = (initialcapacity <= 0 ? default_initial_capacity : initialcapacity); maxcapacity = (maxcapacity < initialcapacity ? default_max_capacity : maxcapacity); cachemap = new lrulinkedhashmap<object, cacheelement>(initialcapacity ,maxcapacity); if(logger.isinfoenabled()){ logger.info("initcache -- ok"); } } /* * (non-javadoc) */ @suppresswarnings("unchecked") public <t> t getcache(object key) { if(!isenabled()){ return null; } long st = system.currenttimemillis(); t objvalue = null; cacheelement cacheobj = cachemap.get(key); if (isexpiredcache(cacheobj)) { cachemap.remove(key); }else { objvalue = (t) (cacheobj == null ? null : cacheobj.getvalue()); } long et = system.currenttimemillis(); if((et - st)>timeout){ if(this.logger.iswarnenabled()){ this.logger.warn("getcache_timeout_" + (et - st) + "_[" + key + "]"); } } if(logger.isdebugenabled()){ string message = ("get( " + key + ") return: " + objvalue); logger.debug(message); } return objvalue; } /* * (non-javadoc) */ public void putcache(object key, object value ,long lifetime) { if(!isenabled()){ return; } long st = system.currenttimemillis(); lifetime = (null == lifetime ? defaultlifetime : lifetime); cacheelement cacheobj = new cacheelement(); cacheobj.setcreatetime(system.currenttimemillis()); cacheobj.setlifetime(lifetime); cacheobj.setvalue(value); cacheobj.setkey(key); cachemap.put(key, cacheobj); long et = system.currenttimemillis(); if((et - st)>timeout){ if(this.logger.iswarnenabled()){ this.logger.warn("putcache_timeout_" + (et - st) + "_[" + key + "]"); } } if(logger.isdebugenabled()){ string message = ("putcache( " + cacheobj + " ) , 耗时 " + (et - st) + "(毫秒)."); logger.debug(message); } if(lifetime > defaultlifetime && this.iswarnlongerlifetime()){ if(logger.iswarnenabled()){ string message = ("lifetime[" + (lifetime/1000) + "秒] too long for putcache(" + cacheobj + ")"); logger.warn(message); } } } /** * key 是否过期 * @param key * @return */ protected boolean isexpiredkey(object key) { cacheelement cacheobj = cachemap.get(key); return this.isexpiredcache(cacheobj); } /** * cacheobj 是否过期 * @param key * @return */ protected boolean isexpiredcache(cacheelement cacheobj) { if (cacheobj == null) { return false; } return cacheobj.isexpired(); } /* * (non-javadoc) */ public void invalidatecaches(){ try{ cachemap.clear(); }catch(exception e){ e.printstacktrace(); } } /* * (non-javadoc) */ public void invalidatecache(object key){ try{ cachemap.remove(key); }catch(exception e){ e.printstacktrace(); } } /* * (non-javadoc) */ public boolean containskey(object key) { return cachemap.containskey(key); } /* * (non-javadoc) */ public int getsize() { return cachemap.size(); } /* * (non-javadoc) */ public iterator<object> getkeyiterator() { return cachemap.keyset().iterator(); } /* * (non-javadoc) */ public boolean isenabled() { return this.enabled; } /* * (non-javadoc) */ public void setenabled(boolean enabled) { this.enabled = enabled; if(!this.enabled){ //清理缓存 this.invalidatecaches(); } } /** * 清理过期缓存 */ protected synchronized boolean gc(){ if(!isenablegc()){ return false; } try{ iterateremoveexpiredcache(); }catch(exception e){ logger.error("gc() has error: " + e.getmessage(), e); } return true; } /** * 迭代模式 - 移除过期的 key * @param exceptkey */ private void iterateremoveexpiredcache(){ long starttime = system.currenttimemillis(); int size = cachemap.size(); if(size ==0){ return; } int keycount = 0; int removedcount = 0 ; int startindex = 0; int endindex = 0; try{ object [] keys = cachemap.keyset().toarray(); keycount = keys.length; int maxindex = keycount -1 ; //初始化扫描下标 if(iteratescanall){ startindex = 0; endindex = maxindex; }else { int gcthreshold = this.getgcthreshold(); int iteratelen = gcthreshold > this.maxiteratesize ? this.maxiteratesize : gcthreshold; startindex = this.iteratelastindex; startindex = ( (startindex < 0 || startindex > maxindex) ? 0 : startindex ); endindex = (startindex + iteratelen); endindex = (endindex > maxindex ? maxindex : endindex); } //迭代清理 boolean flag = false; for(int i=startindex; i<= endindex; i++){ flag = this.removeexpiredkey(keys[i]); if(flag){ removedcount++; } } this.iteratelastindex = endindex; keys = null; }catch(exception e){ logger.error("iterateremoveexpiredcache -- 移除过期的 key时出现异常: " + e.getmessage(), e); } long endtime = system.currenttimemillis(); stringbuffer sb = new stringbuffer(); sb.append("iterateremoveexpiredcache [ size: ").append(size).append(", keycount: ").append(keycount) .append(", startindex: ").append(startindex).append(", endindex: ").append(iteratelastindex) .append(", removedcount: ").append(removedcount).append(", currentsize: ").append(this.cachemap.size()) .append(", timeconsuming: ").append(endtime - starttime).append(", nowtime: ").append(new date()) .append(" ]"); this.lastgcinfo = sb.tostring(); if(logger.isinfoenabled()){ logger.info("iterateremoveexpiredcache -- 清理结果 -- "+ lastgcinfo); } } /** * 随机模式 - 移除过期的 key */ private void randomremoveexpiredcache(){ long starttime = system.currenttimemillis(); int size = cachemap.size(); if(size ==0){ return; } int removedcount = 0 ; try{ object [] keys = cachemap.keyset().toarray(); int keycount = keys.length; boolean removeflag = false; int removerandomtimes = this.getgcthreshold(); removerandomtimes = ( removerandomtimes > this.getmaxrandomtimes() ? this.getmaxrandomtimes() : removerandomtimes ); while(removerandomtimes-- > 0){ int index = random.nextint(keycount); boolean flag = this.removeexpiredkey(keys[index]); if(flag){ removeflag = true; removedcount ++; } } //尝试 移除 首尾元素 if(!removeflag){ this.removeexpiredkey(keys[0]); this.removeexpiredkey(keys[keycount-1]); } keys=null; }catch(exception e){ logger.error("randomremoveexpiredcache -- 移除过期的 key时出现异常: " + e.getmessage(), e); } long endtime = system.currenttimemillis(); stringbuffer sb = new stringbuffer(); sb.append("randomremoveexpiredcache [ size: ").append(size).append(", removedcount: ").append(removedcount) .append(", currentsize: ").append(this.cachemap.size()).append(", timeconsuming: ").append(endtime - starttime) .append(", nowtime: ").append(new date()) .append(" ]"); this.lastgcinfo = sb.tostring(); if(logger.isinfoenabled()){ logger.info("randomremoveexpiredcache -- 清理结果 -- "+ lastgcinfo); } } private boolean removeexpiredkey(object key){ boolean flag = false; cacheelement cacheobj = null; if(null != key){ try{ cacheobj = cachemap.get(key); boolean isexpiredcache = this.isexpiredcache(cacheobj); if(isexpiredcache){ cachemap.remove(key); flag = true; } }catch(exception e){ logger.error("removeexpired(" + key + ") -- error: " + e.getmessage(), e); } } if(!flag && loggcdetail){ this.logger.warn("removeexpiredkey(" + key + ") return [" + flag + "]--" + cacheobj); } return flag; } public int getinitialcapacity() { return initialcapacity; } public int getmaxcapacity() { return maxcapacity; } public int getgcmode() { return gcmode; } public void setgcmode(int gcmode) { this.gcmode = gcmode; } public int getgcintervaltime() { return gcintervaltime; } public void setgcintervaltime(int gcintervaltime) { this.gcintervaltime = gcintervaltime; } public boolean isenablegc() { return enablegc; } public void setenablegc(boolean enablegc) { this.enablegc = enablegc; } public boolean isiteratescanall() { return iteratescanall; } public void setiteratescanall(boolean iteratescanall) { this.iteratescanall = iteratescanall; } public float getgcfactor() { return gcfactor; } public void setgcfactor(float gcfactor) { this.gcfactor = gcfactor; } /** * gc 阀值 * @return */ public int getgcthreshold() { int threshold = (int)( this.cachemap.getmaxcapacity() * gcfactor ); return threshold; } public string getlastgcinfo() { return lastgcinfo; } public void setlastgcinfo(string lastgcinfo) { this.lastgcinfo = lastgcinfo; } public boolean isloggcdetail() { return loggcdetail; } public void setloggcdetail(boolean loggcdetail) { this.loggcdetail = loggcdetail; } public int gettimeout() { return timeout; } public void settimeout(int timeout) { this.timeout = timeout; } public int getmaxiteratesize() { return maxiteratesize; } public void setmaxiteratesize(int maxiteratesize) { this.maxiteratesize = maxiteratesize; } public int getmaxrandomtimes() { return maxrandomtimes; } public void setmaxrandomtimes(int maxrandomtimes) { this.maxrandomtimes = maxrandomtimes; } public boolean isinitflag() { return initflag; } public long getdefaultlifetime() { return defaultlifetime; } public boolean iswarnlongerlifetime() { return warnlongerlifetime; } public void setwarnlongerlifetime(boolean warnlongerlifetime) { this.warnlongerlifetime = warnlongerlifetime; } //======================== dynmaxcapacity ======================== private int dynmaxcapacity = maxcapacity; public int getdynmaxcapacity() { return dynmaxcapacity; } public void setdynmaxcapacity(int dynmaxcapacity) { this.dynmaxcapacity = dynmaxcapacity; } public void resetmaxcapacity(){ if(dynmaxcapacity > initialcapacity && dynmaxcapacity != maxcapacity){ if(logger.isinfoenabled()){ logger.info("resetmaxcapacity( " + dynmaxcapacity + " ) start..."); } synchronized(cachemap){ lrulinkedhashmap<object, cacheelement> cachemap0 = new lrulinkedhashmap<object, cacheelement>(initialcapacity ,dynmaxcapacity); cachemap.clear(); cachemap = cachemap0; this.maxcapacity = dynmaxcapacity; } if(logger.isinfoenabled()){ logger.info("resetmaxcapacity( " + dynmaxcapacity + " ) ok."); } }else { if(logger.iswarnenabled()){ logger.warn("resetmaxcapacity( " + dynmaxcapacity + " ) no."); } } } //======================== showcacheelement ======================== private string showcachekey; public string getshowcachekey() { return showcachekey; } public void setshowcachekey(string showcachekey) { this.showcachekey = showcachekey; } public object showcacheelement(){ object v = null; if(null != this.showcachekey){ v = cachemap.get(showcachekey); } return v; } }
以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持。
上一篇: Mysql查询语句优化技巧