2020-12-31
程序员文章站
2022-06-28 08:00:19
ElasticSearch之将网页上爬取的数据传到elastic search中。我们一般会将数据保存到elasticsearch中来实现搜索。项目源码: 源码地址1.从网页爬取数据网页解析数据一般会用Jsoup包。首先引进来吧。接下来,我会从彼岸图网这个网站爬取一些图片,(仅测试娱乐,无商业用途)并将信息保存到elasticsearch上。开始吧。新建一个springboot项目。引入jsoup包。以及会用到的json解析包。 ...
Eureka server启动流程图和源码分析
流程图
源码解析
由流程图看出,Eureka server启动时会做两件事
1.服务同步
因为eureka服务可以搭建集群,所以每个服务节点启动时,会从相邻的eureka节点中同步实例数据,源码如下:
@Override
public int syncUp() {
// Copy entire entry from neighboring DS node
int count = 0;
//当我们配置register-with-eureka: true时候,getRegistrySyncRetries的返回值为5
for (int i = 0; ((i < serverConfig.getRegistrySyncRetries()) && (count == 0)); i++) {
if (i > 0) {
try {
//从第二次开始,休眠30秒
Thread.sleep(serverConfig.getRegistrySyncRetryWaitMs());
} catch (InterruptedException e) {
logger.warn("Interrupted during registry transfer..");
break;
}
}
//eurekaClient是当前节点的最近一个节点,在启动时会初始化
Applications apps = eurekaClient.getApplications();
//循环节点上的实例
for (Application app : apps.getRegisteredApplications()) {
for (InstanceInfo instance : app.getInstances()) {
try {
if (isRegisterable(instance)) {
//将实例同步到本地内存
register(instance, instance.getLeaseInfo().getDurationInSecs(), true);
count++;
}
} catch (Throwable t) {
logger.error("During DS init copy", t);
}
}
}
}
return count;
}
当然,若我们的eureka服务为单机的话,可以设置以下配置
eureka:
client:
fetch-registry: false #是否把eureka服务当成client注册到配置中心
register-with-eureka: false #eureka服务是否需要读取配置中心的实例
2.服务剔除
当完成服务同步后,eureka会开启服务剔除的定时任务,源码如下:
protected void postInit() {
renewsLastMin.start();
if (evictionTaskRef.get() != null) {
evictionTaskRef.get().cancel();
}
evictionTaskRef.set(new EvictionTask());
//执行java Timer 定时器任务
evictionTimer.schedule(evictionTaskRef.get(),
serverConfig.getEvictionIntervalTimerInMs(), //60秒后执行任务,可配置
serverConfig.getEvictionIntervalTimerInMs()); //每次执行完任务后,间隔60秒再执行一次
}
EvictionTask实现了Runnable方法,其run方法会调用evict方法,evict方法源码如下
//additionalLeaseMs为当前时间
public void evict(long additionalLeaseMs) {
logger.debug("Running the evict task");
if (!isLeaseExpirationEnabled()) {
logger.debug("DS: lease expiration is currently disabled.");
return;
}
// We collect first all expired items, to evict them in random order. For large eviction sets,
// if we do not that, we might wipe out whole apps before self preservation kicks in. By randomizing it,
// the impact should be evenly distributed across all applications.
List<Lease<InstanceInfo>> expiredLeases = new ArrayList<>();
//循环实例
for (Entry<String, Map<String, Lease<InstanceInfo>>> groupEntry : registry.entrySet()) {
Map<String, Lease<InstanceInfo>> leaseMap = groupEntry.getValue();
if (leaseMap != null) {
for (Entry<String, Lease<InstanceInfo>> leaseEntry : leaseMap.entrySet()) {
Lease<InstanceInfo> lease = leaseEntry.getValue();
//如果该实例最后一次心跳续约记录日期与当前时间间隔大于90秒,则add到List,后续执行剔除操作
if (lease.isExpired(additionalLeaseMs) && lease.getHolder() != null) {
expiredLeases.add(lease);
}
}
}
}
// To compensate for GC pauses or drifting local time, we need to use current registry size as a base for
// triggering self-preservation. Without that we would wipe out full registry.
//设置最大剔除数量
int registrySize = (int) getLocalRegistrySize();
int registrySizeThreshold = (int) (registrySize * serverConfig.getRenewalPercentThreshold());
int evictionLimit = registrySize - registrySizeThreshold;
int toEvict = Math.min(expiredLeases.size(), evictionLimit);
if (toEvict > 0) {
logger.info("Evicting {} items (expired={}, evictionLimit={})", toEvict, expiredLeases.size(), evictionLimit);
//随机剔除实例
Random random = new Random(System.currentTimeMillis());
for (int i = 0; i < toEvict; i++) {
// Pick a random item (Knuth shuffle algorithm)
int next = i + random.nextInt(expiredLeases.size() - i);
Collections.swap(expiredLeases, i, next);
Lease<InstanceInfo> lease = expiredLeases.get(i);
String appName = lease.getHolder().getAppName();
String id = lease.getHolder().getId();
EXPIRED.increment();
logger.warn("DS: Registry: expired lease for {}/{}", appName, id);
internalCancel(appName, id, false);
}
}
}
本文地址:https://blog.csdn.net/qq_34210372/article/details/112024449
下一篇: 彻底剖析JVM类加载机制(二)
推荐阅读