2020-12-31

程序员文章站 2022-03-19 20:09:24

ElasticSearch之将网页上爬取的数据传到elastic search中。我们一般会将数据保存到elasticsearch中来实现搜索。项目源码：源码地址1.从网页爬取数据网页解析数据一般会用Jsoup包。首先引进来吧。接下来，我会从彼岸图网这个网站爬取一些图片，（仅测试娱乐，无商业用途）并将信息保存到elasticsearch上。开始吧。新建一个springboot项目。引入jsoup包。以及会用到的json解析包。 ...

Eureka server启动流程图和源码分析

流程图

2020-12-31

源码解析

由流程图看出，Eureka server启动时会做两件事

1.服务同步

因为eureka服务可以搭建集群，所以每个服务节点启动时，会从相邻的eureka节点中同步实例数据，源码如下：

 @Override
    public int syncUp() {
        // Copy entire entry from neighboring DS node
        int count = 0;
        //当我们配置register-with-eureka: true时候，getRegistrySyncRetries的返回值为5
        for (int i = 0; ((i < serverConfig.getRegistrySyncRetries()) && (count == 0)); i++) {
            if (i > 0) {
                try {
                    //从第二次开始，休眠30秒
                    Thread.sleep(serverConfig.getRegistrySyncRetryWaitMs());
                } catch (InterruptedException e) {
                    logger.warn("Interrupted during registry transfer..");
                    break;
                }
            }
            //eurekaClient是当前节点的最近一个节点，在启动时会初始化
            Applications apps = eurekaClient.getApplications();
            //循环节点上的实例
            for (Application app : apps.getRegisteredApplications()) {
                for (InstanceInfo instance : app.getInstances()) {
                    try {
                        if (isRegisterable(instance)) {
                            //将实例同步到本地内存
                            register(instance, instance.getLeaseInfo().getDurationInSecs(), true);
                            count++;
                        }
                    } catch (Throwable t) {
                        logger.error("During DS init copy", t);
                    }
                }
            }
        }
        return count;
    }

当然，若我们的eureka服务为单机的话，可以设置以下配置

eureka：
  client:
    fetch-registry: false   #是否把eureka服务当成client注册到配置中心
    register-with-eureka: false  #eureka服务是否需要读取配置中心的实例

2.服务剔除

当完成服务同步后，eureka会开启服务剔除的定时任务，源码如下：

protected void postInit() {
        renewsLastMin.start();
        if (evictionTaskRef.get() != null) {
            evictionTaskRef.get().cancel();
        }
        evictionTaskRef.set(new EvictionTask());
        //执行java Timer 定时器任务
        evictionTimer.schedule(evictionTaskRef.get(),
                serverConfig.getEvictionIntervalTimerInMs(), //60秒后执行任务，可配置
                serverConfig.getEvictionIntervalTimerInMs()); //每次执行完任务后，间隔60秒再执行一次
    }

EvictionTask实现了Runnable方法，其run方法会调用evict方法，evict方法源码如下

//additionalLeaseMs为当前时间
public void evict(long additionalLeaseMs) {
        logger.debug("Running the evict task");

        if (!isLeaseExpirationEnabled()) {
            logger.debug("DS: lease expiration is currently disabled.");
            return;
        }

        // We collect first all expired items, to evict them in random order. For large eviction sets,
        // if we do not that, we might wipe out whole apps before self preservation kicks in. By randomizing it,
        // the impact should be evenly distributed across all applications.
        List<Lease<InstanceInfo>> expiredLeases = new ArrayList<>();
        //循环实例
        for (Entry<String, Map<String, Lease<InstanceInfo>>> groupEntry : registry.entrySet()) {
            Map<String, Lease<InstanceInfo>> leaseMap = groupEntry.getValue();
            if (leaseMap != null) {
                for (Entry<String, Lease<InstanceInfo>> leaseEntry : leaseMap.entrySet()) {
                    Lease<InstanceInfo> lease = leaseEntry.getValue();
                    //如果该实例最后一次心跳续约记录日期与当前时间间隔大于90秒，则add到List，后续执行剔除操作
                    if (lease.isExpired(additionalLeaseMs) && lease.getHolder() != null) {
                        expiredLeases.add(lease);
                    }
                }
            }
        }

        // To compensate for GC pauses or drifting local time, we need to use current registry size as a base for
        // triggering self-preservation. Without that we would wipe out full registry.
        //设置最大剔除数量
        int registrySize = (int) getLocalRegistrySize();
        int registrySizeThreshold = (int) (registrySize * serverConfig.getRenewalPercentThreshold());
        int evictionLimit = registrySize - registrySizeThreshold;

        int toEvict = Math.min(expiredLeases.size(), evictionLimit);
        if (toEvict > 0) {
            logger.info("Evicting {} items (expired={}, evictionLimit={})", toEvict, expiredLeases.size(), evictionLimit);
            //随机剔除实例
            Random random = new Random(System.currentTimeMillis());
            for (int i = 0; i < toEvict; i++) {
                // Pick a random item (Knuth shuffle algorithm)
                int next = i + random.nextInt(expiredLeases.size() - i);
                Collections.swap(expiredLeases, i, next);
                Lease<InstanceInfo> lease = expiredLeases.get(i);

                String appName = lease.getHolder().getAppName();
                String id = lease.getHolder().getId();
                EXPIRED.increment();
                logger.warn("DS: Registry: expired lease for {}/{}", appName, id);
                internalCancel(appName, id, false);
            }
        }
    }

本文地址：https://blog.csdn.net/qq_34210372/article/details/112024449