欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页

logstash-过滤ES数据到新的ES

程序员文章站 2022-07-06 19:58:25
...

背景:过滤ES集群中特定类型的某个时间段的数据到新的ES集群
具体操作下:

###类型TCC
input{
  elasticsearch {
    hosts => "10.10.x.x:9200"
    index => "cqct_20200508_03"
    query => '{"query":{"bool":{"must":[{"range":{"optime":{"gt":"1591754706000","lt":"1591754706006"}}}],"must_not":[],"should":[]}},"from":0,"sort":[],"aggs":{}}'
    docinfo => true
    }
}
filter{
        if [@metadata][_type] != "TCC" {
        drop{}
  }
}
output{
        elasticsearch{
                hosts => "10.10.y.y:9200"
                index => "cqct_20200508_03"
                document_type => "TCC"
                document_id => "%{id}"
                codec => json_lines
        }
        stdout{
                codec => json_lines
        }

}

##RFID

input{
  elasticsearch {
    hosts => "10.10.x.x:9200"
    index => "cqct_20200508_*"
    query => '{"query":{"bool":{"must":[{"range":{"collectTime":{"gt":"1591754706000","lt":"1591754706006"}}}],"must_not":[],"should":[]}},"from":0,"sort":[],"aggs":{}}'
    docinfo => true
    }
}
filter{
         if [@metadata][_type] != "RFID" {
        drop{}
  }
}
output{
        elasticsearch{
                hosts => "10.10.y.y:9200"
                index => "cqct_20200508_03"
                document_type => "RFID"
                document_id => "%{id}"
                codec => json_lines
        }
        stdout{
                codec => json_lines
        }
}