欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页

Flink入门 wordcount——Java (代码)

程序员文章站 2022-06-16 15:19:47
...

官网讲解连接

https://ci.apache.org/projects/flink/flink-docs-release-1.6/dev/batch/index.html#example-program

Flink入门 wordcount——Java (代码)

导入jar包

本地安装的是1.0版本的,修改成自己对应版本即可

<!--java 依赖包导入-->
<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-java</artifactId>
    <version>1.6.1</version>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-streaming-java_2.11</artifactId>
    <version>1.6.1</version>
    <scope>provided</scope>
</dependency>	

import org.apache.flink.api.java.ExecutionEnvironment;
import org.apache.flink.api.java.operators.AggregateOperator;
import org.apache.flink.api.java.operators.DataSource;
import org.apache.flink.api.java.tuple.Tuple2;

/**
 * @program: Flink
 * @description:
 * @author: wenglei
 * @create: 2020-06-03 14:55
 **/
public class WordCount {
    public static void main(String[] args) throws Exception {
        //初始化环境
       final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
        //初始化数据
        DataSource<String> data = env.fromElements("buzhdiao, zhaoliying yangmi zhangxiulian zhangxiulian");
        //指定数据转换,自定义分割方法
        AggregateOperator<Tuple2<String, Integer>> sum = data
                .flatMap(new LineSplitter())
                .groupBy(0)       //以Tuple的第一个字段分组
                .sum(1);            //对Tuple的第二个字段求和

        //控制台数据输出
        sum.print();
    }

}
import org.apache.flink.api.common.functions.FlatMapFunction;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.util.Collector;

/**
 * @program: Flink
 * @description:
 * @author: wenglei
 * @create: 2020-06-03 15:01
 **/
public class LineSplitter implements FlatMapFunction<String, Tuple2<String, Integer>> {

    // 数据源数据处理,将文本串解析成(单词-数量)元组
    public void flatMap(String s, Collector<Tuple2<String, Integer>> collector) throws Exception {
        //对数据进行分割
        String[] split = s.split(" ");

        for (String s1 : split) {
            collector.collect(new Tuple2<String,Integer>(s1,1));
        }

    }
}

Flink入门 wordcount——Java (代码)