欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页

SparkSql拼接SQL优化

程序员文章站 2024-01-30 15:30:10
...

减少对象的创建

  val dfHive: DataFrame = spark.sql("select "+str+ " from t_emp "+joinName+" t_dept on "+joinCondition)
//优化后
    val builder = new StringBuilder("select ")
    val sql = builder.append(str).append(" from t_emp ").append(joinName).append(" t_dept on ").append(joinCondition).toString()
    val dfHive: DataFrame = spark.sql(sql)