欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页

SPARK的线性代数库BLAS

程序员文章站 2022-06-03 12:23:52
...

spark中的BLAS中引入了两个包

import com.github.fommil.netlib.{BLAS => NetlibBLAS, F2jBLAS}
import com.github.fommil.netlib.BLAS.{getInstance => NativeBLAS}

因为private[spark] object BLAS extends Serializable with Logging{ ....}
BLAS是private[spark]修饰,那么我们如何在其他地方之间使用这个类呢?
(1)因此我们在外部使用这个类需要自己创建的类与这个BLAS的包相同;
(2)编译源码。

另外加入依赖

<dependency>     <groupId>com.github.fommil.netlib</groupId>
        <artifactId>all</artifactId>
        <version>1.1.2</version>
        <type>pom</type>
</dependency>

有一个地方不太清楚,希望有人可以解答?

  // For level-3 routines, we use the native BLAS.
  private def nativeBLAS: NetlibBLAS = {
    if (_nativeBLAS == null) {
      _nativeBLAS = NativeBLAS
    }
    _nativeBLAS
  }

routines是如何翻译的?
查阅了资料查到了netlib的BLAS是有三个级别的routines三个级别的routines链接

http://www.netlib.org/blas/#_blas_routines

package org.apache.spark
/**
  * Created by hhy on 2017/12/08.
  */
import org.apache.spark.mllib.linalg.{BLAS, Vectors}
import org.apache.spark.mllib.linalg._
object BLASdemo {
  def main(args: Array[String]):Unit={
    val X=Vectors.dense(1,2,3)
    val Y=Vectors.dense(4,5,6)
    //println(com.github.fommil.netlib.BLAS.getInstance().getClass().getName())
    //println(com.github.fommil.netlib.LAPACK.getInstance().getClass().getName())

    /**
      * 点积 返回值为Double
      * dot(x: DenseVector, y: DenseVector): Double
      * dot(x: SparseVector, y: SparseVector): Double
      *  dot(x: SparseVector, y: DenseVector): Double
      *  dot(x: DenseVector, y: SparseVector): Double
      * */
    val Z=BLAS.dot(X,Y)
    println(Z)   //32


    //x = a * x  scal(a: Double, x: Vector): Unit
    val a=3.0
    BLAS.scal(a,X)    //X是1,2,3
    println("SCAL后的X:"+X)  //SCAL后的X:[3.0,6.0,9.0]

    //y = x  copy(x: Vector, y: Vector): Unit
    val y=Vectors.dense(1,2,3)  //注意两个向量维度要一样否则报错 Exception in thread "main" java.lang.IllegalArgumentException: requirement failed
    //var y=Vectors.dense(1,2,3)  //var  val都修改了y
    BLAS.copy(X,y)        //X是3,6,9
    println("复制X到y得到y:"+y)  //复制X到y得到y:[3.0,6.0,9.0]

    /**
      * y += a * x
      * axpy(a: Double, x: DenseVector, y: DenseVector): Unit
      * axpy(a: Double, x: Vector, y: Vector): Unit
      * axpy(a: Double, x: SparseVector, y: DenseVector): Unit
      * private[spark] def axpy(a: Double, X: DenseMatrix, Y: DenseMatrix): Unit
      * */
    val b=4
    BLAS.axpy(b,X,Y)  //X是3,6,9    Y是4,5,6
    println("Y:"+Y)  //Y:[16.0,29.0,42.0]

    /**矩阵相乘 并且可以设置相乘矩阵的一个系数 同时还可以加上另外一个矩阵
      * C := alpha * A * B + beta * C
      * A:m x k.
      * B:k x n
      * C: m x n.
      *  * */
    def gemm(alpha: Double,A: Matrix,B: DenseMatrix,beta: Double,C: DenseMatrix): Unit
    def gemm(alpha: Double,A: DenseMatrix,B: DenseMatrix,beta: Double,C: DenseMatrix): Unit
    def gemm(alpha: Double,A: SparseMatrix,B: DenseMatrix,beta: Double,C: DenseMatrix): Unit

    /**
      * y := alpha * A * x + beta * y
      * A: m x n
      * x:n x 1
      * y:m x 1
      * */
    def gemv(alpha: Double, A: Matrix, x: Vector, beta: Double, y: DenseVector): Unit={}
    def gemv(alpha: Double, A: DenseMatrix, x: DenseVector, beta: Double, y: DenseVector): Unit={}
    def gemv(alpha: Double, A: DenseMatrix, x: SparseVector, beta: Double, y: DenseVector): Unit={}
    def gemv(alpha: Double, A: SparseMatrix, x: SparseVector, beta: Double, y: DenseVector): Unit={}
    def gemv(alpha: Double, A: SparseMatrix, x: DenseVector, beta: Double, y: DenseVector): Unit={}

    /**
      * 增加alpha * v * v.t 到矩阵  U是以列为主序的矩阵的上三角部分
      * */
    def spr(alpha: Double, v: Vector, U: Array[Double]): Unit={}
    def spr(alpha: Double, v: Vector, U: Array[Double]): Unit={}

    /**
      * A := alpha * x * x^T^ + A
      * x有n个元素  A是 n x n.方阵
      * */
    def syr(alpha: Double, x: Vector, A: DenseMatrix){}
    private def syr(alpha: Double, x: DenseVector, A: DenseMatrix){}
    private def syr(alpha: Double, x: SparseVector, A: DenseMatrix){}
  }
}