SPARK的线性代数库BLAS
程序员文章站
2022-06-03 12:23:52
...
spark中的BLAS中引入了两个包
import com.github.fommil.netlib.{BLAS => NetlibBLAS, F2jBLAS}
import com.github.fommil.netlib.BLAS.{getInstance => NativeBLAS}
因为private[spark] object BLAS extends Serializable with Logging{ ....}
BLAS是private[spark]修饰,那么我们如何在其他地方之间使用这个类呢?
(1)因此我们在外部使用这个类需要自己创建的类与这个BLAS的包相同;
(2)编译源码。
另外加入依赖
<dependency> <groupId>com.github.fommil.netlib</groupId>
<artifactId>all</artifactId>
<version>1.1.2</version>
<type>pom</type>
</dependency>
有一个地方不太清楚,希望有人可以解答?
// For level-3 routines, we use the native BLAS.
private def nativeBLAS: NetlibBLAS = {
if (_nativeBLAS == null) {
_nativeBLAS = NativeBLAS
}
_nativeBLAS
}
routines是如何翻译的?
查阅了资料查到了netlib的BLAS是有三个级别的routines三个级别的routines链接
http://www.netlib.org/blas/#_blas_routines
package org.apache.spark
/**
* Created by hhy on 2017/12/08.
*/
import org.apache.spark.mllib.linalg.{BLAS, Vectors}
import org.apache.spark.mllib.linalg._
object BLASdemo {
def main(args: Array[String]):Unit={
val X=Vectors.dense(1,2,3)
val Y=Vectors.dense(4,5,6)
//println(com.github.fommil.netlib.BLAS.getInstance().getClass().getName())
//println(com.github.fommil.netlib.LAPACK.getInstance().getClass().getName())
/**
* 点积 返回值为Double
* dot(x: DenseVector, y: DenseVector): Double
* dot(x: SparseVector, y: SparseVector): Double
* dot(x: SparseVector, y: DenseVector): Double
* dot(x: DenseVector, y: SparseVector): Double
* */
val Z=BLAS.dot(X,Y)
println(Z) //32
//x = a * x scal(a: Double, x: Vector): Unit
val a=3.0
BLAS.scal(a,X) //X是1,2,3
println("SCAL后的X:"+X) //SCAL后的X:[3.0,6.0,9.0]
//y = x copy(x: Vector, y: Vector): Unit
val y=Vectors.dense(1,2,3) //注意两个向量维度要一样否则报错 Exception in thread "main" java.lang.IllegalArgumentException: requirement failed
//var y=Vectors.dense(1,2,3) //var val都修改了y
BLAS.copy(X,y) //X是3,6,9
println("复制X到y得到y:"+y) //复制X到y得到y:[3.0,6.0,9.0]
/**
* y += a * x
* axpy(a: Double, x: DenseVector, y: DenseVector): Unit
* axpy(a: Double, x: Vector, y: Vector): Unit
* axpy(a: Double, x: SparseVector, y: DenseVector): Unit
* private[spark] def axpy(a: Double, X: DenseMatrix, Y: DenseMatrix): Unit
* */
val b=4
BLAS.axpy(b,X,Y) //X是3,6,9 Y是4,5,6
println("Y:"+Y) //Y:[16.0,29.0,42.0]
/**矩阵相乘 并且可以设置相乘矩阵的一个系数 同时还可以加上另外一个矩阵
* C := alpha * A * B + beta * C
* A:m x k.
* B:k x n
* C: m x n.
* * */
def gemm(alpha: Double,A: Matrix,B: DenseMatrix,beta: Double,C: DenseMatrix): Unit
def gemm(alpha: Double,A: DenseMatrix,B: DenseMatrix,beta: Double,C: DenseMatrix): Unit
def gemm(alpha: Double,A: SparseMatrix,B: DenseMatrix,beta: Double,C: DenseMatrix): Unit
/**
* y := alpha * A * x + beta * y
* A: m x n
* x:n x 1
* y:m x 1
* */
def gemv(alpha: Double, A: Matrix, x: Vector, beta: Double, y: DenseVector): Unit={}
def gemv(alpha: Double, A: DenseMatrix, x: DenseVector, beta: Double, y: DenseVector): Unit={}
def gemv(alpha: Double, A: DenseMatrix, x: SparseVector, beta: Double, y: DenseVector): Unit={}
def gemv(alpha: Double, A: SparseMatrix, x: SparseVector, beta: Double, y: DenseVector): Unit={}
def gemv(alpha: Double, A: SparseMatrix, x: DenseVector, beta: Double, y: DenseVector): Unit={}
/**
* 增加alpha * v * v.t 到矩阵 U是以列为主序的矩阵的上三角部分
* */
def spr(alpha: Double, v: Vector, U: Array[Double]): Unit={}
def spr(alpha: Double, v: Vector, U: Array[Double]): Unit={}
/**
* A := alpha * x * x^T^ + A
* x有n个元素 A是 n x n.方阵
* */
def syr(alpha: Double, x: Vector, A: DenseMatrix){}
private def syr(alpha: Double, x: DenseVector, A: DenseMatrix){}
private def syr(alpha: Double, x: SparseVector, A: DenseMatrix){}
}
}
上一篇: EXCELPOI的样式设置