欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页  >  IT编程

java 矩阵乘法的mapreduce程序实现

程序员文章站 2023-12-18 11:49:34
java 矩阵乘法的mapreduce程序实现 map函数:对于矩阵m中的每个元素m(ij),产生一系列的key-value对<(i,k),(m,j,m(ij))&...

java 矩阵乘法的mapreduce程序实现

map函数:对于矩阵m中的每个元素m(ij),产生一系列的key-value对<(i,k),(m,j,m(ij))>

其中k=1,2.....知道矩阵n的总列数;对于矩阵n中的每个元素n(jk),产生一系列的key-value对<(i , k) , (n , j ,n(jk)>, 其中i=1,2.......直到i=1,2.......直到矩阵m的总列数。

map

package com.cb.matrix;

import static org.mockito.matchers.intthat;

import java.io.ioexception;

import org.apache.hadoop.conf.configuration;
import org.apache.hadoop.io.text;
import org.apache.hadoop.mapred.filesplit;
import org.apache.hadoop.mapreduce.mapper;

import com.sun.org.apache.bcel.internal.generic.new;


public class matrixmapper extends mapper<object, text, text, text> {
 private text map_key=new text();
 private text map_value= new text();
 private int columnn;
 private int rowm;
 /**
 * 执行map()函数前先由conf.get()得到main函数中提供的必要变量
 * 也就是从输入文件名中得到的矩阵维度信息
 */
 
 @override
 protected void setup(mapper<object, text, text, text>.context context) throws ioexception, interruptedexception {
 // todo auto-generated method stub
 configuration config=context.getconfiguration();
 columnn=integer.parseint(config.get("columnn"));
 rowm =integer.parseint(config.get("rowm"));
 }
 
 @override
 protected void map(object key, text value, mapper<object, text, text, text>.context context)
  throws ioexception, interruptedexception {
 // todo auto-generated method stub
 //得到文件名,从而区分输入矩阵m和n
 filesplit filesplit=(filesplit)context.getinputsplit();
 string filename=filesplit.getpath().getname();
 
 if (filename.contains("m")) {
  string[] tuple =value.tostring().split(",");
  int i =integer.parseint(tuple[0]);
  string[] tuples=tuple[1].split("\t");
  int j=integer.parseint(tuples[0]);
  int mij=integer.parseint(tuples[1]);
  for(int k=1;k<columnn+1;k++){
  map_key.set(i+","+k);
  map_value.set("m"+","+j+","+mij);
  context.write(map_key, map_value);
  }
  
 }
 else if(filename.contains("n")){
  string[] tuple=value.tostring().split(",");
  int j=integer.parseint(tuple[0]);
  string[] tuples =tuple[1].split("\t");
  int k=integer.parseint(tuples[0]);
  int njk=integer.parseint(tuples[1]);
  for(int i=1;i<rowm+1;i++){
  map_key.set(i+","+k);
  map_value.set("n"+","+j+","+njk);
  context.write(map_key, map_value);
  }
 }
 
 }

}

reduce函数:对于每个键(i,k)相关联的值(m,j,m(ij))及(n,j,n(jk)),根据相同的j值将m(ij)和n(jk)分别存入不同的数组中,然后将俩者的第j个元素抽取出来分别相乘,最后相加,即可得到p(jk)的值。

reducer

package com.cb.matrix;


import java.io.ioexception;

import org.apache.hadoop.conf.configuration;
import org.apache.hadoop.io.text;
import org.apache.hadoop.mapreduce.reducer;



public class matrixreducer extends reducer<text, text, text, text> {
 private int sum=0;
 private int columnm;
 @override
 protected void setup(reducer<text, text, text, text>.context context) throws ioexception, interruptedexception {
 // todo auto-generated method stub
 configuration conf =context.getconfiguration();
 columnm=integer.parseint(conf.get("columnm"));
 }
 @override
 protected void reduce(text arg0, iterable<text> arg1, reducer<text, text, text, text>.context arg2)
  throws ioexception, interruptedexception {
 // todo auto-generated method stub
 int[] m=new int[columnm+1];
 int[] n=new int[columnm+1];
 
 for(text val:arg1){
  string[] tuple=val.tostring().split(",");
  if(tuple[0].equals("m")){
  m[integer.parseint(tuple[1])]=integer.parseint(tuple[2]);
  
  }else{
  n[integer.parseint(tuple[1])]=integer.parseint(tuple[2]);
  }
  for(int j=1;j<columnm+1;j++){
  sum+=m[j]*n[j];
  }
  arg2.write(arg0, new text(integer.tostring(sum)));
  sum=0;
 }
 }

}

 感谢阅读,希望能帮助到大家,谢谢大家对本站的支持!

上一篇:

下一篇: