(转)Java 文件分块上传服务器端源代码
本博客将介绍如何进行文件的分块上传。如果读者还想了解文件的“分块”下载相关内容可以去参考博客《Java 服务器端支持断点续传的源代码【支持快车、迅雷】》。
本文侧重介绍服务器端,客户端端请参考本篇博客的姊妹篇《Java 文件分块上传客户端源代码》,关于分块上传的思想及其流程,已在该博客中进行了详细说明,这里不再赘述。
直接上代码。接收客户端 HTTP 分块上传请求的 Spring MVC 控制器源代码如下:
@Controller public class UploadController extends BaseController { private static final Log log = LogFactory.getLog(UploadController.class); private UploadService uploadService; private AuthService authService; /** * 大文件分成小文件块上传,一次传递一块,最后一块上传成功后,将合并所有已经上传的块,保存到File Server * 上相应的位置,并返回已经成功上传的文件的详细属性. 当最后一块上传完毕,返回上传成功的信息。此时用getFileList查询该文件, * 该文件的uploadStatus为2。client请自行处理该状态下文件如何显示。(for UPS Server) * */ @RequestMapping("/core/v1/file/upload") @ResponseBody public Object upload(HttpServletResponse response, @RequestParam(value = "client_id", required = false) String appkey, @RequestParam(value = "sig", required = false) String appsig, @RequestParam(value = "token", required = false) String token, @RequestParam(value = "uuid", required = false) String uuid, @RequestParam(value = "block", required = false) String blockIndex, @RequestParam(value = "file", required = false) MultipartFile multipartFile, @RequestParam Map<String, String> parameters) { checkEmpty(appkey, BaseException.ERROR_CODE_16002); checkEmpty(token, BaseException.ERROR_CODE_16007); checkEmpty(uuid, BaseException.ERROR_CODE_20016); checkEmpty(blockIndex, BaseException.ERROR_CODE_20006); checkEmpty(appsig, BaseException.ERROR_CODE_10010); if (multipartFile == null) { throw new BaseException(BaseException.ERROR_CODE_20020);// 上传文件不存在 } Long uuidL = parseLong(uuid, BaseException.ERROR_CODE_20016); Integer blockIndexI = parseInt(blockIndex, BaseException.ERROR_CODE_20006); Map<String, Object> appMap = getAuthService().validateSigature(parameters); AccessToken accessToken = CasUtil.checkAccessToken(token, appMap); Long uid = accessToken.getUid(); String bucketUrl = accessToken.getBucketUrl(); // 从上传目录拷贝文件到工作目录 String fileAbsulutePath = null; try { fileAbsulutePath = this.copyFile(multipartFile.getInputStream(), multipartFile.getOriginalFilename()); } catch (IOException ioe) { log.error(ioe.getMessage(), ioe); throw new BaseException(BaseException.ERROR_CODE_20020);// 上传文件不存在 } File uploadedFile = new File(Global.UPLOAD_TEMP_DIR + fileAbsulutePath); checkEmptyFile(uploadedFile);// file 非空验证 Object rs = uploadService.upload(uuidL, blockIndexI, uid, uploadedFile, bucketUrl); setHttpStatusOk(response); return rs; } // TODO 查看下这里是否有问题 // 上传文件非空验证 private void checkEmptyFile(File file) { if (file == null || file.getAbsolutePath() == null) { throw new BaseException(BaseException.ERROR_CODE_20020);// 上传文件不存在 } } /** * 写文件到本地文件夹 * * @throws IOException * 返回生成的文件名 */ private String copyFile(InputStream inputStream, String fileName) { OutputStream outputStream = null; String tempFileName = null; int pointPosition = fileName.lastIndexOf("."); if (pointPosition < 0) {// myvedio tempFileName = UUID.randomUUID().toString();// 94d1d2e0-9aad-4dd8-a0f6-494b0099ff26 } else {// myvedio.flv tempFileName = UUID.randomUUID() + fileName.substring(pointPosition);// 94d1d2e0-9aad-4dd8-a0f6-494b0099ff26.flv } try { outputStream = new FileOutputStream(Global.UPLOAD_TEMP_DIR + tempFileName); int readBytes = 0; byte[] buffer = new byte[10000]; while ((readBytes = inputStream.read(buffer, 0, 10000)) != -1) { outputStream.write(buffer, 0, readBytes); } return tempFileName; } catch (IOException ioe) { // log.error(ioe.getMessage(), ioe); throw new BaseException(BaseException.ERROR_CODE_20020);// 上传文件不存在 } finally { if (outputStream != null) { try { outputStream.close(); } catch (IOException e) { } } if (inputStream != null) { try { inputStream.close(); } catch (IOException e) { } } } } /** * 测试此服务是否可用 * * @param response * @return * @author zwq7978 */ @RequestMapping("/core/v1/file/testServer") @ResponseBody public Object testServer(HttpServletResponse response) { setHttpStatusOk(response); return Global.SUCCESS_RESPONSE; } public UploadService getUploadService() { return uploadService; } public void setUploadService(UploadService uploadService) { this.uploadService = uploadService; } public void setAuthService(AuthService authService) { this.authService = authService; } public AuthService getAuthService() { return authService; } }
比如要上传的文件是 test450k.mp4。对照《Java 文件分块上传客户端源代码》中分块上传服务器对分块文件参数定义的名字"file",upload 方法里使用的是 MultipartFile 接收该对象。对于每次的 HTTP 请求,使用 copyFile 方法将文件流输出到服务器本地的一个临时文件夹里,比如作者的是 D:/defonds/syncPath/uploadTemp,该文件下会有 50127019-b63b-4a54-8f53-14efd1e58ada.mp4 临时文件生成用于保存上传文件流。
分块依次上传。当所有块都上传完毕之后,将这些临时文件都转移到服务器指定目录中,比如作者的这个目录是 D:/defonds/syncPath/file,在该文件夹下会有/1/temp_dir_5_1 目录生成,而 uploadTemp 的临时文件则被挨个转移到这个文件夹下,生成形如 5.part0001 的文件。以下是文件转移的源代码:
/** * 把所有块从临时文件目录移到指定本地目录或S2/S3 * * @param preUpload */ private void moveBlockFiles(BlockPreuploadFileInfo preUpload) { @SuppressWarnings("unchecked") String[] s3BlockUrl=new String[preUpload.getBlockNumber()]; String[] localBlockUrl=new String[preUpload.getBlockNumber()];//本地的块文件路径 以便以后删除 List<BlockUploadInfo> blocks = (List<BlockUploadInfo>) getBaseDao().queryForList( "upload.getBlockUploadFileByUuid", preUpload.getUuid()); String tempDirName = SyncUtil.getTempDirName(preUpload.getUuid(), preUpload.getUid()); String parentPath = Global.UPLOAD_ABSOLUTE_PAHT_ + Global.PATH_SEPARATIVE_SIGN + String.valueOf(preUpload.getUid()); String dirPath = parentPath + Global.PATH_SEPARATIVE_SIGN + tempDirName; new File(dirPath).mkdirs();//创建存放块文件的文件夹 (本地) int j=0; for (BlockUploadInfo info : blocks) { try { String strBlockIndex = createStrBlockIndex(info.getBlockIndex()); String suffixPath = preUpload.getUuid() + ".part" + strBlockIndex; String tempFilePath = info.getTempFile(); File tempFile = new File(tempFilePath); File tmpFile = new File(dirPath + suffixPath); if (tmpFile.exists()) { FileUtils.deleteQuietly(tmpFile); } FileUtils.moveFile(tempFile, tmpFile); localBlockUrl[j]=dirPath + suffixPath; j++; info.setStatus(Global.MOVED_TO_NEWDIR); getBaseDao().update("upload.updateBlockUpload", info); if (log.isInfoEnabled()) log.info(preUpload.getUuid() + " " + info.getBuId() + " moveBlockFiles"); } catch (IOException e) { log.error(e.getMessage(), e); throw new BaseException("file not found"); } } preUpload.setLocalBlockUrl(localBlockUrl); preUpload.setDirPath(dirPath); preUpload.setStatus(Global.MOVED_TO_NEWDIR); getBaseDao().update("upload.updatePreUploadInfo", preUpload); } private String createStrBlockIndex(int blockIndex) { String strBlockIndex; if (blockIndex < 10) { strBlockIndex = "000" + blockIndex; } else if (10 <= blockIndex && blockIndex < 100) { strBlockIndex = "00" + blockIndex; } else if (100 <= blockIndex && blockIndex < 1000) { strBlockIndex = "0" + blockIndex; } else { strBlockIndex = "" + blockIndex; } return strBlockIndex; }
最后是文件的组装源代码:
/** * 组装文件 * */ private void assembleFileWithBlock(BlockPreuploadFileInfo preUpload) { String dirPath = preUpload.getDirPath(); // 开始在指定目录组装文件 String uploadedUrl = null; String[] separatedFiles; String[][] separatedFilesAndSize; int fileNum = 0; File file = new File(dirPath); separatedFiles = file.list(); separatedFilesAndSize = new String[separatedFiles.length][2]; Arrays.sort(separatedFiles); fileNum = separatedFiles.length; for (int i = 0; i < fileNum; i++) { separatedFilesAndSize[i][0] = separatedFiles[i]; String fileName = dirPath + separatedFiles[i]; File tmpFile = new File(fileName); long fileSize = tmpFile.length(); separatedFilesAndSize[i][1] = String.valueOf(fileSize); } RandomAccessFile fileReader = null; RandomAccessFile fileWrite = null; long alreadyWrite = 0; int len = 0; byte[] buf = new byte[1024]; try { uploadedUrl = Global.UPLOAD_ABSOLUTE_PAHT_ + Global.PATH_SEPARATIVE_SIGN + preUpload.getUid() + Global.PATH_SEPARATIVE_SIGN + preUpload.getUuid(); fileWrite = new RandomAccessFile(uploadedUrl, "rw"); for (int i = 0; i < fileNum; i++) { fileWrite.seek(alreadyWrite); // 读取 fileReader = new RandomAccessFile((dirPath + separatedFilesAndSize[i][0]), "r"); // 写入 while ((len = fileReader.read(buf)) != -1) { fileWrite.write(buf, 0, len); } fileReader.close(); alreadyWrite += Long.parseLong(separatedFilesAndSize[i][1]); } fileWrite.close(); preUpload.setStatus(Global.ASSEMBLED); preUpload.setServerPath(uploadedUrl); getBaseDao().update("upload.updatePreUploadInfo", preUpload); if(Global.BLOCK_UPLOAD_TO!=Global.BLOCK_UPLOAD_TO_LOCAL) { //组装完毕没有问题 删除掉S2/S3上的block String[] path=preUpload.getS3BlockUrl(); for (String string : path) { try { if(Global.BLOCK_UPLOAD_TO==Global.BLOCK_UPLOAD_TO_S2) { S2Util.deleteFile(preUpload.getBucketUrl(), string); }else { S3Util.deleteFile(preUpload.getBucketUrl(), string); } } catch (Exception e) { log.error(e.getMessage(), e); } } } if (log.isInfoEnabled()) log.info(preUpload.getUuid() + " assembleFileWithBlock"); } catch (IOException e) { log.error(e.getMessage(), e); try { if (fileReader != null) { fileReader.close(); } if (fileWrite != null) { fileWrite.close(); } } catch (IOException ex) { log.error(e.getMessage(), e); } } }
BlockPreuploadFileInfo 是我们自定义的业务文件处理 bean。
OK,分块上传的服务器、客户端源代码及其工作流程至此已全部介绍完毕,以上源代码全部是经过项目实践过的,大部分现在仍运行于一些项目之中。有兴趣的朋友可以自己动手,将以上代码自行改造,看看能否运行成功。如果遇到问题可以在本博客下跟帖留言,大家一起讨论讨论。