欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页

Netty源码学习-FileRegion

程序员文章站 2022-04-22 22:21:52
...
今天看org.jboss.netty.example.http.file.HttpStaticFileServerHandler.java
可以直接往channel里面写入一个FileRegion对象,而不需要相应的encoder:


//pipeline(没有诸如“FileRegionEncoder”的handler):
public ChannelPipeline getPipeline() throws Exception {
ChannelPipeline pipeline = pipeline();
pipeline.addLast("decoder", new HttpRequestDecoder());
pipeline.addLast("aggregator", new HttpChunkAggregator(65536));
pipeline.addLast("encoder", new HttpResponseEncoder());
pipeline.addLast("chunkedWriter", new ChunkedWriteHandler());

pipeline.addLast("handler", new HttpStaticFileServerHandler());
return pipeline;
}

public class HttpStaticFileServerHandler extends SimpleChannelUpstreamHandler {
public void messageReceived...{
RandomAccessFile raf = new RandomAccessFile(file, "r");
long fileLength = raf.length();

HttpResponse response = new DefaultHttpResponse(HTTP_1_1, OK);
setContentLength(response, fileLength);
setContentTypeHeader(response, file);
setDateAndCacheHeaders(response, file);

Channel ch = e.getChannel();

// Write the initial line and the header.
ch.write(response);

// Write the content.
// No encryption - use zero-copy.
final FileRegion region =
new DefaultFileRegion(raf.getChannel(), 0, fileLength);

//直接写入FileRegion
ch.write(region);
}
}



这是为什么?往channel里面写的数据最后不是都要转成ChannelBuffer吗?

我们一步步的分析:
ch.write(region)会触发downstream事件(把region“装入”MessageEvent),
会一路经过各个handler,最后去到“sink”:


//我们以NioServerSocketPipelineSink为例:
private static void handleAcceptedSocket(ChannelEvent e) {
if (e instanceof MessageEvent) {
MessageEvent event = (MessageEvent) e;
NioSocketChannel channel = (NioSocketChannel) event.getChannel();
boolean offered = channel.writeBufferQueue.offer(event);
assert offered;
channel.worker.writeFromUserCode(channel);
}
}

//最终的写操作在AbstractNioWorker(只保留关键代码):
protected void write0(AbstractNioChannel<?> channel) {
final WritableByteChannel ch = channel.channel;
final Queue<MessageEvent> writeBuffer = channel.writeBufferQueue;
channel.currentWriteEvent = evt = writeBuffer.poll();

/*关键在这里:把FileRegion封装成一个SendBuffer,
SendBuffer的transferTo调用的是FileRegion的transferTo方法,
而这个方法调用的是FileChannel的transferTo方法:
This method is potentially much more efficient than a simple loop that reads from this channel and writes to the target channel. Many operating systems can transfer bytes directly from the filesystem cache to the target channel without actually copying them.
大体意思就是“Java NIO Channel to Channel Transfers”不需要内存复制,速度更快
*/
channel.currentWriteBuffer = buf = sendBufferPool.acquire(evt.getMessage());
buf.transferTo(ch);
}

//这里证明了,往channel写入的数据,类型既可以是ChannelBuffer,也可以是FileRegion
SendBuffer acquire(Object message) {
if (message instanceof ChannelBuffer) {
return acquire((ChannelBuffer) message);
} else if (message instanceof FileRegion) {
return acquire((FileRegion) message);
}

throw new IllegalArgumentException(
"unsupported message type: " + message.getClass());
}

private SendBuffer acquire(FileRegion src) {
if (src.getCount() == 0) {
return EMPTY_BUFFER;
}
return new FileSendBuffer(src);
}

class FileSendBuffer {
private final FileRegion file;
public long transferTo(WritableByteChannel ch) throws IOException {
long localWrittenBytes = file.transferTo(ch, writtenBytes);
writtenBytes += localWrittenBytes;
return localWrittenBytes;
}
}

class DefaultFileRegion...{
private final FileChannel file;
public long transferTo(WritableByteChannel target, long position) throws IOException {
return file.transferTo(this.position + position, count, target);
}
}




最后,记录一下java NIO对大文件的读写方法:

java.nio.channels.FileChannel的map方法可以把FileChannel“包装”成MappedByteBuffer:

public abstract MappedByteBuffer map(FileChannel.MapMode mode,
long position,
long size)

对于大文件,转成MappedByteBuffer再读写,速度更快
举例:


public class ReadingHugeFilesUsingMemoryMappedBuffer {
/**
* use a MappedByteBuffer to wrap a huge file. Using a MappedByteBuffer does
* not load the file in JVM but reads it directly off the file system
* memory. The file can be opened in read, write or private mode.
*/
// to test you can use any video movie file if you dont have any other large
// file for testing.
private static String hugeFile = "A Huge File";

public static void main(String[] args) throws IOException {
File file = new File(hugeFile);
FileChannel fileChannel = new RandomAccessFile(file, "r").getChannel();
MappedByteBuffer buffer = fileChannel.map(
FileChannel.MapMode.READ_ONLY, 0, fileChannel.size());
// the buffer now reads the file as if it were loaded in memory. note
// that for smaller files it would be faster
// to just load the file in memory
// lets see if this buffer is loaded fully into memory
System.out.println(buffer.isLoaded());
// the mappedbytebuffer can be used as a normal buffer to do read and/or
// write operations
// read the size
System.out.println(buffer.capacity());

}
}

相关标签: java netty