欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页  >  移动技术

android8.1 mtk camera hal各种操作流程

程序员文章站 2022-04-12 22:36:22
写在前面的话:从前一直以为能力比学历重要,直到最近接连面了几家大厂,技术面都过了,但最后因为本科不是全日制的,是成考的,所以都卡在hr那里的学历上。哎,想找个大平台,安安静静的做我的技术,一直做到退休,咋就这么难呢? 好了,言归正传。最近一年,一直在做android上的视频编解码和录相、以及camera hal和系统框架这一块。随着做的慢慢的深入,越发觉得mtk的camera hal这一块,有其独到之处。偏偏网上相关的资料却是极少,对新入手的人而言,很难从上到下吃透。趁着最......

最近一年,一直在做android上的视频编解码和录相、以及camera hal和系统框架这一块。随着做的慢慢的深入,越发觉得mtk的camera hal这一块,有其独到之处。偏偏网上相关的资料却是极少,对新入手的人而言,很难从上到下吃透。今天闲来无事,把最近一年来,对mtk camera hal层的理解,从上到下梳理了下。现在记录下来,希望能帮到更多的有志于mtk camera hal模块的人。

强调下,这篇博客,在自己理解的基础上,也大量参照了https://blog.csdn.net/luozirong/article/details/52244031这篇文章,有兴趣的,可以去参考一下。另外说明一下,我是家里抽了半天写的这篇文章,手头没有源代码,有些函数和文件,可能描述的不太准确,但大致思路是清晰的。

对mtk camera hal的分析,我是从open、display、preview、previewcallback、recorder、takepicture以及hal层的buff流转来着手的。虽然写的很简单,只是从上到下的调用堆栈,但是大伙只要顺着这几条线看下去,mtk camera hal 的pass2以及往上的部份,就都能理得明白了。大伙可以再配合我前几篇camera的文章来看,对mtk camera hal应该就有个比较好的把握了。好了,下面开讲了。

IImgBufProviderClient是用来管理IImgBufProvider,即让CamAdapter通过ImgBufProvidersManager管理DisplayClient的IImgBufProvider

这里的IImgBufProvider是我们DiaplayClient数据队列的管理者ImgBufQueue的父类。管理着mTodoImgBufQue和mDoneImgBufQue

上面说了ImgBufQueue继承了Provider和Process,并实现了他们的函数,其中Provider是提供给外部的类提出数据包和插入数据包,而Process是用来给内部整理和读取数据包。就如我们的DisplayClient的ImgBufQueue,DisplayClient内部生成和读取Buf通过Process系列函数,而DisplayClient把Provider系列函数提供给了CamAdapter操作数据包

CamAdapter管理了ImageSensor,ISP等硬件的初始化和调度,3A,数据的分配流向和处理等,虽然CamAdapter没有去执行Preview,Capture,Record等操作,但却管理到了他们的数据的来源和数据的动向,最后Camadapter里还包含了Buf manager和整个Camera的状态。例如在CamAdapter中Preview的管理就包括了PreviewCmdQueThread和PreviewBufMgr,PreviewCmdQueThread的线程里接收到通知后,从Sensor取数据并交由ISP处理,3A等处理后把Buf交给PreviewBufMgr进行分配到相应的操作。如果PreviewBufMgr收到的Buf需要显示,则通过ImgBufProvidersManager找到DisplayClient里的ImgBufQueue,再把Buf插入到队列里,由ImgBufQueue进行通知显示

  1. camera的打开的过程:

1.) app上通过Camera.open(0);

2.)frameworks\base\core\java\android\hardware\Camera.java里open->cameraInitNormal(cameraId)->cameraInitVersion->native_setup

3.)frameworks\base\core\jni\android_hardware_Camera.cppandroid_hardware_Camera_native_setup

4.)frameworks\av\camera\CameraBase.cppconnect

5.)frameworks\av\services\camera\libcameraservice\CameraService.cpp里的connectHelper

6.)CameraService::makeClient

7.)new CameraClient

8.)CameraClient::initialize

9.)new CameraHardwareInterface->initialize

10.)CameraHardwareInterface::initialize

11.)CameraDevice::open

12.)CameraDevice1Base::open->CameraDeviceManagerBase->startOpenDevice 在这个open函数里还调用了initHalPreviewWindow();它主要是将app上传进来的surface,传给hal层。它的定义如下

void CameraDevice1Base::initHalPreviewWindow()

{

mHalPreviewWindow.cancel_buffer = sCancelBuffer;

mHalPreviewWindow.lock_buffer = sLockBuffer;

mHalPreviewWindow.dequeue_buffer = sDequeueBuffer;

mHalPreviewWindow.enqueue_buffer = sEnqueueBuffer;

mHalPreviewWindow.set_buffer_count = sSetBufferCount;

mHalPreviewWindow.set_buffers_geometry = sSetBuffersGeometry;

mHalPreviewWindow.set_crop = sSetCrop;

mHalPreviewWindow.set_timestamp = sSetTimestamp;

mHalPreviewWindow.set_usage = sSetUsage;

mHalPreviewWindow.set_swap_interval = sSetSwapInterval;


mHalPreviewWindow.get_min_undequeued_buffer_count =

sGetMinUndequeuedBufferCount;

}


  1. camera设置preview窗口的过程
  1. )app上调用camera.setPreviewDisplay(SurfaceHolder);
  2. )frameworks/base/core/java/android/hardware/Camera.java里setPreviewDisplay

3.) Camera.java里setPreviewSurface(holder.getSurface());

4.) frameworks\base\core\jni\android_hardware_Camera.cppandroid_hardware_Camera_setPreviewSurface

5.) frameworks/av/camera/Camera.cpp里Camera::setPreviewTarget(const sp<IGraphicBufferProducer>& bufferProducer)

6.) CameraClient::setPreviewTarget( const sp<IGraphicBufferProducer>& bufferProducer)

7.)CameraClient::setPreviewWindow(const sp<IBinder>& binder, const sp<ANativeWindow>& window)

8.)CameraHardwareInterface::setPreviewWindow(const sp<ANativeWindow>& buf)

9.)CameraDevice.cpp::setPreviewWindow

10.)CameraDevice1Base.cpp::setPreviewWindow

11.)CameraDevice1Base.cpp::initDisplayClient

12.)DisplayClient::setWindow()

13.)DisplayClient:: set_preview_stream_ops() ----> mpStreamOps = window


  1. 预览的过程
  1. )紧跟着上面一部份,在)CameraDevice1Base.cpp::initDisplayClient
  2. )IDisplayClient::createInstance()
  3. )DisplayClient::init()
  4. )createDisplayThread()&& createImgBufQueue()
  5. )cameraDevice1Base::enableDisplayClient()
  6. )DisplayClient::enableDisplay()
  7. )DisplayClient::enableDisplay()-
  8. ) mpDisplayThread-> postCommand(Command(Command::eID_WAKEUP))
  9. )DisplayThread::threadLoop()
  10. )DisplayClient::onThreadLoop()
  11. )DisplayClient::waitAndHandleReturnBuffers
  12. )rpBufQueue->dequeProcessor(vQueNode)
  13. )DisplayClient::handleReturnBuffers()
  14. )enquePrvOps()
  15. )mpStreamOps->enqueue_buffer(mpStreamOps,rpImgBuf->getBufHndlPtr()) (mpStreamOps这个就是app上传下来的surface)

如果把上面enqueue_buffer这一部份给删除,则预览界面无显示. 所以总体来说是在DisplayClient的init的时候创建了一个线程:

mpDisplayThread =IDisplayThread::createInstance(this)

一个队列:

mpImgBufQueue = newImgBufQueue(IImgBufProvider::eID_DISPLAY, "CameraDisplay@ImgBufQue");

然后mpDisplayThread等待ImgBufQueue有数据的时候通过dequeProcessor取到要渲染的数据,交给mpStreamOps也就是preview_stream_ops对象,其实preview_stream_ops只是对ANativeWindow的简单封装,最后调用的其实是ANativeWindow的queueBuffer函数


  1. 录相的过程
  1. )CameraClient::startRecordingMode
  2. )CameraHardwareInterface::startRecording()

3.)vendor\mediatek\proprietary\hardware\mtkcam\main\hal\device\1.x\device\CameraDevice.cpp

mpDevice->startRecording();

4.)CameraDevice1Base::startRecording()->mpCamClient->startRecording

5.)vendor\mediatek\proprietary\hardware\mtkcam\middleware\v1\client\CamClient\CamClient::startRecording()->mpRecordClient->startRecording()

6.)vendor\mediatek\proprietary\hardware\mtkcam\middleware\v1\client\CamClient\Record\RecordClient.cpp


RecordClient::startRecording()


7.) RecordClient::
onStateChanged()->postCommand(Command(Command::eID_WAKEUP));

8.)RecordClient.Thread.cpp里的
threadLoop()

9.)RecordClient.Thread.cpp里的onClientThreadLoop, prepareAllTodoBuffers->pBufQueue->startProcessor()->waitAndHandleReturnBuffers(pBufQueue, blocking)

在onClientThreadLoop函数里,通过prepareAllTodoBuffers去准备好接收收据的buff,prepareAllTodoBuffers的第一个参数就是在init里初始化的mpImgBufQueue, 在这里赋值给了pBufQueue,在prepareAllTodoBuffers里, 通过enqueProcessor将当前的帧放到了mpImgBufQueue 里。

再在onClientThreadLoop里, 调用pBufQueue->startProcessor() 去通知处理。      然后再通过waitAndHandleReturnBuffers去处理这些buff.这个函数里,通过rpBufQueue->dequeProcessor(vQueNode);来取刚刚得到的帧buff,最后丢给handleReturnBuffers处理理。     waitAndHandleReturnBuffers定义在vendor\mediatek\proprietary\hardware\mtkcam\middleware\v1\client\CamClient\Record\RecordClient.BufOps.cpp里. 在waitAndHandleReturnBuffers中,通过rpBufQueue->dequeProcessor(vQueNode);,去获取当前帧,然后通过handleReturnBuffers去处理, 最后取到帧后,会丢给performRecordCallback去回调给app


IImgBufProvider的设置过程:

DisplayClient::enableDisplay ()---->DisplayClient::setImgBufProviderClient (mpCamAdapter)----> IImgBufProviderClient::onImgBufProviderCreated(mpImgBufQueue) ---->BaseCamAdapter::

onImgBufProviderCreated(sp<IImgBufProvider>const&rpProvider) ----> ImgBufProvidersManager::setProvider()

Camera的数据来源

DefaultCam1Device::onStartPreview()----> Cam1DeviceBase::initCameraAdapter() ----> CamAdapter::init() ---->IPreviewCmdQueThread::createInstance() ----> CamAdapter::startPreview() ---->StateIdle::onStartPreview() ----> CamAdapter::onHandleStartPreview() ---->mpPreviewCmdQueThread->postCommand(PrvCmdCookie::eUpdate, PrvCmdCookie::eSemBefore)----> PreviewCmdQueThread::threadLoop()----> PreviewCmdQueThread::update()----> PreviewCmdQueThread::updateOne() ----> PreviewBufMgr::enqueBuffer()----> IImgBufProvider:: enqueProvider()

5.Camera预览数据回调流程

api1去预览,并且在app上通过setPreviewCallBack设置预览回调,想获取预览数据的流程如下:

  1. )app调用Camera.java的startPreview, setPreviewCallback
  2. )android_hardware_Camera.cpp里的android_hardware_Camera_startPreview
  3. )Camera.cpp Camera::startPreview()
  4. )CameraClient::startPreview()
  5. CameraClient::startCameraMode
  6. CameraClient::startPreviewMode()
  7. CameraHardwareInterface::startPreview
  8. )CameraDevice::startPreview

{

//(1) 此函数在其子类DefaultCam1Device中实现

//仅是初始化了CameraAdapter

onStartPreview();


//(2) 初始化DisplayClient,重要,稍后研究

enableDisplayClient();

//(3)

mpCamClient->startPreview();

//(4) 我们通过(3)和(4)开始研究,再返回去看(1)(2)

mpCamAdapter->startPreview();


enableMsgType(CAMERA_MSG_PREVIEW_METADATA);

//

mIsPreviewEnabled = true;

}

  1. )CamClient::startPreview()里的mpCamclient->startPreview()
  2. )PreviewClient::startPreview(),这个里面会去调用initBuffers();

PreviewClient::startPreview()

{

//获得预览参数,这里参数为800*480,yuv420sp

ms8PrvTgtFmt = mpParamsMgr->getPreviewFormat();

mpParamsMgr->getPreviewSize(&mi4PrvWidth, &mi4PrvHeight);

//初始化预览Buf

initBuffers();

//

return onStateChanged();

}


PreviewClient::initBuffers()

{

//预览数据的Buf

mpImgBufMgr = ImgBufManager::alloc(ms8PrvTgtFmt, mi4PrvWidth,

mi4PrvHeight, eMAX_PREVIEW_BUFFER_NUM,

"PreviewClientCb", mpCamMsgCbInfo->mRequestMemory,

0, 0);


//预览数据的处理类,这里只是保留了一个处理接口

//里面并没有什么东西,可自行填充

mpExtImgProc = ExtImgProc::createInstance();

mpExtImgProc->init();

}


PreviewClient::onStateChanged()

{ //发送了一个eID_WAKEUP的消息

postCommand(Command(Command::eID_WAKEUP));

}


//接收eID_WAKEUP消息

PreviewClient::threadLoop()

{

Command cmd;

if ( getCommand(cmd) )

{

switch (cmd.eId)

{

case Command::eID_WAKEUP:

case Command::eID_PREVIEW_FRAME:

case Command::eID_POSTVIEW_FRAME:

onClientThreadLoop(cmd);

break;

//

case Command::eID_EXIT:

//...

}

}

  1. )PreviewClient::onClientThreadLoop开始处理数据Buf
  2. )PreviewClient::waitAndHandleReturnBuffers(sp<IImgBufQueue>const& rpBufQueue)
  3. )PreviewClient::handleReturnBuffers(Vector<ImgBufQueNode>const& rvQueNode). 这个函数会调用到performPreviewCallback
  4. )PreviewClient::performPreviewCallback(sp<ICameraImgBuf>const& pCameraImgBuf, int32_t const msgType)。这个函数里,会调用到pCamMsgCbInfo->mDataCb函数。这个函数,就是在CameraClient.cpp的initialize()函数里,通过mHardware->setCallbacks(notifyCallback, dataCallback, dataCallbackTimestamp, (void *)mCameraId);这句话其中的dataCallback,设置的mDataCb函数。也就是app上调用这句话其中的dataCallback,设置了mDataCb函数传下来的回调函数。这里会将数据传到app层上去。

6.拍照的流程:

CamAdapter::takePicture()-> IState::onCapture() ->IStateHandler::onHandleCapture() ->CamAdapter::onHandleCapture()CamAdapter::onHandleCapture() ->CaptureCmdQueThread::onCapture() -> CaptureCmdQueThread::threadLoop() ->CamAdapter::onCaptureThreadLoop()->CapBufShot::sendCommand()->CapBufShot::onCmd_capture()-> SingleShot::startOne()在这里组装IImgBuffer

7.拍照回调的流程:

SingleShot::startOne() --> CamShotImp::handleDataCallback() --> CamShotImp::onDataCallback--> CapBufShot::fgCamShotDataCb --> CapBufShot::handleJpegData  -->CamAdapter::onCB_RawImage -->mpCamMsgCbInfo->mDataCb

##注意两个两个标红的和标蓝的,他们这样就头尾对起来了。


1.所谓的Camera主要就是从摄像头取一点数据放到LCD上显示,所以打蛇打七寸,我们首先介绍HAL层的Window对象是mpStrwamOps。而这个对象被设置的流程是:

Cam1DeviceBase::setPreviewWindow()  --->  initDisplayClient()  ---> DisplayClient::setWindow()  ---> set_preview_stream_ops()  ---> mpStreamOps = window

2.另一个面我们看下数据的获取流程,Camera HAL层数据主要通过ImgBufQueue对象传递,而这个对象最开始的赋值过程如下:

CamAdapter::takePicture() -> IState::onCapture() ->IStateHandler::onHandleCapture() -> CamAdapter::onHandleCapture()CamAdapter::onHandleCapture() -> CaptureCmdQueThread::onCapture() -> CaptureCmdQueThread::threadLoop() -> CamAdapter::onCaptureThreadLoop()->CapBufShot::sendCommand()->CapBufShot::onCmd_capture() -> SingleShot::startOne()在这里组装IImgBuffer




8.)CamAdapter, pass1分发到pass2的过程

1.)我们从DefaultCam1Device.cpp开始看起, DefaultCam1Device::onStartPreview()

2.)Cam1DeviceBase::initCameraAdapter()

{

....

mpCamAdapter = ICamAdapter::createInstance(mDevName, mi4OpenId, mpParamsMgr);

....

// (.4) [DisplayClient] set Image Buffer Provider Client if needed.

mpDisplayClient->setImgBufProviderClient(mpCamAdapter);

// (.5) [CamClient] set Image Buffer Provider Client if needed.

mpCamClient->setImgBufProviderClient(mpCamAdapter);

}

3.)ICamAdapter::createInstance会调到MtkDefaultCamAdapter.cpp里的

CamAdapter::init()

{

//PreviewBufMgr

mpPreviewBufMgr = IPreviewBufMgr::createInstance(mpImgBufProvidersMgr);

//PreviewCmdQueThread,注意第一个参数为PreviewBufMgr

mpPreviewCmdQueThread = IPreviewCmdQueThread::createInstance(mpPreviewBufMgr, getOpenId(), mpParamsMgr);

mpPreviewCmdQueThread->run();


//CaptureCmdQueThread

mpCaptureCmdQueThread = ICaptureCmdQueThread::createInstance(this);

mpCaptureCmdQueThread->run();


//就是相机的3个Auto,自动曝光,自动对焦,自动白平衡

init3A();

}

4.)CamAdapter::startPreview()

5.)state.cpp里的StateIdle::onStartPreview(IStateHandler* pHandler)

6.)CamAdapter::onHandleStartPreview()里会调用mpPreviewCmdQueThread->postCommand(PrvCmdCookie::eStart, PrvCmdCookie::eSemAfter); ”

7.)PreviewCmdQueThread.cpp PreviewCmdQueThread::threadLoop()

{

....

case PrvCmdCookie::eStart:

isvalid = start();

break;

....

}

8.)PreviewCmdQueThread::start()

{

....

for (int32_t i = 0; i < PASS1BUFCNT; i++)

{

//取出刚才创建的Buf

mspPreviewBufHandler->dequeBuffer(eID_Pass1Out, Pass1Node);

//获取bufinfo

mapNode2BufInfo(eID_Pass1Out, Pass1Node, BufInfo);

//收集bufinfo

vBufPass1Out.push_back(BufInfo);

}

......

预览

if (flag & eID_Pass2DISPO)

{

dispNode.getImgBuf()->setTimestamp(pass1LatestTimeStamp);

//通知Client的接收队列,进行接收处理

mspPreviewBufHandler->enqueBuffer(dispNode);

}

//录相

if (flag & eID_Pass2VIDO)

{

vidoNode.getImgBuf()->setTimestamp(pass1LatestTimeStamp);

mspPreviewBufHandler->enqueBuffer(vidoNode);

}

}

9.)PreviewBufMgr::enqueBuffer(ImgBufQueNode const& node)

{

....

//这里将从pass1取出来的数据分发给pass2

case eBuf_Disp:

{

//从Provider的队列中找到Id为eID_DISPLAY的Provider,

sp<IImgBufProvider> bufProvider = mspImgBufProvidersMgr->getDisplayPvdr();

/**

在DisplayClient::waitAndHandleReturnBuffers()中,调用了ImgBufQueue.cpp dequeProcessor()

dequeProcessor()里mDoneImgBufQueCond.wait()会一直在等待。直到下面的函数被调用了

就会把buf传进去,并 mDoneImgBufQueCond.broadcast();通知接收buf

**/

bufProvider->enqueProvider(node);

}

break;

//...

case eBuf_Rec:

....

}

看到这里验证了我们之前的猜想,CamAdapter管理了ImageSensor,ISP等硬件的初始化和调度,3A,数据的分配流向和处理等,虽然CamAdapter没有去执行Preview,Capture,Record等操作,但却管理到了他们的数据的来源和数据的动向,最后Camdapter里还包含了Buf manager和整个Camera的状态。例如在CamAdapter中Preview的管理就包括了PreviewCmdQueThread和PreviewBufMgr,PreviewCmdQueThread的线程里接收到通知后,从Sensor取数据并交由ISP处理,3A等处理后把Buf交给PreviewBufMgr进行分配到相应的操作。如果PreviewBufMgr收到的Buf需要显示,则通过ImgBufProvidersManager找到DisplayClient里的ImgBufQueue,再把Buf插入到队列里,由ImgBufQueue进行通知显示

本文地址:https://blog.csdn.net/xuhui_7810/article/details/107774427