手机系统与相机的工作原理（详细介绍Camera API2 HAL3）

OVERVIEW

目录：

思路：
1.APP做了啥
2.Framework做了啥
3.hal做了啥
4.kernel做了啥
5.这些部分中间做了啥

Let’s Start！

前言 基本概念简介

IPC Inter-Process Communication

cameraserver 26354 1 32880 14144 binder_ioctl_write_read 0 S cameraserver

cameraserver 26355 1 250508 57096 binder_ioctl_write_read 0 S camerahalserver

u0_a56 30751 409 4438900 134716 SyS_epoll_wait 0 S com.mediatek.camera

RPC Remote Process Communication

AIDL Android Interface Description Language

与 CameraService 关联的 Binder 接口可在 frameworks/av/camera/aidl/android/hardware 中找到。 生成的代码会调用较低级别的原生代码以获取对实体相机的访问权限，并返回用于在框架级别创建 CameraDevice 并最终创建 CameraCaptureSession 对象的数据。

安卓接口描述语言，用于Framework层的java代码访问cpp代码的接口描述，高大上的说就是进程间通信描述。

HIDL Hardware Interface Description Language

HIDL 是用于指定 HAL 与其用户之间接口的一个接口描述语言（Interface Description Language），它允许将指定的类型与函数调用收集到接口（Interface）和包（Package）中。更广泛地说，HIDL 是一个可以让那些独立编译的代码库（Libraries）之间进行通信的系统。
HIDL 实际上是用于进行进程间通信（Inter-process Communication，IPC）的。进程间的通信可以称为 Binder 化（Binderized）。对于必须连接到进程的库，也可以使用 passthough 模式（但在Java中不支持）。

设计 HIDL 这个机制的目的，主要是想把框架（framework）与 HAL 进行隔离，使得框架部分可以直接被覆盖、更新，而不需要重新对 HAL 进行编译。HAL 的部分将会放在设备的 /vendor 分区中，并且是由设备供应商（vendors）或 SOC 制造商来构建。这使得框架部分可以通过 OTA 方式更新，同时不需要重新编译 HAL。来源:https://source.android.com/devices/architecture/hidl

硬件接口描述语言,就是cpp的代码与cpp的代码沟通的接口描述，在mtk平台上，cameraserver和camerahalserver的沟通就是通过hidl描述的

Binder

binder通信，camera中进程间通信大都是这样完成的，这才是真正的传送信息的工具，可以理解为电话线，AIDL和HIDL是大家约定好的电话协议

安卓camera架构

mtk hal3架构

一 手机开机的时候发生了什么?

1.kernel在干啥

kernel先起来，kernel需要注册设备驱动，以及创建ioctl接口，再启动hal层和framework层的服务结束

等hal启动后再调用kernel接口扫描sensor

2.hal和framework在干啥

kernel层后会启动cameraserver和camerahalserver，如下图：

左边的main_cameraserver.cpp是Framework层的，右边的service.cpp是hal层的

抓log可以看出hal层的server先起来：

adb logcat | grep -E "camerahalserver|ServiceMana"
03-09 10:13:45.429   374   374 I ServiceManager: service 'media.camera' died
03-09 10:13:45.593 26271 26271 I camerahalserver: Camera HAL Server is starting..., ADV_CAM_SUPPORT(0)
03-09 10:13:45.834 26270 26270 I cameraserver: ServiceManager: 0xee61c160 

HAL层注册ICameraProvider后停滞此处等待Framework层来调用

Service.cpp (z:\mnt\data\projects\zal1885\vendor\mediatek\proprietary\hardware\mtkcam3\main\hal\service)	6133	12/28/2018
int main()
{
ALOGI("Camera HAL Server is starting..., ADV_CAM_SUPPORT(%d)", MTKCAM_ADV_CAM_SUPPORT);

    // The camera HAL may communicate to other vendor components via
    // /dev/vndbinder
    android::ProcessState::initWithDriver("/dev/vndbinder");
    configureRpcThreadpool(16, true /*callerWillJoin*/);
    //  AOSP ICameraProvider HAL Interface
    {
        using android::hardware::camera::provider::V2_4::ICameraProvider;
        registerPassthroughServiceImplementation<ICameraProvider>("internal/0" /*"internal" for binderized mode*/);
    }
} 

framework层的cameraserver服务启动：

/frameworks/av/camera/cameraserver/cameraserver.rc
service cameraserver /system/bin/cameraserver
    class main
    user cameraserver
    group audio camera input drmrpc
    ioprio rt 4
    writepid /dev/cpuset/camera-daemon/tasks /dev/stune/top-app/tasks

Main_cameraserver.cpp (z:\mnt\data\projects\zal1885\frameworks\av\camera\cameraserver)	1223	12/28/2018
int main(int argc __unused, char** argv __unused)
{
    signal(SIGPIPE, SIG_IGN);

    // Set 3 threads for HIDL calls
    hardware::configureRpcThreadpool(3, /*willjoin*/ false);

    sp<ProcessState> proc(ProcessState::self());
    sp<IServiceManager> sm = defaultServiceManager();
    ALOGI("ServiceManager: %p", sm.get());
    CameraService::instantiate();
    ProcessState::self()->startThreadPool();
    IPCThreadState::self()->joinThreadPool();
} 

这里的cameraservice::instantiate()会去调用cameraservice.cpp里面的初始化

CameraService.cpp (z:\mnt\data\projects\zal1885\frameworks\av\services\camera\libcameraservice)	115317	12/28/2018
void CameraService::onFirstRef()
{
    ALOGI("CameraService process starting");

    BnCameraService::onFirstRef();

    // Update battery life tracking if service is restarting
    BatteryNotifier& notifier(BatteryNotifier::getInstance());
    notifier.noteResetCamera();
    notifier.noteResetFlashlight();

    status_t res = INVALID_OPERATION;

    res = enumerateProviders();
    if (res == OK) {
        mInitialized = true;
    }

    CameraService::pingCameraServiceProxy();

    mUidPolicy = new UidPolicy(this);
    mUidPolicy->registerSelf();
} 

这里最主要调用的是enumerateProviders()函数

CameraService.cpp (z:\mnt\data\projects\zal1885\frameworks\av\services\camera\libcameraservice)	115317	12/28/2018
status_t CameraService::enumerateProviders() {
    status_t res;

    std::vector<std::string> deviceIds;
    {
        Mutex::Autolock l(mServiceLock);

        if (nullptr == mCameraProviderManager.get()) {
            mCameraProviderManager = new CameraProviderManager();
            res = mCameraProviderManager->initialize(this);
            if (res != OK) {
                ALOGE("%s: Unable to initialize camera provider manager: %s (%d)",
                        __FUNCTION__, strerror(-res), res);
                return res;
            }
        }


        // Setup vendor tags before we call get_camera_info the first time
        // because HAL might need to setup static vendor keys in get_camera_info
        // TODO: maybe put this into CameraProviderManager::initialize()?
        mCameraProviderManager->setUpVendorTags();

        if (nullptr == mFlashlight.get()) {
            mFlashlight = new CameraFlashlight(mCameraProviderManager, this);
        }

        res = mFlashlight->findFlashUnits();
        if (res != OK) {
            ALOGE("Failed to enumerate flash units: %s (%d)", strerror(-res), res);
        }

        deviceIds = mCameraProviderManager->getCameraDeviceIds();
    }


    for (auto& cameraId : deviceIds) {
        String8 id8 = String8(cameraId.c_str());
        onDeviceStatusChanged(id8, CameraDeviceStatus::PRESENT);
    }

    return OK;
} 

res = mCameraProviderManager->initialize(this);初始化cameraprovider;

初始化的时候会调用mServiceProxy->getService(newProvider);拿到接口代理;

拿到hal层接口，再去创建providerInfo实例sp providerInfo =new ProviderInfo(newProvider, interface, this);

最后再初始化providerInfo: status_t res = providerInfo->initialize();

status_t CameraProviderManager::initialize(wp<CameraProviderManager::StatusListener> listener,
        ServiceInteractionProxy* proxy) {
    std::lock_guard<std::mutex> lock(mInterfaceMutex);
    if (proxy == nullptr) {
        ALOGE("%s: No valid service interaction proxy provided", __FUNCTION__);
        return BAD_VALUE;
    }
    mListener = listener;
    mServiceProxy = proxy;

    // Registering will trigger notifications for all already-known providers
    bool success = mServiceProxy->registerForNotifications(
        /* instance name, empty means no filter */ "",
        this);
    if (!success) {
        ALOGE("%s: Unable to register with hardware service manager for notifications "
                "about camera providers", __FUNCTION__);
        return INVALID_OPERATION;
    }

    // See if there's a passthrough HAL, but let's not complain if there's not
    addProviderLocked(kLegacyProviderName, /*expected*/ false);
    addProviderLocked(kExternalProviderName, /*expected*/ false);

    return OK;
}
调用到
status_t CameraProviderManager::addProviderLocked(const std::string& newProvider, bool expected) {
...
    sp<provider::V2_4::ICameraProvider> interface;
    interface = mServiceProxy->getService(newProvider);
...
    sp<ProviderInfo> providerInfo =
            new ProviderInfo(newProvider, interface, this);
    status_t res = providerInfo->initialize();
    if (res != OK) {
        return res;
    }

    mProviders.push_back(providerInfo);

    return OK;
} 

status_t CameraProviderManager::ProviderInfo::initialize() {
...
// cameraDeviceStatusChange callbacks may be called (and causing new devices added)
    // before setCallback returns
    hardware::Return<Status> status = mInterface->setCallback(this); 

前面在拿HAL层接口的时候，代理会帮忙我们干一系列事情最终会调用到： HIDL_FETCH_ICameraProvider()

这个函数会新建hal层的cameraprovider对象
这个函数通过ioctl给kernel下指令probe sensor

Instance.cpp (z:\mnt\data\projects\zal1885\vendor\mediatek\proprietary\hardware\mtkcam3\main\hal\devicemgr\depend)	6003	12/28/2018
extern "C"
ICameraProvider*
HIDL_FETCH_ICameraProvider(const char* name)
{
    //  name must be either "internal/<id>" or "legacy/<id>".
    std::string const strProviderName(name);
    size_t const pos = strProviderName.find('/');
    if ( 0 == pos || std::string::npos == pos ) {
        MY_LOGE("provider name (%s) with bad \'/\' at position %zu", name, pos);
        return nullptr;
    }
    //
    if ( 0 != strProviderName.compare(0, pos, getProviderType()) ) {
        MY_LOGW("provider name (%s) with mismatched type(%s) and \'/\' at position %zu",
            name, getProviderType().c_str(), pos);
        return nullptr;
    }
    //
    return createICameraProvider_V2_4(name, getCameraDeviceManager());
}
extern "C"
NSCam::ICameraDeviceManager*
getCameraDeviceManager()
{
    static NSCam::CameraDeviceManagerImpl singleton(getProviderType().c_str());
    static bool init = singleton.initialize();
    if ( ! init ) {
        MY_LOGE("CameraDeviceManagerImpl::initialize fail %p", &singleton);
        return nullptr;
    }
    return &singleton;
} 

这里的initialize()后来会调用enumerateDevicesLocked来搜索sensor
-> enumerateDevicesLocked()
-> onEnumerateDevicesLocked()
-> pHalDeviceList->searchDevices()

CameraDeviceManagerBase.cpp (z:\mnt\data\projects\zal1885\vendor\mediatek\proprietary\hardware\mtkcam3\main\hal\devicemgr)	21908	12/28/2018
auto
CameraDeviceManagerBase::
initialize() -> bool
{
    CAM_TRACE_NAME(LOG_TAG ":initialize");

    // global vendor tags should be setup before enumerating devices...
    auto pVendorTagDesc = NSCam::getVendorTagDescriptor();
...

    // loading libraries in charge of creating devices.
    auto loadDeviceFactory = [](char const* libname, char const* symbol) {
        VirtEnumDeviceFactory item;
        item.mLibHandle = ::dlopen(libname, RTLD_NOW);
...
        *(void **)(&item.mCreateVirtualCameraDevice) = ::dlsym(item.mLibHandle, symbol);
...
        return item;
    };
    mVirtEnumDeviceFactoryMap[3] = loadDeviceFactory("libmtkcam_device3.so", "createVirtualCameraDevice");

    // enumerating devices...
    status_t status = OK;
    MY_LOGI("+");
    RWLock::AutoWLock _l(mDataRWLock);
    {
        status = enumerateDevicesLocked();
    }
    MY_LOGI("-");
    return (OK==status);
} 

CameraDeviceManagerImpl.cpp (z:\mnt\data\projects\zal1885\vendor\mediatek\proprietary\hardware\mtkcam3\main\hal\devicemgr\depend)	11989	12/28/2018
auto
CameraDeviceManagerImpl::
onEnumerateDevicesLocked() -> ::android::status_t
{
...
    //IHalSensorList*const pHalSensorList = MAKE_HalSensorList();
    //size_t const sensorNum = pHalSensorList->searchSensors();
    ///
    IHalLogicalDeviceList* pHalDeviceList;
    pHalDeviceList = MAKE_HalLogicalDeviceList();//IHalLogicalDeviceList::get();
    size_t const deviceNum = pHalDeviceList->searchDevices();
    ///
    CAM_LOGI("pHalDeviceList:%p searchDevices:%zu queryNumberOfDevices:%d", pHalDeviceList, deviceNum, pHalDeviceList->queryNumberOfDevices());

    mVirtEnumDeviceMap.setCapacity(deviceNum*2);
    for (size_t instanceId = 0; instanceId < deviceNum; instanceId++)
    {
        //
        sp<IMetadataProvider> pMetadataProvider;
        pMetadataProvider = IMetadataProvider::create(instanceId);
        NSMetadataProviderManager::add(instanceId, pMetadataProvider.get());
        MY_LOGD("[0x%02zx] IMetadataProvider:%p sensor:%s", instanceId, pMetadataProvider.get(), pHalDeviceList->queryDriverName(instanceId));
        addVirtualDevicesLocked(instanceId, pMetadataProvider);
    }

    size_t const sensorNum = pHalDeviceList->queryNumberOfSensors();
    IHalSensorList*const pHalSensorList = MAKE_HalSensorList();
    mPhysEnumDeviceMap.setCapacity(sensorNum);
    for (size_t sensorId = 0; sensorId < sensorNum; sensorId++)
    {
        sp<PhysEnumDevice> pPhysDevice = new PhysEnumDevice;
        sp<IMetadataProvider> pMetadataProvider = NSMetadataProviderManager::valueFor(sensorId);
        pPhysDevice->mMetadataProvider   = pMetadataProvider;
        pPhysDevice->mSensorName         = pHalSensorList->queryDriverName(sensorId);
        pPhysDevice->mInstanceId         = sensorId;
        pPhysDevice->mFacing             = pMetadataProvider->getDeviceFacing();
        pPhysDevice->mWantedOrientation  = pMetadataProvider->getDeviceWantedOrientation();
        pPhysDevice->mSetupOrientation   = pMetadataProvider->getDeviceSetupOrientation();
        pPhysDevice->mHasFlashUnit       = pMetadataProvider->getDeviceHasFlashLight();

        mPhysEnumDeviceMap.add(sensorId, pPhysDevice);
    }
    return OK;
} 

这里 pHalDeviceList->searchDevices();

会调用-> HalLogicalDeviceList.cpp的createDeviceMap();

-> size_t const sensorNum = pHalSensorList->searchSensors();

-> HalSensorList.cpp的 enumerateSensor_Locked();

-> ret = pSensorDrv->searchSensor((IMGSENSOR_SENSOR_IDX)i)) == SENSOR_NO_ERROR

-> Imgsensor_drv.cpp 的searchSensor()

-> GetSensorInitFuncList(&pSensorInitFunc);

去拿imgsensor_src/sensorlist.cpp下bringup的时候的填写的sensorlist拿到后一个个去probe

featureControl(sensorIdx, SENSOR_FEATURE_SET_DRIVER, (MUINT8 *)&idx, &featureParaLen);

MUINT
HalSensorList::
enumerateSensor_Locked()
{
    int ret = 0;
    NSFeature::SensorInfoBase* pSensorInfo ;

    SensorDrv *const pSensorDrv = SensorDrv::get();
    SeninfDrv *const pSeninfDrv = SeninfDrv::createInstance();
...

    if((ret = pSeninfDrv->init()) < 0) {
        MY_LOGE("pSeninfDrv->init() fail");
        return 0;
    }

    /*search sensor using 8mA driving current*/
    pSeninfDrv->setAllMclkOnOff(ISP_DRIVING_8MA, TRUE);

    pSensorDrv->init();

#ifdef MTK_SUB2_IMGSENSOR
    for (MUINT i = IMGSENSOR_SENSOR_IDX_MIN_NUM; i <= IMGSENSOR_SENSOR_IDX_SUB2; i++) {
        MY_LOGD("impSearchSensor search to sub2\n");

#else
#ifdef MTK_MAIN2_IMGSENSOR
    for (MUINT i = IMGSENSOR_SENSOR_IDX_MIN_NUM; i <= IMGSENSOR_SENSOR_IDX_MAIN2; i++) {
        MY_LOGD("impSearchSensor search to main_2\n");
#else
#ifdef MTK_SUB_IMGSENSOR
    for (MUINT i = IMGSENSOR_SENSOR_IDX_MIN_NUM; i <= IMGSENSOR_SENSOR_IDX_SUB; i++) {
        MY_LOGD("impSearchSensor search to sub\n");
#else
    for (MUINT i = IMGSENSOR_SENSOR_IDX_MIN_NUM; i < IMGSENSOR_SENSOR_IDX_SUB; i++) {
        MY_LOGD("impSearchSensor search to main\n");
#endif
#endif
#endif

        if((ret = pSensorDrv->searchSensor((IMGSENSOR_SENSOR_IDX)i)) == SENSOR_NO_ERROR) {
            //query sensorinfo
            querySensorDrvInfo((IMGSENSOR_SENSOR_IDX)i);
            //fill in metadata
            buildSensorMetadata((IMGSENSOR_SENSOR_IDX)i);

            pSensorInfo = pSensorDrv->getSensorInfo((IMGSENSOR_SENSOR_IDX)i);
            addAndInitSensorEnumInfo_Locked(
                (IMGSENSOR_SENSOR_IDX)i,
                mapToSensorType(pSensorInfo->GetType()),
                pSensorInfo->getDrvMacroName());
        }
    }
...
    return  mEnumSensorList.size();
} 

ret = pSensorDrv->searchSensor((IMGSENSOR_SENSOR_IDX)i)) == SENSOR_NO_ERROR

这里开始下了searchsensor的指令后，开始对porting好几个camera进行配置，
对第一颗兼容的camera进行上电–>读取camera的ID–>匹配ID–>匹配上挂载上，匹配不上就去下一颗循环一直到兼容列表跑完

至此camera启动初始化完成

二 打开camera做了哪些事情

• 返回主目录
  
  来看一张图：
  

1.APP做了啥

CameraManager manager = (CameraManager) activity.getSystemService(Context.CAMERA_SERVICE);

...
CameraCharacteristics characteristics
                = manager.getCameraCharacteristics(cameraId);
...
 manager.openCamera(mCameraId, mStateCallback, mBackgroundHandler);
...
private final CameraDevice.StateCallback mStateCallback = new CameraDevice.StateCallback() {

    public void onOpened(@NonNull CameraDevice cameraDevice) {
        // This method is called when the camera is opened.  We start camera preview here.
        mCameraOpenCloseLock.release();
        mCameraDevice = cameraDevice;  //返回CameraDevice对象引用
        createCameraPreviewSession();
    }

    public void onDisconnected(@NonNull CameraDevice cameraDevice) {
        mCameraOpenCloseLock.release();
        cameraDevice.close();
        mCameraDevice = null;
    }

    public void onError(@NonNull CameraDevice cameraDevice, int error){}
} 

拿到cameradevice对象后开始创建camera会话：mCameraDevice.createCaptureSession

创建会话的时候会回调onConfigured函数获得cameraCaptureSession对象

创建会话后，立马准备请求预览数据给到surface显示: mCaptureSession.setRepeatingRequest{mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW),callback,handle}

private void createCameraPreviewSession() {
        try {
...
            // We configure the size of default buffer to be the size of camera preview we want.
            texture.setDefaultBufferSize(mPreviewSize.getWidth(), mPreviewSize.getHeight());

            // This is the output Surface we need to start preview.
            Surface surface = new Surface(texture);

            // We set up a CaptureRequest.Builder with the output Surface.
            mPreviewRequestBuilder
                    = mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
            mPreviewRequestBuilder.addTarget(surface);

            // Here, we create a CameraCaptureSession for camera preview.
            mCameraDevice.createCaptureSession(Arrays.asList(surface, mImageReader.getSurface()),
                    new CameraCaptureSession.StateCallback() {

            @Override
            public void onConfigured(@NonNull CameraCaptureSession cameraCaptureSession) {
...
            // When the session is ready, we start displaying the preview.
            mCaptureSession = cameraCaptureSession;
...         // Finally, we start displaying the camera preview.
            mPreviewRequest = mPreviewRequestBuilder.build();
            mCaptureSession.setRepeatingRequest(mPreviewRequest,
                mCaptureCallback, mBackgroundHandler);
            }
    } 

2.Framework和hal做了啥

1.首先呢，APP拿到systemservice中的cameramanager

拿到cameramanger后APP在调用manager.openCamera()

open的时候会链接到cameraservice: cameraUser = cameraService.connectDevice，最终返回的是CameraDeviceImpl接口

CameraManager.java (z:\mnt\data\projects\zal1885\frameworks\base\core\java\android\hardware\camera2)
openCamera() 
-> openCameraForUid(cameraId, callback, executor, USE_CALLING_UID); 
-> openCameraDeviceUserAsync(cameraId, callback, executor, clientUid);
-> android.hardware.camera2.impl.CameraDeviceImpl deviceImpl =
                    new android.hardware.camera2.impl.CameraDeviceImpl(
                        cameraId,
                        callback,
                        executor,
                        characteristics,
                        mContext.getApplicationInfo().targetSdkVersion);

            ICameraDeviceCallbacks callbacks = deviceImpl.getCallbacks();
            // Use cameraservice's cameradeviceclient implementation for HAL3.2+ devices
            //利用binder通信间接调用cameraservice.cpp中的函数
            ICameraService cameraService = CameraManagerGlobal.get().getCameraService();

            cameraUser = cameraService.connectDevice(callbacks, cameraId,
                            mContext.getOpPackageName(), uid);
            deviceImpl.setRemoteDevice(cameraUser);
            device = deviceImpl;
        //这里返回的是很重要的ICameraDeviceImpl接口，可以在为APP调用cameraservice设下铺垫
        return device; 

//--------------
connectDevice
->> connectHelper()
->> if(!(ret = makeClient(this, cameraCb, clientPackageName,
        cameraId, api1CameraId, facing,
        clientPid, clientUid, getpid(), legacyMode,
        halVersion, deviceVersion, effectiveApiLevel,
        /*out*/&tmp)).isOk()) 
    //初始化ICameraDeviceCallbacks,CameraDeviceClient
    err = client->initialize(mCameraProviderManager, mMonitorTags);
//-----------------设置IcameraDeviceUser的远程对象mRemoteDevice，后面的日子需要用到
//Set remote device, which triggers initial onOpened/onUnconfigured callbacks
deviceImpl.setRemoteDevice(cameraUser);
->> mRemoteDevice = new ICameraDeviceUserWrapper(remoteDevice);
    public ICameraDeviceUserWrapper(ICameraDeviceUser remoteDevice) {
        if (remoteDevice == null) {
            throw new NullPointerException("Remote device may not be null");
        }
        mRemoteDevice = remoteDevice;
    }
    //函数结束时自动回调到APP的onOpened函数
    mDeviceExecutor.execute(mCallOnOpened); 

2.刚才在openCamera的时候最终是回调回了APP的onOpened函数，在这里APP将取走我们的对象，为其所用:

mCameraDevice = cameraDevice; //返回CameraDevice对象引用

拿到我们梦寐以求的cameraDevice后就开始干活。

安卓在设计的时候说，要干活必须得以会议的形式在进行，好处是高效且正式

APP拿了对象后开始创建会议：

mCameraDevice.createCaptureSession()
-> createCaptureSessionInternal()
-> configureSuccess = configureStreamsChecked(inputConfig, outputConfigurations,
                        operatingMode, sessionParams);
    newSession = new CameraCaptureSessionImpl(mNextSessionId++, input,
                        callback, executor, this, mDeviceExecutor, configureSuccess); 

3.创建会议的时候我们要做哪些事呢? I.配置流 II.创建CameraCaptureSession

配置流函数：configureStreamsChecked(ICameraDeviceUserWrapper.java)

这个文件很出众，因为他是IPC通信的关键人物—传话筒，让java代码调用cpp代码
ICameraDeviceUserWrapper.java (z:\mnt\data\projects\zal1885\frameworks\base\core\java\android\hardware\camera2\impl)

他的对端是:

CameraDeviceClient.cpp (z:\mnt\data\projects\zal1885\frameworks\av\services\camera\libcameraservice\api2)

------------------------------------------AIDL--------------------------------------------

graph LR
JAVAFramework-->cameraserver 

还记得刚刚讲的mRemoteDevice后面的日子需要用到吗？

这里java层就开始疯狂的给cpp层下指令干活了

配置输入流，camera出的帧和metadata数据都要配置好:

configureStreamsChecked
-> checkInputConfiguration(inputConfig);
    mRemoteDevice.beginConfigure();
    int streamId = mRemoteDevice.createInputStream(inputConfig.getWidth(),
                                inputConfig.getHeight(), inputConfig.getFormat());
    int streamId = mRemoteDevice.createStream(outConfig);
    mRemoteDevice.endConfigure(operatingMode, sessionParams.getNativeCopy()); 

Camera3Device.cpp (z:\mnt\data\projects\zal1885\frameworks\av\services\camera\libcamera
mRemoteDevice.endConfigure
-> status_t err = mDevice->configureStreams(sessionParams, operatingMode);
-> res = mInterface->configureStreams(sessionBuffer, &config, bufferSizes);
-> auto err = mHidlSession_3_4->configureStreams_3_4() 

Camera3Device.cpp这里要通过mHidlSession_3_4调用HAL层代码:

CameraDevice3SessionImpl.cpp (z:\mnt\data\projects\zal1885\vendor\mediatek\proprietary\hardware\mtkcam3\main\hal\device\3.x\device)

终于让我们见识到hidl的存在了，

通过这个接口就可以从进程cameraserver到camerahalserver的调用了

-----------------------------------------------HIDL---------------------------------------------------

graph LR
cameraserver-->camerahalserver 

这里framework需要hal层:

I.配置APPstream流

II.通过metadata获得app下的请求信息

III.从app下的请求信息去决策hal层要给什么样的流输出去

CameraDevice3SessionImpl.cpp
mHidlSession_3_4->configureStreams_3_4()
-> onConfigureStreamsLocked(requestedConfiguration, halStreamConfiguration);
-> auto pAppStreamManager = getSafeAppStreamManager();
   err = pAppStreamManager->beginConfigureStreams(requestedConfiguration, halConfiguration, appStreams);
   auto pPipelineModel = getSafePipelineModel();
   mStaticInfo.mMetadataConverter->convertFromHidl(&((V3_4::StreamConfiguration&)requestedConfiguration).sessionParams, pParams->sessionParams) )
   err = pPipelineModel->configure(pParams);
   err = pAppStreamManager->endConfigureStreams(halConfiguration); 

4.再回来看下创建CameraCaptureSessionImpl并获取Session的回调，并回调onConfigured

mCurrentSession = new CameraCaptureSessionImpl(mNextSessionId++, input,
                        callback, executor, this, mDeviceExecutor, configureSuccess);
mSessionStateCallback = mCurrentSession.getDeviceStateCallback(); 

回调回来之后APP继续拿走会议对象：
mCaptureSession = cameraCaptureSession;

拿走后就可以进一步胡作非为了,做之前先利用cameraDevice请求下他的需求：

就像是填个申请单

mPreviewRequestBuilder
    = mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
mPreviewRequestBuilder.addTarget(surface);
-> templatedRequest = mRemoteDevice.createDefaultRequest(templateType); 

再利用mCaptureSession提交请求，将申请单提交给底层，最终setRepeatingRequest，但是大致流程是有循环一直在往mRequestQueue里面扔请求

mPreviewRequest = mPreviewRequestBuilder.build();
mCaptureSession.setRepeatingRequest(mPreviewRequest,
        mCaptureCallback, mBackgroundHandler);
}
//////////////////
CameraDeviceImpl.java (z:\mnt\data\projects\zal1885\frameworks\base\core\java\android\hardware\camera2\impl)	102567	12/28/2018
    public int setRepeatingRequest(CaptureRequest request, CaptureCallback callback,
            Executor executor) throws CameraAccessException {
        List<CaptureRequest> requestList = new ArrayList<CaptureRequest>();
        requestList.add(request);
        return submitCaptureRequest(requestList, callback, executor, /*streaming*/true);
    }
-> requestInfo = mRemoteDevice.submitRequestList(requestArray, repeating);
-> err = mDevice->setStreamingRequestList(metadataRequestList, surfaceMapList,
                &(submitInfo->mLastFrameNumber));
-> return submitRequestsHelper(requestsList, surfaceMaps, /*repeating*/true, lastFrameNumber);
->    if (repeating) {
        res = mRequestThread->setRepeatingRequests(requestList, lastFrameNumber);
    } else {
        res = mRequestThread->queueRequestList(requestList, lastFrameNumber);
    } 

我们在初始化cameradevice的时候RequestThread也一并起来了

Camera3Device.cpp (z:\mnt\data\projects\zal1885\frameworks\av\services\camera\libcameraservice\device3)	212638	12/28/2018
initializeCommonLocked(){
    /** Start up request queue thread */
    mRequestThread = new RequestThread(this, mStatusTracker, mInterface, sessionParamKeys);
} 

起来后一直在运行Camera3Device::RequestThread::threadLoop() 等待mRequestQueue请求的到来


一旦有新的请求，做一些准备工作后，通过hidl发送请求给hal层,hal层对应的就是APPstreamManager和pipelinemodel

CameraDevice3SessionImpl.cpp (z:\mnt\data\projects\zal1885\vendor\mediatek\proprietary\hardware\mtkcam3\main\hal\device\3.x\device)
submitRequestSuccess = sendRequestsBatch();
-> res = mInterface->processBatchCaptureRequests(requests, &numRequestProcessed);
-> err = hidlSession_3_4->proce*tureRequest_3_4(captureRequests_3_4, cachesToRemove,
    [&status, &numRequestProcessed] (auto s, uint32_t n) {
        status = s;
        *numRequestProcessed = n;
    });
-------------------hidl------------------
-> onProce*tureRequest()
-> err = pAppStreamManager->submitRequest(requests, appRequests);
   err = pPipelineModel->submitRequest(vPipelineRequests, numRequestProcessed); 

pipelineModel下面的submitRequest如下图

5.请求下去后pipelinemodule就开始决定走什么样的流程，

要把这些node想象成流水线上的工人，一个人干完自己的事后递交给下一个人继续最终完成想要的产品递交给厂长(AppStreamManager)，他再提交给客户(APP)
预览流Node图如下：

三 点击拍照按钮的时候做了哪些事情

1.APP在做啥

上节camera已经开始预览了，相对应的cameraeManager和CameraDevice都拿到了，那是如何拍照的呢？

1.首先我们得创建requestbuilder:mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_STILL_CAPTURE);

2.让captureSession去帮我们拍照:mCaptureSession.capture(captureBuilder.build(), CaptureCallback, null)

3.最后拍照完成后回调onCaptureCompleted()保存图片到mImageReader或者做其他处理

takepic(){
// This is the CaptureRequest.Builder that we use to take a picture.
            final CaptureRequest.Builder captureBuilder =
                    mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_STILL_CAPTURE);
            captureBuilder.addTarget(mImageReader.getSurface());

            // Use the same AE and AF modes as the preview.
            captureBuilder.set(CaptureRequest.CONTROL_AF_MODE,
                    CaptureRequest.CONTROL_AF_MODE_CONTINUOUS_PICTURE);
            setAutoFlash(captureBuilder);

            CameraCaptureSession.CaptureCallback CaptureCallback
                    = new CameraCaptureSession.CaptureCallback() {

                @Override
                public void onCaptureCompleted(@NonNull CameraCaptureSession session,
                                               @NonNull CaptureRequest request,
                                               @NonNull TotalCaptureResult result) {
                    showToast("Saved: " + mFile);
                    Log.d(TAG, mFile.toString());
                    unlockFocus();
                }
            };

            mCaptureSession.stopRepeating();
            mCaptureSession.abortCaptures();
            mCaptureSession.capture(captureBuilder.build(), CaptureCallback, null);
} 

2.Framework和hal做了啥

提交的请求大致相同，略微有所不同，node上也差异不大，但是咱们添加算法最多的就是添加在capture上面了，所以分享下添加的node大致位置:

参考：

1.Camera APK Code：

https://github.com/googlesamples/android-Camera2Basic

https://github.com/googlesamples/android-Camera2Video

2.https://www.2cto.com/kf/201706/653434.html camera3 概述

3.https://online.mediatek.com/

4.https://source.android.google.cn/devices/camera

5.https://blog.csdn.net/qq_16775897/article/details/81240600

6.https://source.android.google.cn/devices/architecture

本文地址：https://blog.csdn.net/qq_39417870/article/details/107895616