MTK官方资料/优秀学习博客分享

MTK HAL算法集成之单帧算法
MTK HAL算法集成之双摄算法
3rd Party ALGO Integrate Capture 3rd ALGO

HAL三方算法集成

1.三方算法类型

超级夜景首先是集成在拍照上的即需要挂载到CaptureNode，其次类似超夜，HDR都需要多帧EV- EV0 EV+多帧处理，因此还需要挂载到MultiFrame中，以下可以仿照FID_AIHDR进行添加
vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/CaptureFeature_Common.cpp

    case FID_AIHDR:                 return "aihdr";
+   case FID_SUPERNIGHT:            return "supernight";

vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/nodes/MultiFrameNode.cpp

    else if (rProperty.mFeatures & MTK_FEATURE_AIHDR)featId = FID_AIHDR;
+   else if (rProperty.mFeatures & TP_FEATURE_SUPERNIGHT)
+       featId = FID_SUPERNIGHT;

vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/3rdparty/customer/customer_feature_type.h

    TP_FEATURE_ZOOM_FUSION  = 1ULL << 47,   // used by zoom scenario
+   TP_FEATURE_SUPERNIGHT   = 1ULL << 48,// TODO: reserve for customer feature index (bit 32-63)

vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/feature/featurePipe/ICaptureFeaturePipe.h

    FID_AIHDR,
+   FID_SUPERNIGHT,

vendor/mediatek/proprietary/hardware/mtkcam3/pipeline/hwnode/p2/P2_CaptureProcessor.cpp

    if (feature & MTK_FEATURE_AIHDR)pCapRequest->addFeature(FID_AIHDR);
+   if (feature & TP_FEATURE_SUPERNIGHT)
+        pCapRequest->addFeature(FID_SUPERNIGHT);

2.三方算法挂载到对应的SCENARIO

get_capture_scenario根据上层的提示或者AppMeta信息，决定senario，超夜原生就有对应的senario为CUSTOMER_CAMERA_SCENARIO_CAPTURE_SUPER_NIGHT，可通过pParsedAppConfiguration->isSuperNightMode去控制这个senario走不走，默认是
vendor/mediatek/proprietary/custom/mt6853/hal/camera/camera_custom_feature_table.cpp

+   #define MTK_FEATURE_COMBINATION_SUPERNIGHT     (TP_FEATURE_SUPERNIGHT   |MTK_FEATURE_NR| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE)+   CAMERA_SCENARIO_START(CUSTOMER_CAMERA_SCENARIO_CAPTURE_SUPER_NIGHT)//
+   ADD_CAMERA_FEATURE_SET(TP_FEATURE_SUPERNIGHT,    MTK_FEATURE_COMBINATION_SUPERNIGHT)
+   ADD_CAMERA_FEATURE_SET(MTK_FEATURE_MFNR,     MTK_FEATURE_COMBINATION_MFNR)
+   ADD_CAMERA_FEATURE_SET(NO_FEATURE_NORMAL,    MTK_FEATURE_COMBINATION_SINGLE)
+   CAMERA_SCENARIO_END

以下可通过log去确认超夜挂载的feature和scenario是否正确

mtkcam-customer_scenario_mgr: [customer_get_capture_scenario] scenarioHint(isCShot:0, isDualCam:0, dualDevicePath:0, multiCamFeatureMode:-1, isLogical:1 captureScenarioIndex:-1)
mtkcam-customer_scenario_mgr: [customer_get_capture_scenario] capture OpMode:0x10600
mtkcam-customer_scenario_mgr: [customer_get_capture_scenario] scenario:103
mtkcam-customer_scenario_mgr: [customer_get_features_table_by_scenario] scenario:103, table size:1
mtkcam-customer_scenario_mgr: [customer_get_features_table_by_scenario] find features for scenario(103 : CUSTOMER_CAMERA_SCENARIO_CAPTURE_SUPER_NIGHT) in logical map
mtkcam-FeatureSettingPolicy: [evaluateCaptureConfiguration] find scenario:CUSTOMER_CAMERA_SCENARIO_CAPTURE_SUPER_NIGHT for (openId:0, scenario:103)
mtkcam-FeatureSettingPolicy: [evaluateCaptureConfiguration] scenario(CUSTOMER_CAMERA_SCENARIO_CAPTURE_SUPER_NIGHT) support feature:TP_FEATURE_SUPERNIGHT(0x2000000000000), feature combination:MTK_FEATURE_COMBINATION_SUPERNIGHT(0x20000000300d0)
mtkcam-FeatureSettingPolicy: [evaluateCaptureConfiguration] scenario(CUSTOMER_CAMERA_SCENARIO_CAPTURE_SUPER_NIGHT) support feature:MTK_FEATURE_MFNR(0x1), feature combination:MTK_FEATURE_COMBINATION_MFNR(0x300f1)
mtkcam-FeatureSettingPolicy: [evaluateCaptureConfiguration] scenario(CUSTOMER_CAMERA_SCENARIO_CAPTURE_SUPER_NIGHT) support feature:NO_FEATURE_NORMAL(0), feature combination:MTK_FEATURE_COMBINATION_SINGLE(0x300f0)
mtkcam-FeatureSettingPolicy: [evaluateCaptureConfiguration] support features:0x20000000300f1
mtkcam-FeatureSettingPolicy: [evaluateCaptureConfiguration] (0xb4000072dbf6dc70) uniqueKey:82405207

3.三方算法库，build动态库

新建一个floder用于存放三方算法的sdk库，包括cfg，头文件，允许库等方便三方算法接口执行函数的调用
vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/supernight_sdk/Android.mk

include $(CLEAR_VARS)
LOCAL_MODULE := libSupperLongExposure
LOCAL_SRC_FILES_64 := arm64-v8a/libSupperLongExposure.so
LOCAL_MODULE_TAGS := optional
LOCAL_MODULE_CLASS := SHARED_LIBRARIES
LOCAL_MODULE_SUFFIX := .so
LOCAL_SHARED_LIBRARIES := liblog libcutils libmtk_drvb libutils
LOCAL_PROPRIETARY_MODULE := true
LOCAL_MODULE_OWNER := mtk
LOCAL_MULTILIB := 64
LOCAL_CHECK_ELF_FILES := false
include $(BUILD_PREBUILT)

4.三方算法接口，build静态库

新建一个floder用于存放三方算法的接口PipelinePlugin API
vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/supernight/Android.mk

LOCAL_MODULE := libmtkcam.plugin.supernight
LOCAL_PROPRIETARY_MODULE := true
LOCAL_MODULE_OWNER := mtk
LOCAL_MODULE_TAGS := optional
include $(MTK_STATIC_LIBRARY)

超夜三方算法Provider继承自MultiFramePLugin:IProvider，API接口主要有property negotiate init process abort uninit，每种类型的PipelinePlugin，Property成员的定义都不一样，具体说明可以参考DCC文档“MTK_camera_plugin_interface_user_guide”中4.4 CPI PipelinePlugin API的内容，其中一定需要注意填写的是mName与mFeature字段，在negotiate函数中配置算法需要的输入、输出图像的格式、尺寸，raw算法则输入输出对应eImgFmt_BAYER10_UNPAK。

prop.mName = "THIRD_PARTY_SuperNight";
prop.mFeature = TP_FEATURE_SUPERNIGHT;

目前超夜算法的实现，已知有两种
方案一:Exp time + ISO (AE Lock + AE mode off (AE mode = 0))
方案二 EV compensation(AE lock)

5.三方算法链接

vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/Android.mk

LOCAL_C_INCLUDES +=$(TOP)/$(MTK_PATH_SOURCE)/hardware/mtkcam3/3rdparty/supernight_sdk/inc
LOCAL_WHOLE_STATIC_LIBRARIES += libmtkcam.plugin.supernight
LOCAL_SHARED_LIBRARIES += libSupperLongExposure

三方算法参数区分

1.解析App下发OpMode的区分是否进去超夜模式

vendor/mediatek/proprietary/hardware/mtkcam3/pipeline/model/session/PipelineModelSessionFactory.cpp

if(pParsedAppConfiguration->operationMode == 自定义){pParsedAppConfiguration->isSuperNightMode = true;property_set("vendor.debug.camera.supernightmode","1");
}else{pParsedAppConfiguration->isSuperNightMode = false;property_set("vendor.debug.camera.supernightmode","0");
}

或者通过App下发VendorTag的方式控制，tryGetMetadata(pAppMeta, CONTROL_SUPERNIGHT_MODE, superNight);

2.添加一个新的EApp用于超夜参数区分

vendor/mediatek/proprietary/custom/mt6853/hal/inc/tuning_mapping/cam_idx_struct_ext.h

#define Comm_EApp_SuperNight 14typedef enum
{EApp_MTKCam = Comm_EApp_MTKCam,EApp_Facebook = Comm_EApp_Facebook,...EApp_3rd_party = Comm_EApp_3rd_party,EApp_END2ENDHDR = Comm_EApp_END2ENDHDR,
+   EApp_SuperNight = Comm_EApp_SuperNight,EApp_NUM,
} EApp_T;

3.通过map_AppName_to_MappingInfo区分找到对应模式的EApp

3.1 不传参AppName，直接控制判断，返回对应的模式的EApp

vendor/mediatek/proprietary/custom/mt6853/hal/camera_isp/ver1/isp_tuning_custom.cpp

MINT32
IspTuningCustom::
map_AppName_to_MappingInfo(const char* pAppname, const MINT32 feature){+   int supernightmode = 0;
+   supernightmode = property_get_int32("vendor.debug.camera.supernightmode", 0);
+   MY_LOG("[%s] supernightmode:%d \n", __FUNCTION__, supernightmode);if (strstr(AppName, "Facebook")){return EApp_Facebook;...else{if (strstr(pAppname, "MTKCam") && (feature & MTK_HDR_FEATURE_HDR_HAL_MODE_MSTREAM_CAPTURE_PREVIEW) ){return EApp_HDR;}//+++++++++++ For customization, please remove this section.+      else{+          if (supernightmode){+              return EApp_SuperNight;+          }elsereturn EApp_MTKCam;}}

3.2 通过ATMs机制的中的setFactor设置对应模式的EApp

vendor/mediatek/proprietary/custom/mt6853/hal/camera_isp/ver1/isp_tuning_custom.cpp

MINT32
IspTuningCustom::
map_AppName_to_MappingInfo(const char* pAppname, const MINT32 feature){if (strstr(AppName, "Facebook")){return EApp_Facebook;
+   if (strstr(AppName, "SuperNight")){
+       return EApp_SuperNight;...

这个是之前项目上遇到过的别人使用的方案NSCam::ATMsEventMgr::getInstance()->setFactor(“APP_NAME”, “SuperNight”);其中在解问题的时候发现有参数没有区分开，类似log如下，通道会跳转到Scene_Preview

MtkCam/MappingMgr: [query] [Caller: getISPIndex] [Dev:1-Mod:DCE(8)] (Idx 5) (Night_Vision_Capture) (ISO_GROUP 2)
MtkCam/MappingMgr: [query] [Caller: getISPIndex] [Dev:1-Mod:DCE(8)] (Idx 5) (Night_Vision_Capture) (PF Capture_DCE, SM Preview, Bin 0, P2 0, FLASH 0, APP Night_Vision, FD 0, ZOOM 0, LV 3, CT 9, ISO 8, ISO_GROUP 2) (Base_Idx 5)
MtkCam/MappingMgr: [query] [Caller: TaskRequest] [Dev:1-Mod:LTM(36)] (Idx 2) (Scene_Preview) (ISO_GROUP 2)
MtkCam/MappingMgr: [query] [Caller: TaskRequest] [Dev:1-Mod:LTM(36)] (Idx 2) (Scene_Preview) (PF Preview, SM Preview, Bin 0, P2 0, FLASH 0, APP MTKCam, FD 0, ZOOM 0, LV 0, CT 0, ISO 0, ISO_GROUP 2) (Base_Idx 2)

正常按参数配置表中的ScenarioMap应该是，所以需要修改Flow控制
IspProfile SensorMode AppName Scenario
A_Normal_Capture Preview Night_Vision Night_Vision_Capture

vendor/mediatek/proprietary/hardware/mtkcam/aaa/source/isp_6s/wrapper/Hal3ARawImp.cpp

MVOID
Hal3ARawImp::
TaskRequest(Request_Service service_id, MVOID* input, MVOID* output)
{switch (service_id) {case REQ_LTM_INPUT:{CAM_IDX_QRY_COMB rMapping_InfoLTM = {};rMapping_InfoLTM.eIspProfile = m_rParam.eIspProfile;rMapping_InfoLTM.eSensorMode = static_cast<NSIspTuning::ESensorMode_T>(m_u4SensorMode);+           IspTuningCustom* pIspTuningCustom = IspTuningCustom::createInstance((ESensorDev_T)m_i4SensorDev, m_i4SensorIdx, m_i4SensorIdx);
+           //MBOOL ret = NSCam::ATMsEventMgr::getInstance()->getFactor("APP_NAME", str_factor_value);
+           EApp_T NV = EApp_MTKCam;+           if (pIspTuningCustom)
+           {
+                NV = (EApp_T)pIspTuningCustom->map_AppName_to_MappingInfo(str_factor_value.c_str(), -1);
+           }else
+           {
+                CAM_LOGD("[%s] pIspTuningCustom is NULL", __FUNCTION__);
+           }+           if(NV == EApp_Night_Vision && m_i4SensorDev == 1 ){
+                 rMapping_InfoLTM.eApp = EApp_Night_Vision;
+                 rMapping_InfoLTM.eIspProfile = EIspProfile_Capture;
+                 m_rParam.eIspProfile = EIspProfile_Capture;}

Android MTK三方算法集成学习相关推荐

MTK 双摄算法集成
和你一起终身学习,这里是程序员Android 经典好文推荐,通过阅读本文,您将收获以下知识点: 一. 双摄算法简介二. 选择feature和配置feature table 三.挂载算法四. APP ...
机器学习-Adaboost 算法(集成学习)
本文结构: 什么是集成学习? 为什么集成的效果就会好于单个学习器? 如何生成个体学习器? 什么是 Boosting? Adaboost 算法? 什么是集成学习集成学习就是将多个弱的学习器结合起来组成 ...
机器学习算法——集成学习1（个体与集成）
集成学习(Ensemble Learning)通过构建并结合多个学习器来完成学习任务. 上图为集成学习的一般结构:先产生一组"个体学习器",再用某种策略将它们结合起来. 个体学习器 ...
android 绘制歌词算法,深度学习算法写歌词
AI开发平台ModelArts ModelArts是面向开发者的一站式AI开发平台,为机器学习与深度学习提供海量数据预处理及半自动化标注.大规模分布式Training.自动化模型生成,及端-边-云模型 ...
Android系统编译时集成三方APK
1. 前言最近在学习Android系统开发,在开发过程中可能要到将三方apk集成到系统中的需求.目前我了解的方法有两种. 一种是在系统编译完毕之后,再将三方apk文件放置到系统目录之下.参考如何将a ...
android 三方_面试官送你一份Android热门三方库源码面试宝典及学习笔记
前言众所周知,优秀源码的阅读与理解是最能提升自身功力的途径,如果想要成为一名优秀的Android工程师,那么Android中优秀三方库源码的分析和理解则是必备技能.就拿比较热门的图片加载框架Glid ...
04 集成学习 - Boosting - AdaBoost算法构建
03 集成学习 - Boosting - AdaBoost算法原理十.AdaBoost算法构建上一章最后说明了每个基模型的权值α是如何求得的,于是我就可以对模型进行更新操作了. 构建过程一 1.假 ...
通俗讲解集成学习算法！
↑↑↑关注后"星标"Datawhale 每日干货 & 每月组队学习,不错过 Datawhale干货作者:黄星源,Datawhale优秀学习者本文以图文的形式对模型算法中 ...
集成学习Bagging和Boosting算法总结
一.集成学习综述 1.集成方法或元算法是对其他算法进行组合的一种方式,下面的博客中主要关注的是AdaBoost元算法.将不同的分类器组合起来,而这种组合结果被称为集成方法/元算法.使用集成算法时会有很 ...

Android MTK三方算法集成学习