首页 / 技术分享 / 移动开发 / 正文
原创

(移动开发知识)Android Framework源码解读 - Audio - SoundTrigger(2)

来源:CSDN博客 作者:lzqustc 2021-06-23 13:51:57
点击转到:移动开发精品课程推荐

在上一篇中Android Framework源码解读 - Audio - SoundTrigger(1) - AIDL,提到startRecognition如何到达HAL层(本例中使用的唤醒词检测驱动来自厂商,HAL层的代码实现也是厂商提供),请看下图:

(移动开发知识)Android Framework源码解读 - Audio - SoundTrigger(2)

相关代码位于./frameworks/av/soundtrigger/ISoundTrigger.cpp 、./frameworks/av/services/soundtrigger/SoundTriggerHwService.cpp

共4步: BpSoundTrigger::transact(START_RECOGNITION)到达 BnSoundTrigger::onTransaction(START_RECOGNITION),再到具体实现类的③ ModuleClient::startRecognition,再由④ Module::startRecognition -》mHalInterface->startRecognition走到Hal层的SoundTriggerHalHidl::startRecognition,代码位于./frameworks/av/services/soundtrigger/SoundTriggerHalHidl.cpp

(移动开发知识)Android Framework源码解读 - Audio - SoundTrigger(2)

这里调用了sp<ISoundTriggerHw> soundtrigger = getService(); 从而得到了BpSoundTriggerHw对象,其接口定义在:./hardware/interfaces/soundtrigger/2.0/ISoundTriggerHw.hal 编译时会由工具生成中间文件:SoundTriggerHwAll.cpp。需要注意的是getService()的返回值被做了一个castFrom转换,最终返回的是 BpHwSoundTriggerHw对象,然后通过BpHwSoundTriggerHw::startRecognition -> _hidl_startRecognition方法调用::IInterface::asBinder(_hidl_this)->transact(5 /* startRecognition */, _hidl_data, &_hidl_reply); 告知对应的binder server端(BnHwSoundTriggerHw) 。

binder server端 BnHwSoundTriggerHw::onTransaction(5 /*startRecognition*/,,,,)收到后会调用::_hidl_startRecognition,最终调用它的具体实现类的方法:int32_t _hidl_out_retval = static_cast<BnHwSoundTriggerHw*>(_hidl_this)->_hidl_mImpl->startRecognition(modelHandle, *config, callback, cookie); 也就是SoundTriggerHalImpl类,代码位于./hardware/interfaces/soundtrigger/2.0/default/SoundTriggerHalImpl.h、 ./hardware/interfaces/soundtrigger/2.0/default/SoundTriggerHalImpl.cpp,头文件可以看到它继承了ISoundTriggerHw 接口: class SoundTriggerHalImpl : public ISoundTriggerHw {}

(移动开发知识)Android Framework源码解读 - Audio - SoundTrigger(2)

mHwDevice是在SoundTriggerHalImpl对象首次创建的时候被open的时候,具体函数如下:

(移动开发知识)Android Framework源码解读 - Audio - SoundTrigger(2)      

./hardware/libhardware/include/hardware/sound_trigger.h 、./hardware/rockchip/audio/dmdx_soundtrigger/sound_trigger_hw.c

static inline int sound_trigger_hw_device_open()方法也是调用的stdev_open(),主要功能如下图: ①、函数指针的映射; ②、stdev_init_driver(stdev); ③、创建检测线程monitor_thread_loop;

(移动开发知识)Android Framework源码解读 - Audio - SoundTrigger(2)

下面我们主要分析第2, 3步,首先来看stdev_init_driver(),相关代码位于./hardware/libhardware/include/hardware/sound_trigger.h、./hardware/rockchip/audio/dmdx_soundtrigger/sound_trigger_hw.c、./hardware/rockchip/audio/tinyalsa_hal/alsa_mixer.c

(移动开发知识)Android Framework源码解读 - Audio - SoundTrigger(2)

stdev->card_probe_method = DMDX_MIXER_CARD_PROBE_METHOD;
#define DMDX_MIXER_CARD_PROBE_METHOD     PROBE_CARD_BY_NAME_PARTIAL

//Probing mixer #card 1: Mixer name: 'dmdx'
stdev_init_mixer(stdev) { 
   stdev_init_mixer_by_card_name(stdev){  
     stdev->mixer = mixer_open(card_id=1);
     stdev_load_mixer_controls(stdev, stdev->mixer);
   }
}
stdev->cur_usecase = DMDX_UC_UNDEFINED;
stdev_update_usecase(stdev) {
   stdev_configure_usecase(stdev, DMDX_UC_UNDEFINED) {
       stdev_streaming_enable(stdev, false, NULL);
       mixer_ctl_set_value(stdev->ctl_usecase_manager, 0, cmd);
   }
}

这里我们要关注的一点是,最终device的操作是使用了Linux ALSA框架(在这Android系统里面,对应的是tinyalsa)的mixer功能,然后对应的sound card index=1,name=‘dmdx’。

接着看monitor_thread_loop(),

(移动开发知识)Android Framework源码解读 - Audio - SoundTrigger(2)
 

#define STREAMING_TIMEOUT             5 
#define MONITOR_THREAD_TIMEOUT        1 
#define AUDIO_SOURCE_CLOSE_TIMEOUT    1 

monitor_thread_loop()监测线程的主要工作就是检测stdev进入is_streaming状态后,在规定时间内是否被正确的close。 比如收到audio_close_request后,2s(AUDIO_SOURCE_CLOSE_TIMEOUT + MONITOR_THREAD_TIMEOUT) 内没正常关闭,就主动调用close。 比如is_streaming后, 如果6s(STREAMING_TIMEOUT + MONITOR_THREAD_TIMEOUT)内还没有read动作,则主动调用close。

stdev_open()成功后,存入stdev->device,得到mHwDevice对象,我们再来看stdev->device.start_recognition = stdev_start_recognition;

static int stdev_start_recognition(const struct sound_trigger_hw_device *dev,
           sound_model_handle_t sound_model_handle,
           const struct sound_trigger_recognition_config *config,
           recognition_callback_t callback,
           void *cookie)
{
    int status = 0;
    operation_lock(stdev);
    struct recognition_context *model_context =
                    get_model_context(stdev, sound_model_handle);

    if (model_context->model_started && !(stdev->is_streaming)) {
        status = dmdx_stop_recognition_for_model(stdev, model_context, false);
        if (status < 0) {
            goto exit;
        }
    }
    model_context->recognition_callback = callback;
    model_context->recognition_cookie = cookie;
    if (stdev->is_streaming) {
        status = dmdx_set_pending_command_for_model(stdev,
                               model_context, ST_CMD_START);
        goto exit;
    } else if (stdev->is_suspended == false) {
        status = dmdx_start_recognition_for_model(stdev, model_context);
    }

    if (stdev->is_suspended == false &&  !(stdev->callback_thread_active)) {
        /* Start trigger thread */
        status = pthread_create(&stdev->callback_thread,
                (const pthread_attr_t *) NULL,
                callback_thread_loop, stdev);
    }

    status = stdev_update_usecase(stdev);
exit:
    operation_unlock(stdev);
    return status;
}

./hardware/rockchip/audio/dmdx_soundtrigger/sound_trigger_hw.c, 代码稍微有点复杂(上面的代码片段去掉了一些细节部分),不过没关系,我们就抓住几个重要函数:

dmdx_stop_recognition_for_model();  //告诉驱动,先停止唤醒词检测模式

dmdx_set_pending_command_for_model();  //更新model_context->pending_command_mask

dmdx_start_recognition_for_model();  //告诉驱动,开启唤醒词检测模式

status = pthread_create(&stdev->callback_thread, (const pthread_attr_t *) NULL, callback_thread_loop, stdev); //创建callback_thread_loop线程,监听唤醒事件(uevent)

stdev_update_usecase();  //调用mixer设置声卡的usecase属性

结合上一篇文章,startRecognition从APK层到HAL层,再到驱动层的流程就基本打通了(HAL层和驱动成 属于厂商代码,这里就不做细节展开)。

最后HAL层会创建一个线程:callback_thread_loop通过uevent方式监听内核事件(也就是驱动检测到唤醒词后会上报uevent),HAL层收到event再通过recognition_callback_t callback 一层层报到APK层。APK层收到后,就通过AudioRecord来采集后续的音频数据,这也是就是上一篇提到的 soundtrigger框架原理:

The function of the Sound Trigger stack is to deliver discrete events that represent acoustic, trigger events. In most cases, the Sound Trigger stack does not deal with audio. Upon receipt of the trigger events, apps get access to the actual audio stream, surrounding the time of the events, by opening an AudioRecord object via the Audio framework. The Sound Trigger HAL APIs provide a handle with the triggered event that is used with the Audio Framework. Hence, since the Sound Trigger HAL and Audio HAL are connected under the hood, they typically share a process.

粗体字节部分是关键点,意思就是soundtrigger主要负责的是声学(acoustic)唤醒词的识别,识别后会触发相应的events给到上层。它不负责处理后续音频数据,应用层应该通过AudioRecord的方式来读取后续的音频数据。

 

 

版权声明:什么课程值得买倡导尊重与保护知识产权。未经许可,任何人不得复制、转载、或以其他方式使用本站《原创》内容,违者将追究其法律责任。本站文章内容,部分图片来源于网络,如有侵权,请联系我们修改或者删除处理。

精品课程推荐

热门文章

在线
客服

官方客服

如遇课程问题,可联系客服为您解决

电话客服:18482185493

Q Q客服:联系客服

工作时间:9:00-18:00,节假日休息

顶部