【我的DIY设计】+ stm32f3之语音识别复杂算法初探

只看该作者 · 2013-1-20 15:03

本帖最后由 smartmcu 于 2013-1-20 19:11 编辑

收到stm32f3 discovery 好些天了，一直忙于市场调查、数据挖掘、算法的工作中，快到答应的交作业期限了，硬着头皮弄一点吧 :)

stm32f3好在哪里？dsp、fpu、opamp、adc多，价格合理、购买容易（这就使交易成本低、总成本更低----虽然freescale的类似东西，价格也类似，但其交易成本高、不确定性强，期权价格高昂，导致非不得已不用），于是市场自然会认它为老大，而“利见大人”是我等高级电工的生存本能，当然就用它了。

既然dsp、fpu那么牛，是该物尽其用，图像识别，好像还不够用，那语音、特别是高级语音识别应该可以很好发挥其效用。
我们知道，语音识别有以下步骤，采样--滤波--预加重--端点识别--提取特征--分类。其中端点识别、特征提取、分类，涉及到的计算量最大，最能考验dsp、fpu的能力。当然，这些年的发展，很多快速简单的语音算法都可以在很低级的mcu上使用了，但效果参差不齐，既然要考验dsp、fpu的能力，咱们就来点耗时的、高级的东西。

平时只要涉及到单片机都是捉襟见肘的性能和容量，这回遇到stm32f3这种档次的，怎么都该吃顿好的嘛！:)
首先是端点识别，目前看到效果比较好的是谱熵算法，主要原理是噪声部分的分布比较均匀，而有声部分不太均匀，而熵是表征不确定性的一个很好的指标，特别地用dct算出来的比fft的效果好，那么端点识别就选dct，其中还要用到对数计算，更能测试fpu的效果。
然后是特征提取，目前语音常用的是lpcc和mfcc和plp，mfcc效果很好，计算用到fft和小规模的dct还有矩阵乘法，应该选它；
最后是分类，动态时间规划DTW和隐性马尔可夫hmm是语音识别常用的，hmm前期的训练工作，计算量、存储要求有点过高，DTW其实就很好了，选DTW。

既然选用了以上这些，那仔细看看，基本上就是FFT DCT和对数、矩阵计算，那先搞个可行性的试验，如果可行，以后再慢慢具体实现之，于是今天的任务就是搞点 FFT DCT和对数、矩阵等密集计算的，看看可行否。

也不建具体的project了，直接用stm32f3 discovery的demo（STM32F3-Discovery_FW_V1.1.0\Project\Demonstration\MDK-ARM\Demo.uvproj）把它的实验功能里的3个功能多加一个。
这涉及到新加一个文件tester.c，修改3个文件，一个是stm32f30x_it.c，一个是main.h，一个是main.c。

为了编译成功，还要在option 里的preprocessor symbols里加入
USE_STDPERIPH_DRIVER,STM32F30X,ARM_MATH_CM4, ARM_MATH_MATRIX_CHECK, ARM_MATH_ROUNDING, ARM_MATH_LITTLE_ENDIAN, __FPU_PRESENT = 1
还要在工程里加入一个文件 \Keil\ARM\CMSIS\Lib\ARM\arm_cortexM4lf_math.lib以保证link成功。

stm32f30x_it.c：179行改成    if (UserButtonPressed > 0x0F)

main.h： 43行，前插入 void Tester_2013_0120(int loop_times);

main.c：424行，前插入

/* LEDs Off */
STM_EVAL_LEDOff(LED3);
STM_EVAL_LEDOff(LED6);
STM_EVAL_LEDOff(LED7);
STM_EVAL_LEDOff(LED4);
STM_EVAL_LEDOff(LED10);
STM_EVAL_LEDOff(LED8);
STM_EVAL_LEDOff(LED9);
STM_EVAL_LEDOff(LED5);

/* Waiting User Button is pressed */
while (UserButtonPressed > 0x02)
{
   /* Toggle LD3 */
   STM_EVAL_LEDToggle(LED3);
   /* Insert Tester delay */
   Tester_2013_0120(UserButtonPressed-2);
   /* Toggle LD5 */
   STM_EVAL_LEDToggle(LED5);
   /* Insert Tester delay */
   Tester_2013_0120(UserButtonPressed-2);
   /* Toggle LD7 */
   STM_EVAL_LEDToggle(LED7);
   /* Insert Tester delay */
   Tester_2013_0120(UserButtonPressed-2);
   /* Toggle LD9 */
   STM_EVAL_LEDToggle(LED9);
   /* Insert Tester delay */
   Tester_2013_0120(UserButtonPressed-2);
   /* Toggle LD10 */
   STM_EVAL_LEDToggle(LED10);
   /* Insert Tester delay */
   Tester_2013_0120(UserButtonPressed-2);
   /* Toggle LD8 */
   STM_EVAL_LEDToggle(LED8);
   /* Insert Tester delay */
   Tester_2013_0120(UserButtonPressed-2);
   /* Toggle LD6 */
   STM_EVAL_LEDToggle(LED6);
   /* Insert Tester delay */
   Tester_2013_0120(UserButtonPressed-2);
   /* Toggle LD4 */
   STM_EVAL_LEDToggle(LED4);
   /* Insert Tester delay */
   Tester_2013_0120(UserButtonPressed-2);
}

tester.c的内容后附

只看该作者 · 2013-1-20 22:20

本帖最后由 smartmcu 于 2013-1-23 13:46 编辑

以下是 tester.c的代码，注释了一下，可能还不够详细

#include "arm_math.h"

float32_t testInput_f32[512] =
{
-1.640739552179682,       0.000000000000000,       -1.613393726467190,       0.000000000000000,       -1.156741241731352,       0.000000000000000,       2.527773464519963,       0.000000000000000,
-0.497040638009502,       0.000000000000000,       -0.975817112895589,       0.000000000000000,       -2.866830755546166,       0.000000000000000,       1.120214498507878,       0.000000000000000,
5.986771654661698,       0.000000000000000,       0.398219252656757,       0.000000000000000,       -3.545606013198135,       0.000000000000000,       0.312398099396191,       0.000000000000000,
-2.265327979531788,       0.000000000000000,       0.792121001107366,       0.000000000000000,       -3.736145137670100,       0.000000000000000,       0.762228883650802,       0.000000000000000,
2.283545661214646,       0.000000000000000,       3.780020629583529,       0.000000000000000,       3.117260228608810,       0.000000000000000,       -2.011159255609613,       0.000000000000000,
0.279107700476072,       0.000000000000000,       2.003369134246936,       0.000000000000000,       -1.448171234480257,       0.000000000000000,       0.584697150310140,       0.000000000000000,
0.919508663636197,       0.000000000000000,       -3.071349141675388,       0.000000000000000,       -1.555923649263667,       0.000000000000000,       2.232497079438850,       0.000000000000000,
-0.012662139119883,       0.000000000000000,       0.372825540734715,       0.000000000000000,       2.378543590847629,       0.000000000000000,       1.459053407813062,       0.000000000000000,
-0.967913907390927,       0.000000000000000,       1.322825200678212,       0.000000000000000,       -1.033775820061824,       0.000000000000000,       -1.813629552693142,       0.000000000000000,
4.794348161661486,       0.000000000000000,       0.655279811518676,       0.000000000000000,       -2.224590138589720,       0.000000000000000,       0.595329481295766,       0.000000000000000,
3.364055988866225,       0.000000000000000,       1.863416422998127,       0.000000000000000,       1.930305751828105,       0.000000000000000,       -0.284467053432545,       0.000000000000000,
-0.923374905878938,       0.000000000000000,       1.922988234041399,       0.000000000000000,       0.310482143432719,       0.000000000000000,       0.332122302397134,       0.000000000000000,
-1.659487472408966,       0.000000000000000,       -1.865943507877961,       0.000000000000000,       -0.186775297569864,       0.000000000000000,       -1.700543850628361,       0.000000000000000,
0.497157959366735,       0.000000000000000,       -0.471244843957418,       0.000000000000000,       -0.432013753969948,       0.000000000000000,       -4.000189880113231,       0.000000000000000,
-0.415335170016467,       0.000000000000000,       0.317311950972859,       0.000000000000000,       0.038393428927595,       0.000000000000000,       0.177219909465206,       0.000000000000000,
0.531650958095143,       0.000000000000000,       -2.711644985175806,       0.000000000000000,       0.328744077805156,       0.000000000000000,       -0.938417707547928,       0.000000000000000,
0.970379584897379,       0.000000000000000,       1.873649473917137,       0.000000000000000,       0.177938226987023,       0.000000000000000,       0.155609346302393,       0.000000000000000,
-1.276504241867208,       0.000000000000000,       -0.463725075928807,       0.000000000000000,       -0.064748250389500,       0.000000000000000,       -1.725568534062385,       0.000000000000000,
-0.139066584804067,       0.000000000000000,       1.975514554117767,       0.000000000000000,       -0.807063199499478,       0.000000000000000,       -0.326926659682788,       0.000000000000000,
1.445727032487938,       0.000000000000000,       -0.597151107739100,       0.000000000000000,       2.732557531709386,       0.000000000000000,       -2.907130934109188,       0.000000000000000,
-1.461264832679981,       0.000000000000000,       -1.708588604968163,       0.000000000000000,       3.652851925431363,       0.000000000000000,       0.682050868282879,       0.000000000000000,
-0.281312579963294,       0.000000000000000,       0.554966483307825,       0.000000000000000,       -0.981341739340932,       0.000000000000000,       1.279543331141603,       0.000000000000000,
0.036589747826856,       0.000000000000000,       2.312073745896073,       0.000000000000000,       1.754682200732425,       0.000000000000000,       -0.957515875428627,       0.000000000000000,
-0.833596942819695,       0.000000000000000,       0.437054368791033,       0.000000000000000,       -0.898819399360279,       0.000000000000000,       -0.296050580896839,       0.000000000000000,
-0.785144257649601,       0.000000000000000,       -2.541503089003311,       0.000000000000000,       2.225075846758761,       0.000000000000000,       -1.587290487902002,       0.000000000000000,
-1.421404172056462,       0.000000000000000,       -3.015149802293631,       0.000000000000000,       1.780874288867949,       0.000000000000000,       -0.865812740882613,       0.000000000000000,
-2.845327531197112,       0.000000000000000,       1.445225867774367,       0.000000000000000,       2.183733236584647,       0.000000000000000,       1.163371072749080,       0.000000000000000,
0.883547693520409,       0.000000000000000,       -1.224093106684675,       0.000000000000000,       -1.854501116331044,       0.000000000000000,       1.783082089255796,       0.000000000000000,
2.301508706196191,       0.000000000000000,       -0.539901944139077,       0.000000000000000,       1.962315832319967,       0.000000000000000,       -0.060709041870503,       0.000000000000000,
-1.353139923300238,       0.000000000000000,       -1.482887537805234,       0.000000000000000,       1.273732601967176,       0.000000000000000,       -3.456609915556321,       0.000000000000000,
-3.752320586540873,       0.000000000000000,       3.536356614978951,       0.000000000000000,       0.206035952043233,       0.000000000000000,       5.933966913773842,       0.000000000000000,
-0.486633898075490,       0.000000000000000,       -0.329595089863342,       0.000000000000000,       1.496414153905337,       0.000000000000000,       0.137868749388880,       0.000000000000000,
-0.437192030996792,       0.000000000000000,       2.682750615210656,       0.000000000000000,       -2.440234892848570,       0.000000000000000,       1.433910252426186,       0.000000000000000,
-0.415051506104074,       0.000000000000000,       1.982003013708649,       0.000000000000000,       1.345796609972435,       0.000000000000000,       -2.335949513404370,       0.000000000000000,
1.065988867433025,       0.000000000000000,       2.741844905000464,       0.000000000000000,       -1.754047930934362,       0.000000000000000,       0.229252730015575,       0.000000000000000,
-0.679791016408669,       0.000000000000000,       -2.274097820043743,       0.000000000000000,       0.149802252231876,       0.000000000000000,       -0.139697151364830,       0.000000000000000,
-2.773367420505435,       0.000000000000000,       -4.403400246165611,       0.000000000000000,       -1.468974515184135,       0.000000000000000,       0.664990623095844,       0.000000000000000,
-3.446979775557143,       0.000000000000000,       1.850006428987618,       0.000000000000000,       -1.550866747921936,       0.000000000000000,       -3.632874882935257,       0.000000000000000,
0.828039662992464,       0.000000000000000,       2.794055182632816,       0.000000000000000,       -0.593995716682633,       0.000000000000000,       0.142788156054200,       0.000000000000000,
0.552461945119668,       0.000000000000000,       0.842127129738758,       0.000000000000000,       1.414335509600077,       0.000000000000000,       -0.311559241382430,       0.000000000000000,
1.510590844695250,       0.000000000000000,       1.692217183824300,       0.000000000000000,       0.613760285711957,       0.000000000000000,       0.065233463207770,       0.000000000000000,
-2.571912893711505,       0.000000000000000,       -1.707001531141341,       0.000000000000000,       0.673884968382041,       0.000000000000000,       0.889863883420103,       0.000000000000000,
-2.395635435233346,       0.000000000000000,       1.129247296359819,       0.000000000000000,       0.569074704779735,       0.000000000000000,       6.139436017480722,       0.000000000000000,
0.822158309259017,       0.000000000000000,       -3.289872016222589,       0.000000000000000,       0.417612988384414,       0.000000000000000,       1.493982103868165,       0.000000000000000,
-0.415353391377005,       0.000000000000000,       0.288670764933155,       0.000000000000000,       -1.895650228872272,       0.000000000000000,       -0.139631694475020,       0.000000000000000,
1.445103299005436,       0.000000000000000,       2.877182243683429,       0.000000000000000,       1.192428490172580,       0.000000000000000,       -5.964591921763842,       0.000000000000000,
0.570859795882959,       0.000000000000000,       2.328333316356666,       0.000000000000000,       0.333755014930026,       0.000000000000000,       1.221901577771909,       0.000000000000000,
0.943358697415568,       0.000000000000000,       2.793063983613067,       0.000000000000000,       3.163005066073616,       0.000000000000000,       2.098300664513867,       0.000000000000000,
-3.915313164333447,       0.000000000000000,       -2.475766769064539,       0.000000000000000,       1.720472044894277,       0.000000000000000,       -1.273591949275665,       0.000000000000000,
-1.213451272938616,       0.000000000000000,       0.697439404325690,       0.000000000000000,       -0.309902287574293,       0.000000000000000,       2.622575852162781,       0.000000000000000,
-2.075881936219060,       0.000000000000000,       0.777847545691770,       0.000000000000000,       -3.967947986440650,       0.000000000000000,       -3.066503371806472,       0.000000000000000,
1.193780625937845,       0.000000000000000,       0.214246579281311,       0.000000000000000,       -2.610681491162162,       0.000000000000000,       -1.261224183972745,       0.000000000000000,
-1.165071748544285,       0.000000000000000,       -1.116548474834374,       0.000000000000000,       0.847202164846982,       0.000000000000000,       -3.474301529532390,       0.000000000000000,
0.020799541946476,       0.000000000000000,       -3.868995473288166,       0.000000000000000,       1.757979409638067,       0.000000000000000,       0.868115130183109,       0.000000000000000,
0.910167436737958,       0.000000000000000,       -1.878855115563720,       0.000000000000000,       1.710357104174161,       0.000000000000000,       -1.468933980990902,       0.000000000000000,
1.799544171601169,       0.000000000000000,       -4.922332880027887,       0.000000000000000,       0.219424548939720,       0.000000000000000,       -0.971671113451924,       0.000000000000000,
-0.940533475616266,       0.000000000000000,       0.122510114412152,       0.000000000000000,       -1.373686254916911,       0.000000000000000,       1.760348103896323,       0.000000000000000,
0.391745067829643,       0.000000000000000,       2.521958505327354,       0.000000000000000,       -1.300693516405092,       0.000000000000000,       -0.538251788309178,       0.000000000000000,
0.797184135810173,       0.000000000000000,       2.908800548982588,       0.000000000000000,       1.590902251655215,       0.000000000000000,       -1.070323714487264,       0.000000000000000,
-3.349764443340999,       0.000000000000000,       -1.190563529731447,       0.000000000000000,       1.363369471291963,       0.000000000000000,       -1.814270299924576,       0.000000000000000,
-0.023381588315711,       0.000000000000000,       1.719182048679569,       0.000000000000000,       0.839917213252626,       0.000000000000000,       1.006099633839122,       0.000000000000000,
0.812462674381527,       0.000000000000000,       1.755814336346739,       0.000000000000000,       2.546848681206319,       0.000000000000000,       -1.555300208869455,       0.000000000000000,
1.017053811631167,       0.000000000000000,       0.996591039170903,       0.000000000000000,       -1.228047247924881,       0.000000000000000,       4.809462271463009,       0.000000000000000,
2.318113116151685,       0.000000000000000,       -1.206932520679733,       0.000000000000000,       1.273757685623312,       0.000000000000000,       0.724335352481802,       0.000000000000000,
};

uint32_t fftSize = 256;
uint32_t ifftFlag = 0;
uint32_t doBitReverse = 1;

static float32_t testOutput[512];

void Tester_2013_0120(int loop_times)
{
  uint32_t refIndex = 213, testIndex = 0;
//       arm_status status;
      arm_cfft_radix4_instance_f32 S;
      arm_matrix_instance_f32 mtxA, mtxB, mtxC;
      float32_t maxValue;

  arm_status status = ARM_MATH_SUCCESS;

  int i;
      int j;
  float32_t tt,ps;
      /* Initialize the CFFT/CIFFT module */
      status = arm_cfft_radix4_init_f32(&S, fftSize,
                                                                        ifftFlag, doBitReverse);

      arm_mat_init_f32 (&mtxA, 16, 16, testOutput);
      arm_mat_init_f32 (&mtxB, 16, 16, testOutput);
      arm_mat_init_f32 (&mtxC, 16, 16, testOutput);

      /* 这个是由discovery 板子的user 按钮决定循环次数的
      板子复位后，首先执行的是流水灯，按一下按键就成了陀螺仪
      和加速度计的功能，再按是指南针，后面就是我们测试用循环次数了
  第一次是1次循环，按一次按钮后是2次循环，再一次按钮后是4次循环
  按2的幂提升，直到4096次

  然后用个秒表记录同一盏灯的亮灭时间除以8，再除以循环次数
      就可大概计算出每个循环花费的时间了

      下面的代码大概可以相当于一帧语音的处理的计算量，如果低于10ms，那就基本上是可用了
  */
      for (i=0;i<(1<<(loop_times-1));i++)
         {


  /* 模拟谱熵的计算量 */
      arm_copy_f32 (testInput_f32, testOutput, fftSize*2);
      arm_cfft_radix4_f32(&S, testOutput);
      ps=tt=0.0f;

      /* 即便是累加和用dsp都快了40%
      for (j=0;j<fftSize;j++)
            tt+=testOutput[j];
  */
      arm_mean_f32(testOutput,  fftSize,  &tt);
      tt=1/tt/fftSize;
      arm_scale_f32(testOutput, tt, testOutput,  fftSize);

      /* 恐怖的非dsp运算，加不加居然差10多倍
      for (j=0;j<fftSize;j++)
      ps+=testOutput[j]*log(testOutput[j]);
  */

      /* 模拟mfcc计算量 */
      arm_cfft_radix4_f32(&S, testInput_f32);
      arm_cmplx_mag_f32(testInput_f32, testOutput, fftSize);
      arm_max_f32(testOutput, fftSize, &maxValue, &testIndex);
      arm_cfft_radix4_f32(&S, testInput_f32);

      /* 模拟dtw计算量有点问题，存储量好像不太够的样子，这个有点存疑*/
  arm_mat_mult_f32(&mtxA, &mtxB, &mtxC);
  arm_mat_mult_f32(&mtxA, &mtxB, &mtxC);
  arm_mat_mult_f32(&mtxA, &mtxB, &mtxC);
  arm_mat_mult_f32(&mtxA, &mtxB, &mtxC);

      while (status != ARM_MATH_SUCCESS);

   }

}

只看该作者 · 2013-1-20 22:30

本帖最后由 smartmcu 于 2013-1-20 22:55 编辑

测试了一下，每次循环在4ms以内。虽然有一些存疑，特别是非dsp运算的log运算大幅度地降低了性能，但以后可以改成用谱的方差（也有dsp库），也可达到一定的效果。所以应该有很好的实用性，值得一试，争取完善之。

另，附件里有main.c、main.h、stm32f30x_it.c、tester.c和demo.hex，有兴趣者可以下载下来一试。

总算把作业初稿赶出来了，以后有时间，一定继续搞，兴奋和疲劳的星期天，有点小时候赶假期作业的快感，嚯嚯！