查看: 617|回复: 2

[ESP8266/ESP32] Beetle ESP32 C6 ncnn神经网络数字识别

Beetle ESP32 C6 ncnn神经网络数字识别

# 介绍 Beetle ESP32 C6

Beetle ESP32 C6 ncnn神经网络数字识别图1

Beetle ESP32-C6是一款基于ESP32-C6芯片设计的迷你体积的低功耗物联网开发板

* 超小体积，尺寸仅25*20.5mm
* 搭载ESP32-C6芯片，支持Wi-Fi、BLE、Zigbee、Thread通讯协议
* 支持Wi-Fi 6协议，更低延迟，更低功耗
* 超低功耗，deep-sleep 14uA
* 集成锂电池充电功能
* 支持电池电压检测，了解设备电量信息

# 配置编译开发环境

首先按照 https://github.com/espressif/esp-idf 快速上手指引，配置 esp-idf 编译开发环境

设好代理，clone install export 三连即可

```shell

git clone https://github.com/espressif/esp-idf
git submodule update --init --recursive
sh install.sh
source export.sh
复制代码

```

# 编译 ncnn for esp32c6

github 下载 ncnn 最新源码

```shell

git clone https://github.com/Tencent/ncnn.git
复制代码

```

弄一个 esp32c6 cmake toolchain，稍微精简下 ncnn 编译参数

* 禁用 c++ RTTI exception 减小体积
* esp32c6 只有单核心，禁用 openmp 多线程功能
* 不编译 benchncnn 跑分工具（依赖系统环境）
* esp32c6 存储更大，奢侈些就不需要精简算子

```cmake

set(CMAKE_SYSTEM_NAME freertos)
set(CMAKE_SYSTEM_PROCESSOR riscv)

include($ENV{IDF_PATH}/tools/cmake/toolchain-esp32c6.cmake)

set(CMAKE_TRY_COMPILE_TARGET_TYPE STATIC_LIBRARY)

set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)

# cache flags
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS}" CACHE STRING "c flags")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}" CACHE STRING "c++ flags")

option(NCNN_DISABLE_RTTI "" ON)
option(NCNN_DISABLE_EXCEPTION "" ON)
option(NCNN_DISABLE_PIC "" ON)
option(NCNN_PLATFORM_API "" OFF)
option(NCNN_RUNTIME_CPU "" OFF)

option(NCNN_THREADS "" OFF)
option(NCNN_OPENMP "" OFF)

option(NCNN_BUILD_BENCHMARK "" OFF)
option(NCNN_BUILD_TESTS "" OFF)
option(NCNN_BUILD_TOOLS "" OFF)
option(NCNN_BUILD_EXAMPLES "" OFF)
复制代码

```

放到 ncnn toolchains 目录中，标准 cmake 三段式编译即可

```shell

mkdir build-esp32
cd build-esp32
cmake -DCMAKE_TOOLCHAIN_FILE=../toolchains/esp32c6.toolchain.cmake ..
make -j4
make install
复制代码

```

# 静态内存加载 mnist 模型

Beetle ESP32 C6 ncnn神经网络数字识别图3

依旧是2层conv+2层pool+1层fc的模型结构，极小的模型可以直接在静态内存区中加载

使用 ncnn 工具将模型文件转换为静态数组，方便把模型直接嵌入到代码中

```shell

ncnn2mem mnist.param mnist.bin mnist.id.h mnist.mem.h
复制代码

```

`mnist.mem.h` 文件就是我们需要的代码，里面包含2个数组，分别对应param和bin的内容

```cpp

#include "ncnn/net.h"
#include "mnist.mem.h"

extern "C" void app_main(void)
{
    ncnn::Net net;
    net.load_param(mnist_param_bin);
    net.load_model(mnist_bin);

    // 省略后续 ncnn 推理代码
}
复制代码

```

详细代码已更新到 https://github.com/nihui/ncnn_on_esp32

# ncnn esp32 避坑

在比较新的 ncnn 版本中默认开启了内存池优化，即推理的中间结果所占用的内存不会返还系统，而是保留给下一次使用，减少系统内存管理开销

为此，内存池会增加内存占用，这在 esp32 这种小内存开发板上容易引起内存不足，导致崩溃

在加载模型前设置 `net.opt.use_local_pool_allocator = false;` 禁用内存池功能

```cpp

#include "ncnn/net.h"
#include "mnist.mem.h"

extern "C" void app_main(void)
{
    ncnn::Net net;
    net.opt.use_local_pool_allocator = false; // 增加这行避免内存错误
    net.opt.use_sgemm_convolution = true; // esp32c6 内存更大，可以开sgemm加速

    net.load_param(mnist_param_bin);
    net.load_model(mnist_bin);

    // 省略后续 ncnn 推理代码
}
复制代码

```

# 效果和性能对比

最后编译项目，插上开发板，烧写

```shell

idf.py set-target esp32c6
idf.py menuconfig

idf.py build

idf.py flash
复制代码

```

tio 串口访问观测识别结果

```shell

tio /dev/ttyACM0
复制代码

```

可以看到正确识别到了数字7，以及耗时

```

ESP-ROM:esp32c6-20220919
Build:Sep 19 2022
rst:0xc (SW_CPU),boot:0xe (SPI_FAST_FLASH_BOOT)
Saved PC:0x4001975a
SPIWP:0xee
mode:DIO, clock div:2
load:0x40875720,len:0x1804
load:0x4086c110,len:0xe2c
load:0x4086e610,len:0x2e30
entry 0x4086c11a
I (22) boot: ESP-IDF v5.3-dev-2815-gbe06a6f5ff 2nd stage bootloader
I (23) boot: compile time Apr  6 2024 19:48:25
I (24) boot: chip revision: v0.0
I (26) boot.esp32c6: SPI Speed      : 80MHz
I (31) boot.esp32c6: SPI Mode       : DIO
I (36) boot.esp32c6: SPI Flash Size : 2MB
I (41) boot: Enabling RNG early entropy source...
I (46) boot: Partition Table:
I (50) boot: ## Label            Usage          Type ST Offset   Length
I (57) boot:  0 nvs              WiFi data        01 02 00009000 00006000
I (64) boot:  1 phy_init         RF data          01 01 0000f000 00001000
I (72) boot:  2 factory          factory app      00 00 00010000 00100000
I (79) boot: End of partition table
I (83) esp_image: segment 0: paddr=00010020 vaddr=420b0020 size=0eee0h ( 61152) map
I (117) esp_image: segment 1: paddr=0001ef08 vaddr=40800000 size=01110h (  4368) load
I (121) esp_image: segment 2: paddr=00020020 vaddr=42000020 size=aa960h (698720) map
I (407) esp_image: segment 3: paddr=000ca988 vaddr=40801110 size=08374h ( 33652) load
I (424) esp_image: segment 4: paddr=000d2d04 vaddr=40809490 size=00f64h (  3940) load
I (432) boot: Loaded app from partition at offset 0x10000
I (433) boot: Disabling RNG early entropy source...
I (444) cpu_start: Unicore app
I (454) cpu_start: Pro cpu start user code
I (454) cpu_start: cpu freq: 160000000 Hz
I (454) app_init: Application information:
I (457) app_init: Project name:     main
I (461) app_init: App version:      1
I (466) app_init: Compile time:     Apr  6 2024 19:48:22
I (472) app_init: ELF file SHA256:  1c5a8e8fc...
I (477) app_init: ESP-IDF:          v5.3-dev-2815-gbe06a6f5ff
I (484) efuse_init: Min chip rev:     v0.0
I (488) efuse_init: Max chip rev:     v0.99
I (493) efuse_init: Chip rev:         v0.0
I (498) heap_init: Initializing. RAM available for dynamic allocation:
I (505) heap_init: At 4080B3B0 len 00071260 (452 KiB): RAM
I (511) heap_init: At 4087C610 len 00002F54 (11 KiB): RAM
I (517) heap_init: At 50000000 len 00003FE8 (15 KiB): RTCRAM
I (525) spi_flash: detected chip: generic
I (529) spi_flash: flash io: dio
W (532) spi_flash: Detected size(4096k) larger than the size in the binary image header(2048k). Using the size in the binary image header.
I (546) sleep: Configure to isolate all GPIO pins in sleep state
I (552) sleep: Enable automatic switching of GPIO sleep configuration
I (559) coexist: coex firmware version: d96c1e51f
I (565) coexist: coexist rom version 5b8dcfa
I (570) main_task: Started on CPU0
I (570) main_task: Calling app_main()
Loading ncnn mnist model...Done.
Preparing input...Start Mesuring!
Done!
0: -1.68
1: 3.07
2: 12.28
3: 7.86
4: -17.84
5: -9.98
6: -15.03
7: 16.44
8: 0.54
9: -3.40
I think it is number 7!
Latency, avg: 426.81ms, max: 427.74, min: 426.69. Avg Flops: 1.83MFlops
Restarting now.
复制代码