💡 在Intel-UP2上重现PoseNet：解决摄像头与TensorFlow问题 🚀

This blog will follow the logic:

What did we do? What problems did we meet? What attempts did we make?

This blog can also help you if you'd like to make a reproduction of PoseNet on Intel-UP2.

We are all interested in ML, and want to reproduce a PoseNet on a board, since tiny up2 and ML is much more attractive than a PC.

no camera on up2 boared
We found an external video device and linked it to up2 by usb port.
But something unexpected happened: object not found
First we linked in and out to check if the device worked. It worked and was identified.

ls /dev/v* ls are used to show folder contents, v* are used to show all files starting with v, to check if there is a video.

We tried many libraries to config the external video, such as camorama, gconf2, cheese (no device found), guvcview (no video device found) but all failed, until we found this and tried ffmpeg.

We succeeded!!

By the way, we first met /dev/video0 permission denied because of laking of sudo.

FPS was shown on the video.

PoseNet on up2
We all have interests in tensorflow.js, aiming to reproduct the posenet demo on up2, because we think running on a lighter board rather than a PC is much more interesting and meaningful.
But we met bugs when running a tfjs demo on a website.
this browser does not support video capture,or this device does not have a camera.
After searching, we guess:
Browser does not support webgl: we allowed the gpu acceleration and many others, but we still find the posenet on p5.js not working while the welgl was supported as shown on webglreport.
Here's the problems we found: up2 is lack of discrete graphics card or the GPU version is too low to afford a posenet on p5js.

Therefore we decided to run it on the local computer and based on python language.
We decided to choose a repository on github and tried to reproduce it first. I first chose a pytorch form expecting a higher speed, but we met AssertionError: Torch not compiled with CUDA enabled, and Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from nvidia.com/Download/index.aspx. So finally we turned to the python form without the slight acceleration.

SyntaxError using pip
After I update, upgrade python to python3.5, install pip, and decide to install libraries, SyntaxError: invalid syntax occurred. I had changed the ubuntu source, and tried updating pip, changing pip source to aliyun mirror, unistalling and reinstalling, giving sudo, but still failed. Finally by this site, I learned that in Ubuntu14 and python3.5, the pip should not be so high, or it will meet expectations.

The warning looks like:

So we should go back to the previous version:

wget https://bootstrap.pypa.io/3.5/get-pip.py
python3 get-pip.py
复制代码

though there will be a WARNING that said pip is being invoked by an old script wrapper. This will fail in a future version of pip.

illegal instruction (core dump) importing tensorflow
Following the README, I installed tensorflow but met illegal instruction (core dumped). Then I was told on https://tensorflow.google.cn/install/source that

So I ran

pip install tensorflow==1.5
复制代码

to unistall and get a former vision.
The next step is to install missed libraries. After that it seems that all was done but when I ran it, downloading model: timed out.
I already changed the ubuntu and pip to mirror source, it can only be the Internet's fault.
However, we can change the model to 50 and it will run well.
Run

python3 image_demo.py --model 50
复制代码

we can get a folder named output.
Then I tried to run the webcam demo, a real time demo to detect and draw a man's skeleton. However, OSError: webcam failure.
After debugging function by function and line by line, I found that it was in

cv2.VideoCapture(args.cam_id).read()
复制代码

cv2.VideoCapture failure
Reading cv2 Capture Video from Camera tutorial, I got

Sometimes, cap may not have initialized the capture. In that case, this code shows error. You can check whether it is initialized or not by the method cap.isOpened(). If it is True, OK. Otherwise open it using cap.open().

cap = cv2.VideoCapture(args.cam_id)
cap.isOpened()  # false
cap.open(0) # false   =>  cap.isopen(1) usb camra
cap.read() # none,false
cap.open(n) # false when n=from 1 to inf by loop
复制代码

Luckily we also had Raspberry Pi, and it runs well. And the parameter of cap.open() means built-in camera for 0 and external for 1.
Searching on the Internet, many people said that we need to run

v4l2-ctl --list-devices
复制代码

to obtain the supported camera devices. But on up2 it meet permission deny even if I add sudo.
The key is to run

sudo chmod 777 /dev/video0
复制代码

to give read, write and execute permission.

Here it comes!!

It can not only act well to detect the face, but also the whole body, even if the man was walking or doing other things. And the result was not influenced by the messy background or the lights.
To sum up...
To be honest, we truly had a long journey with up2....
Running PoseNet demo of python seems a piece of cake usually, but on up2 we met unexpected problems on and on and on and on. At first we decided to detect sports using up2, such as role-jumping, squating and others, but now we think it costs too much and the FPS was too low to satisfy such need. Maybe other usage requiring less hashrate are more available.
Whatever, such a journey is unforgettable.

Intel-UP2, Tensorflow, PoseNet, Python, Ubuntu14

check_video.png (17.14 KB, 下载次数: 3068)