IntelUP2-Ubuntu14-PoseNet-Reproduction
本帖最后由 zhanghf 于 2021-3-9 00:27 编辑This blog will follow the logic:
What did we do? What problems did we meet? What attempts did we make?This blog can also help you if you'd like to make a reproduction of PoseNet on Intel-UP2.
We are all interested in ML, and want to reproduce a PoseNet on a board, since tiny up2 and ML is much more attractive than a PC.
no camera on up2 boared
We found an external video device and linked it to up2 by usb port.
But something unexpected happened: object not found
First we linked in and out to check if the device worked. It worked and was identified.
ls /dev/v* ls are used to show folder contents, v* are used to show all files starting with v, to check if there is a video.https://mc.dfrobot.com.cn/forum.php?mod=image&aid=121292&size=300x300&key=3e558b277d6710ca&nocache=yes&type=fixnone
https://internal-api-drive-stream.feishu.cn/space/api/box/stream/download/all/boxcnXjZVHURXqHWRRVZaApPzWb/?mount_node_token=doccnvIlVvbqgMboSpRroHzQVNd&mount_point=doc_image
We tried many libraries to config the external video, such as camorama, gconf2, cheese (no device found), guvcview (no video device found) but all failed, until we found this and tried ffmpeg.
https://internal-api-drive-stream.feishu.cn/space/api/box/stream/download/all/boxcnezkmqJYM5KqcN6uE7Jm2Xe/?mount_node_token=doccnvIlVvbqgMboSpRroHzQVNd&mount_point=doc_image
https://mc.dfrobot.com.cn/forum.php?mod=image&aid=121293&size=300x300&key=663c058098c661fc&nocache=yes&type=fixnone
We succeeded!!
https://mc.dfrobot.com.cn/forum.php?mod=image&aid=121294&size=300x300&key=ea0d1dd413e84358&nocache=yes&type=fixnone
By the way, we first met /dev/video0 permission denied because of laking of sudo.
https://internal-api-drive-stream.feishu.cn/space/api/box/stream/download/all/boxcnl4Ouytax9LuyHC05CGk7qc/?mount_node_token=doccnvIlVvbqgMboSpRroHzQVNd&mount_point=doc_image
FPS was shown on the video.
PoseNet on up2
We all have interests in tensorflow.js, aiming to reproduct the posenet demo on up2, because we think running on a lighter board rather than a PC is much more interesting and meaningful.
But we met bugs when running a tfjs demo on a website.
this browser does not support video capture,or this device does not have a camera.
After searching, we guess:
Browser does not support webgl: we allowed the gpu acceleration and many others, but we still find the posenet on p5.js not working while the welgl was supported as shown on webglreport.
Here's the problems we found: up2 is lack of discrete graphics card or the GPU version is too low to afford a posenet on p5js.
Thereforewe decided to run it on the local computer and based on python language.
We decided tochoose a repository on github and tried to reproduce it first. I first chose a pytorch form expecting a higher speed, but we met AssertionError: Torch not compiled with CUDA enabled, and Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from nvidia.com/Download/index.aspx. So finally we turned to the python form without the slight acceleration.
SyntaxError using pip
After I update, upgrade python to python3.5, install pip, and decide to install libraries, SyntaxError: invalid syntax occurred. I had changed the ubuntu source, and tried updating pip, changing pip source to aliyun mirror, unistalling and reinstalling, giving sudo, but still failed. Finally by this site, I learned that in Ubuntu14 and python3.5, the pip should not be so high, or it will meet expectations.
The warning looks like:https://mc.dfrobot.com.cn/forum.php?mod=image&aid=121295&size=300x300&key=5984cb7ef3824c48&nocache=yes&type=fixnone
https://internal-api-drive-stream.feishu.cn/space/api/box/stream/download/all/boxcnpeEc38kBvafmuc5cLFmDLb/?mount_node_token=doccnvIlVvbqgMboSpRroHzQVNd&mount_point=doc_image
So we should go back to the previous version:
wget https://bootstrap.pypa.io/3.5/get-pip.py
python3 get-pip.py
though there will be a WARNING that said pip is being invoked by an old script wrapper. This will fail in a future version of pip.
illegal instruction (core dump) importing tensorflow
Following the README, I installed tensorflow but met illegal instruction (core dumped). Then I was told on https://tensorflow.google.cn/install/source that
https://internal-api-drive-stream.feishu.cn/space/api/box/stream/download/all/boxcnP5WCX7KRkLBYTYLKhu8hBc/?mount_node_token=doccnvIlVvbqgMboSpRroHzQVNd&mount_point=doc_image
https://mc.dfrobot.com.cn/forum.php?mod=image&aid=121299&size=300x300&key=1ec81af8568f53c2&nocache=yes&type=fixnone
So I ran pip install tensorflow==1.5
to unistall and get a former vision.
The next step is to install missed libraries. After that it seems that all was done but when I ran it, downloading model: timed out.
I already changed the ubuntu and pip to mirror source, it can only be the Internet's fault.
However, we can change the model to 50 and it will run well.
Run python3 image_demo.py --model 50
we can get a folder named output.
Then I tried to run the webcam demo, a real time demo to detect and draw a man's skeleton. However, OSError: webcam failure.
After debugging function by function and line by line, I found that it was in
cv2.VideoCapture(args.cam_id).read()
cv2.VideoCapture failure
Reading cv2 Capture Video from Camera tutorial, I got
Sometimes, cap may not have initialized the capture. In that case, this code shows error. You can check whether it is initialized or not by the method cap.isOpened(). If it is True, OK. Otherwise open it using cap.open().cap = cv2.VideoCapture(args.cam_id)
cap.isOpened()# false
cap.open(0) # false =>cap.isopen(1) usb camra
cap.read() # none,false
cap.open(n) # false when n=from 1 to inf by loop
Luckily we also had Raspberry Pi, and it runs well. And the parameter of cap.open() means built-in camera for 0 and external for 1.
Searching on the Internet, many people said that we need to run v4l2-ctl --list-devices
to obtain the supported camera devices. But on up2 it meet permission deny even if I add sudo.
The key is to run sudo chmod 777 /dev/video0to give read, write and execute permission.
Here it comes!!
https://internal-api-drive-stream.feishu.cn/space/api/box/stream/download/all/boxcn2at9fJMahmUZaAgfkyKqTf/?mount_node_token=doccnvIlVvbqgMboSpRroHzQVNd&mount_point=doc_image
https://mc.dfrobot.com.cn/forum.php?mod=image&aid=121296&size=300x300&key=cfa06b1ffb75b85d&nocache=yes&type=fixnone
https://internal-api-drive-stream.feishu.cn/space/api/box/stream/download/all/boxcnrIBwmoFLyY0Db4LK6ixQUg/?mount_node_token=doccnvIlVvbqgMboSpRroHzQVNd&mount_point=doc_image
https://mc.dfrobot.com.cn/forum.php?mod=image&aid=121297&size=300x300&key=037f0033165c3501&nocache=yes&type=fixnone
https://internal-api-drive-stream.feishu.cn/space/api/box/stream/download/all/boxcnEyMSpdIMP3arhRuODLl3Uh/?mount_node_token=doccnvIlVvbqgMboSpRroHzQVNd&mount_point=doc_image
https://mc.dfrobot.com.cn/forum.php?mod=image&aid=121298&size=300x300&key=6c7df2988bcca2ee&nocache=yes&type=fixnone
https://internal-api-drive-stream.feishu.cn/space/api/box/stream/download/all/boxcnXIaMrQZocfNoIPRNBoHZxb/?mount_node_token=doccnvIlVvbqgMboSpRroHzQVNd&mount_point=doc_image
https://mc.dfrobot.com.cn/forum.php?mod=image&aid=121300&size=300x300&key=c1e8d68a1509d3b1&nocache=yes&type=fixnone
It can not only act well to detect the face, but also the whole body, even if the man was walking or doing other things. And the result was not influenced by the messy background or the lights.
To sum up...
To be honest, we truly had a long journey with up2....
Running PoseNet demo of python seems a piece of cake usually, but on up2 we met unexpected problems on and on and on and on. At first we decided to detect sports using up2, such as role-jumping, squating and others, but now we think it costs too much and the FPS was too low to satisfy such need. Maybe other usage requiring less hashrate are more available.
Whatever, such a journey is unforgettable.
页:
[1]