Ubuntu_ROS中应用kinect v2笔记
个人觉得最重要的资料如下:
1.
Microsoft Kinect v2 Driver Released
http://www.ros.org/news/2014/09/microsoft-kinect-v2-driver-released.html
2.
https://github.com/OpenKinect/libfreenect2
3.
https://github.com/code-iai/iai_kinect2
http://v.youku.com/v_show/id_XMTQyNDAzNTM2OA
http://pan.baidu.com/s/1eRrM4QA
rtabmap kinect2:
http://official-rtab-map-forum.67519.x6.nabble.com/rtabmap-for-kinect-v2-on-ROS-Indigo-td61.html
https://github.com/introlab
附件::
Driver for Kinect for Windows v2 (K4W2) devices (release and developer preview).
Note: libfreenect2 does not do anything for either Kinect for Windows v1 or Kinect for Xbox 360 sensors. Use libfreenect1 for those sensors.
If you are using libfreenect2 in an academic context, please cite our work using the following DOI:
If you use the KDE depth unwrapping algorithm implemented in the library, please also cite this ECCV 2016 paper.
This driver supports:
Missing features:
Watch the OpenKinect wiki at www.openkinect.org and the mailing list at https://groups.google.com/forum/#!forum/openkinect for the latest developments and more information about the K4W2 USB protocol.
The API reference documentation is provided here https://openkinect.github.io/libfreenect2/.
Intel and NEC USB 3.0 host controllers are known to work. ASMedia controllers are known to not work.
Virtual machines likely do not work, because USB 3.0 isochronous transfer is quite delicate.
It has been reported to work for up to 5 devices on a high-end PC using multiple separate PCI Express USB3 expansion cards (with NEC controller chip). If you're using Linux, you may have to increase USBFS memory buffers. Depending on the number of Kinects, you may need to use an even larger buffer size. If you're using an expansion card, make sure it's not plugged into an PCI-E x1 slot. A single lane doesn't have enough bandwidth. x8 or x16 slots usually work.
First, check https://github.com/OpenKinect/libfreenect2/wiki/Troubleshooting for known issues.
When you report USB issues, please attach relevant debug log from running the program with environment variable LIBUSB_DEBUG=3
, and relevant log from dmesg
. Also include relevant hardware information lspci
and lsusb -t
.
This doesn't interfere with the Microsoft SDK. Do not install both UsbDK and libusbK drivers
To uninstall the libusbK driver (and get back the official SDK driver, if installed):
If you already had the official SDK driver installed and you want to use it:
This will enumerate the Kinect sensor again and it will pick up the K4W2 SDK driver, and you should be ready to run KinectService.exe again immediately. You can go back and forth between the SDK driver and the libusbK driver very quickly and easily with these steps.
install_libusb_vs2015.cmd
. If you see some errors, you can always open the cmd files and follow the git commands, and maybe build libusb_201x.sln
with Visual Studio by hand. Building with "Win32" is not recommended as it results in lower performance.c:\libjpeg-turbo64
or depends/libjpeg-turbo64
, or anywhere as specified by the environment variable TurboJPEG_ROOT
.depends/glfw
(rename glfw-3.x.x.bin.WIN64
to glfw
), or anywhere as specified by the environment variable GLFW_ROOT
.C:\Program Files...
).install
, you may change it by editing CMAKE_INSTALL_PREFIX
.
mkdir build && cd build cmake .. -G "Visual Studio 12 2013 Win64" cmake --build . --config RelWithDebInfo --target install
Or -G "Visual Studio 14 2015 Win64"
..\install\bin\Protonect.exe
, or start debugging in Visual Studio.install\bin
to C:\Program Files\OpenNI2\Tools\OpenNI2\Drivers
. Then run C:\Program Files\OpenNI\Tools\NiViewer.exe
. Environment variable LIBFREENECT2_PIPELINE
can be set to cl
, cuda
, etc to specify the pipeline.Use your favorite package managers (brew, ports, etc.) to install most if not all dependencies:
./bin/Protonect
make install-openni2
(may need sudo), then run NiViewer
. Environment variable LIBFREENECT2_PIPELINE
can be set to cl
, cuda
, etc to specify the pipeline.Note: Ubuntu 12.04 is too old to support. Debian jessie may also be too old, and Debian stretch is implied in the following.
sudo dpkg -i debs/libusb*deb
sudo apt-get install libusb-1.0-0-dev
sudo apt-get install libturbojpeg libjpeg-turbo8-dev
sudo apt-get install libturbojpeg0-dev
sudo dpkg -i debs/libglfw3*deb; sudo apt-get install -f; sudo apt-get install libgl1-mesa-dri-lts-vivid
(If the last command conflicts with other packages, don't do it.)cmake -DENABLE_OPENGL=OFF
later.sudo apt-get install libglfw3-dev
sudo apt-add-repository ppa:floe/beignet; sudo apt-get update; sudo apt-get install beignet-dev; sudo dpkg -i debs/ocl-icd*deb
sudo apt-get install beignet-dev
# echo 0 >/sys/module/i915/parameters/enable_cmd_parser
is needed. See more known issues at https://www.freedesktop.org/wiki/Software/Beignet/.apt-get install opencl-headers
.mkdir -p /etc/OpenCL/vendors; echo /usr/lib/arm-linux-gnueabihf/mali-egl/libmali.so >/etc/OpenCL/vendors/mali.icd; apt-get install opencl-headers
.clinfo
to verify if you have correctly set up the OpenCL stack.cuda-repo-ubuntu1404...*.deb
("deb (network)") from Nvidia website, follow their installation instructions, including apt-get install cuda
which installs Nvidia graphics driver.apt-get install cuda
, use sudo prime-select intel
to use Intel GPU for desktop.sudo dpkg -i debs/{libva,i965}*deb; sudo apt-get install -f
sudo apt-get install libva-dev libjpeg-dev
sudo apt-add-repository ppa:deb-rob/ros-trusty && sudo apt-get update
(You don't need this if you have ROS repos), then sudo apt-get install libopenni2-dev
sudo apt-get install libopenni2-dev
.cmake -Dfreenect2_DIR=$HOME/freenect2/lib/cmake/freenect2
for CMake based third-party application to find libfreenect2.sudo cp ../platform/linux/udev/90-kinect2.rules /etc/udev/rules.d/
, then replug the Kinect../bin/Protonect
sudo apt-get install openni2-utils && sudo make install-openni2 && NiViewer2
. Environment variable LIBFREENECT2_PIPELINE
can be set to cl
, cuda
, etc to specify the pipeline.Please read this README and the ones of the individual components throughly before asking questions. We get a lot of repeated questions, so when you have a problem, we urge everyone to check the github issues (including closed ones). Your issue is very likely discussed there already.
The goal of this project is to give you a driver and the tools needed to receive data from the Kinect-2 sensor, in a way useful for robotics. You will still need to know how to use ROS to make use of it. Please follow the ROS tutorials. You will also need to learn how to work with point-clouds, or depth-clouds, or images (computer vision) to do useful things with the data.
Note: Please use the GitHub issues for questions and problems regarding the iai_kinect2 package and its components. Do not write emails.
This is a collection of tools and libraries for a ROS Interface to the Kinect One (Kinect v2).
It contains:
First you should look at this FAQ and the FAQ from libfreenect2.Secondly, look at issue page from libfreenect2 andthe issue page of iai_kinect2 for similar issues and solutions.
Point clouds are only published when the launch file is used. Make sure to start kinect2_bridge with roslaunch kinect2_bridge kinect2_bridge.launch
.
Short answer: No.
Long answer: Yes, it is possible to compile this package with OpenCV 3.0, but it will not work.This is because cv_bridge is used, which itself is compiled with OpenCV 2.4.x in ROS Indigo/Jade andlinking against both OpenCV versions is not possible. Working support for OpenCV 3.0 might come with a future ROS release.
There are many reasons why kinect2_bridge
might not working. The first thing to find out whether the problem is related to kinect2_bridge
or libfreenect2
.A good tool for testing is Protonect
, it is a binary located in libfreenect2/build/bin/Protonect
.It uses libfreenect2 directly with a minimal dependency on other libraries, so it is a good tool for the first tests.
Execute:
./Protonect gl
to test OpenGL support../Protonect cl
to test OpenCL support../Protonect cpu
to test CPU support.Before running kinect2_bridge
please make sure Protonect
is working and showing color, depth and ir images.If some of them are black, than there is a problem not related to kinect2_bridge
and you should look at the issues from the libfreenect2 GitHub page for help.
If one of them works, try out the one that worked with kinect2_bridge
: rosrun kinect2_bridge kinect2_bridge _depth_method:=<opengl|opencl|cpu>
.You can also change the registration method with _reg_method:=<cpu|opencl>
.
If that is the case, you have to make sure that Protonect
uses the same version of libfreenect2
as kinect2_bridge
does.To do so, run make
and sudo make install
in the build folder again. And try out kinect2_bridge
again.
cd libfreenect2/build
make & sudo make install
Also make sure that you are not using OpenCV 3.0.
If it is still crashing, compile it in debug and run it with gdb:
cd <catkin_ws>
catkin_make -DCMAKE_BUILD_TYPE="Debug"
cd devel/lib/kinect2_bridge
gdb kinect2_bridge
// inside gdb: run until it crashes and do a backtrace
run
bt
quit
Open an issue and post the problem description and the output from the backtrace (bt
).
This is the normal behavior. 'kinect2_bridge' will only process data when clients are connected (ROS nodes listening to at least one of the topics).This saves CPU and GPU resources. As soon as you start the kinect_viewer
or rostopic hz
on one of the topics, processing should start.
rosdep
will output errors on not being able to locate [kinect2_bridge]
and [kinect2_registration]
.That is fine because they are all part of the iai_kinect2 package and rosdep
does not know these packages.
This indicates problems with the USB connection.
First of all, check the issue pages on GitHub for similar issues, as they might contain solutions for them.By default you will only see the open issues, but if you click on closed
you will the the ones solved. There is also a search field which helps to find similar issues.
If you found no solution in the issues, feel free to open a new issue for your problem. Please describe your problem in detail and provide error messages and log output.
cmake .. -DENABLE_CXX11=ON
instead of cmake ..
If something is not working, check out the latest stable release, for example git checkout v0.2.0
.rosdep
will output errors on not being able to locate [kinect2_bridge]
and [depth_registration]
.That is fine because they are all part of the iai_kinect2 package and rosdep
does not know these packages.
Note: If you installed libfreenect2 somewhere else than in $HOME/freenect2
or a standard location like /usr/local
you have to specify the path to it by adding -Dfreenect2_DIR=path_to_freenect2/lib/cmake/freenect2
to catkin_make
.kinect2_bridge
:
roslaunch kinect2_bridge kinect2_bridge.launchkinect2_calibration
. Further detailskinect2_bridge/data/<serialnumber>
folder. Further detailskinect2_bridge
and view the results using rosrun kinect2_viewer kinect2_viewer kinect2 sd cloud
.Install the latest version of the AMD Catalyst drivers from https://support.amd.com and follow the instructions. Also install opencl-headers
.
sudo apt-get install opencl-headers
Go to developer.nvidia.com/cuda-downloads and select linux
, x86_64
, Ubuntu
, 14.04
, deb(network)
.Download the file and follow the instructions. Also install nvidia-modprobe
and opencl-headers
.
sudo apt-get install nvidia-modprobe opencl-headers
You also need to add CUDA paths to the system environment, add these lines to you ~/.bashrc
:
export LD_LIBRARY_PATH="/usr/local/cuda/lib64:${LD_LIBRARY_PATH}"
export PATH="/usr/local/cuda/bin:${PATH}"
A system-wide configuration of the libary path can be created with the following commands:
echo "/usr/local/cuda/lib64" | sudo tee /etc/ld.so.conf.d/cuda.conf
sudo ldconfig
You can either install a binary package from a PPA like ppa:floe/beignet, or build beignet yourself.It's recommended to use the binary from the PPA.
sudo add-apt-repository ppa:floe/beignet && sudo apt-get update
sudo apt-get install beignet beignet-dev opencl-headers
If you used iai_kinect2
for your work, please cite it.
@misc{iai_kinect2,
author = {Wiedemeyer, Thiemo},
title = {{IAI Kinect2}},
organization = {Institute for Artificial Intelligence},
address = {University Bremen},
year = {2014 -- 2015},
howpublished = {\url{https://github.com/code-iai/iai\_kinect2}},
note = {Accessed June 12, 2015}
}
The result should look something similar to this (may depend on the bibliography style used):
T. Wiedemeyer, “IAI Kinect2,” https://github.com/code-iai/iai_kinect2,
Institute for Artificial Intelligence, University Bremen, 2014 – 2015,
accessed June 12, 2015.
Here are some screenshots from our toolkit: