Kinect on Windows 7 with Matlab

1 02 2011

This post has been moved to HERE

I have recently purchased a Kinect camera (although I do not even own an X-Box). It’s main purpose is to get the multi-touch working on it. Well, multi-touch: multi-sky-touch that is. For this i would be programming in C, but for now it is easier to develop in Matlab, so let’s focus on Matlab.

Kinect Hardware connections

The Kinect is provided with a normal USB connector and split to a normal power adapter. It gets quite hot after a while.

This post has been updated and moved to a new dedicated website HERE

Kinect Drivers
There are three options to get the driver working on your Windows machine (and one for your OS X beast):

  • CLNUI untested
  • Open-Kinect
  • OpenNI/NITE

Open-Kinect

If we are going to use Open-Kinect with Matlab, we will be using an external DLL written for Matlab after the Kinect is working. For this we will use the Image Acquisition Toolbox (IMAQ). I had a lot of trouble with this. Though, this seemed like the good option since the Image Acquisition-toolbox is made especially for these kinds of streaming video data, for which you would otherwise need buggy loops. So after some problems with IMAQ i switched to the other driver. Also, you need Visual Studio for this approach.

Open-NI and NITE

This worked great for me, although this is not entirely open-source, but i was not going to touch the source-code much anyway. So, lets get this working: Do this:

  1. Download Kinect Drivers (https://github.com/avin2/SensorKinect) and press the DOWNLOAD button. Search for the file dpinst-x86.exe and run it.
  2. Download the unstable binary which worked for me (http://www.openni.org/downloadfiles/2-openni-binaries) otherwise try the unstable binary. Scroll down and press LATEST UNSTABLE and download.
  3. Download the unstable NITE binary (http://www.openni.org/downloadfiles/2-openni-binaries), same fashion as (2), the install key is 0KOIk2JeIBYClPWVnMoRKn5cdY4=
  4. Download the unstable binary from http://www.openni.org/downloadfiles/30-openni-compliant-hardware-binaries and install.
  5. Plug in the Kinect. If you go to the device manager it should say Xbox NUI Audio unknown, this is obvious, and under PrimeSensor you’ve got your Cam and Motor
  6. For this step check http://www.studentguru.gr/blogs/vangos/archive/2011/01/20/how-to-successfully-install-kinect-windows-openni-nite.aspx
  7. Run a test to make sure it works from C:\Program Files\OpenNI\Samples\Bin\Release and C:\Program Files\Prime Sense\NITE\Samples\Bin\Release. If they do not work, do not proceed. Switch binaries; stable or unstable this time.
  8. Get http://sourceforge.net/projects/kinect-mex/ , press DOWNLOAD and unpack: then get to Matlab and run one of the test .m files. You can only execute the files that have _sample in them, the rest is compiled stuff you should not touch. Also, don’t touch the .XML files. Now, hacking begins!
  9. If you have any problems, drop a line in the comments.

Specifications

  • Streaming RGB ánd depth image: 20 FPS
  • Streaming depth image only: 30~35  FPS
  • Streaming without viewing: 35~50 FPS

Here a sample video in which i 3D reconstruct my desk-area.


update The Kinect also records audio ofcourse. There are, as far as i know, no drivers yet to get audio from the Kinect. This holds at least true for Matlab, and i’m not sure if they’ve managed to import audio to windows anyway.

update2 This Kinect method does not work for Matlab 2008. Try upgrading your version.

Advertisements

Actions

Information

33 responses

2 04 2011
tariq

how to get the point cloud data in the format. txt which consists of the value XYZ using Matlab

2 04 2011
Tim Zaman

The data is stored in your “workspace”. Rightclick on the matrix (XYZ data) you want to store.. then select export or something; then you can also export to .txt! But for Matlab-interchange, the .mat format is best.

1 04 2011
hermann

It works, but …additionally I wanna do something more (with matlab programming of course) with the raw IR/Depth data.

Although I use another device (named XtionPro, from PrimeSense and its partner) which seems similiar to kinect except that is has no RGB camera, you mex file and sample_niIRImage still works.

The next step…I wanna implement PointViewer sample of NITE with matlab, which can track the hand movement. The C++ code is available, however I have no experience on making mex file from the provided C++ library.

So…Maybe you are interested in providing more mex files as matlab interface to OpenNI/NITE ? I think it will attract more matlab users.

1 04 2011
Tim Zaman

Actually i recommend waiting for the official microsoft kinect drivers, i expect them to be released this month. ok? 🙂 also, this website will be entirely renovated next week, so im a bit busy! Plus, learning c++ is very useful and i recommend it to you!

1 04 2011
hermann

ok,thanks. ^_^

29 03 2011
khaled hossam emam

everything works perfect , the matlab sample files give no error but there is no output appearing ..only black screens ??

29 03 2011
khaled hossam emam

sorry…… forgot to say thanks for this help and support

17 03 2011
DMZ

Hi again Tim,

The function mxNICreateContext can read SamplesConfig.xml (for RGB+depth image) and SamplesIR_DepthConfig.xml (IR+depth). But it can only read one XML file per Matlab session. Do you have the same problem? How can I resolve, if it’s possible?

17 03 2011
DMZ

This is the error throwed when I try to read the second xml file:

Open failed: Bad Parameter sent to the device!
Check whether SamplesIR_DepthConfig.xml is available.

2 03 2011
Xiao

Excellent tutorial. Every thing works for me exceptc the matlab won’t execute the sample programes. I got this ‘Invalid MEX-file, … mxNiCreateContext.mexw64 is not a valid Win32 application’. I’m runing the matlab on a 64 bit windows 7. not sure if this might have something to do with the mex file? need your help.

2 03 2011
Tim Zaman

(1) do the sample (openni/primesense) executables work?? (2) does matlab say invalid win32 application?; that would be weird since it doesnt want win32 mode. (3) have you tried win32 compatible mode?.. also make sure openni and primesense are both for 64b

3 03 2011
Xiao

Ok, I simply switched to 32bit matlab and the samples started. However I don’t think the program is running correctly. The led light just flashes green, infrared projector is dark too. Don’t know why.

10 03 2011
Tim Zaman

Just hold on another month and Microsoft will release official Kinect drivers; and then, all academic hell will break loose! (in a positive way)

23 02 2011
Reza

Hi Tim,

How can i record video by matlab

23 02 2011
Tim Zaman

to record your video in matlab simply use CamStudio (freeware) that captures your screen. Matlab has a function of its own but its intensive and sucks.

22 02 2011
DMZ

Very good tutorial. Kinect works perfectly, and I can execute OpenNi and NITE samples. But I have a problem with Matlab .m files. When I execute it throws this:
“??? Attempt to execute SCRIPT mxNiCreateContext as a function.
Error in ==> sample_niIRImage at 35
context = mxNiCreateContext(‘SamplesIR_DepthConfig.xml’);”

Do you know where the problem is?

22 02 2011
Tim Zaman

try restart and reinstall of the matlab mex codes. very weird case, and make sure the XML file that is in the same folder is correct. Otherwise i have not yet seen this error.

23 02 2011
DMZ

I have in Matlab workspace two XML files: SamplesConfig.xml and SamplesIRConfig.xml.. and I don’t know what name should be correct.
I’m using Matlab 7.0.1, maybe this is the problem. Now I’m upgrading.

23 02 2011
DMZ

Sorry, Kinect Mex 1.11 includes 4 XML files.

23 02 2011
Tim Zaman

1.1 is for the openni stable build, and 1.11 is for the openni unstable, make sure that’s correct. it has nothing to do with the XML files, thats for sure. there was some trouble with 2008 i recon, try to upgrade; if it doesnt work then i’ll dive into it.

23 02 2011
DMZ

Thank you for your interest Tim. I have installed version 2010b and it works perfect!! 🙂

16 02 2011
Anonymous

Hi Tim,
How can i understand matlab is Ok with this program?

16 02 2011
Tim Zaman

Matlab works great with the kinect, it is so fast it can stream 30fps real-time. Could you maybe elaborate your question?

16 02 2011
Anonymous

Hello,

7- Run a test to make sure it works.

please give me an example

16 02 2011
Tim Zaman

You have installed OpenNI and Prime Sense. They are installed in Program Files. If you indeed installed them, they install sample .exe files. Try one of the .exe files in ..\Program Files\OpenNI\Samples\Bin\Release, for instance the file “SimpleViewer.net.exe”.

16 02 2011
Reza

After executing SimpleViewer.net.exe , I have a small black window after 2 seconds a white blank window is apperad. Is it correct?

16 02 2011
Tim Zaman

no this is not correct. first the black small screen appears, then the info from the kinect. you can see that the kinect works as well when the lens on the left emits some red light (infrared laser)

15 02 2011
Reza

Hi,

my computer is 64 bit i have problem with dpinst-x86

15 02 2011
Tim Zaman

dpinst-x86 is for 32 bit! you have to take the 64bit version =)

15 02 2011
Reza

Thank you
I try C:\Program Files\OpenNI\Samples\Bin\Release and C:\Program Files\Prime Sense\NITE\Samples\Bin\Release for 64 bit version either
stable version and unstable version but it dose not working ?

15 02 2011
Tim Zaman

Try dpinst-amd64.exe (if you have a 64-bit processor). Maybe you could try forcing to run it in 32bit mode or something, isn’t that possible? Otherwise i wouldn’t know, and if you’re really stuck you could try the Matlab IMAQ adapter, which I personally dislike.

9 02 2011
JK

Hey this might be an easy question for you, but I am not very experienced with Matlab and need to use it to process the data coming in from the Kinect to do object detection. I was wondering how you directly input the data to perform an operation on it. I am building a robot that will autonomously navigate a rocky area and matlab seems to be the most capable program. After performing the object detection, i’m going to convert the operation into C and use that in the program I am building. Is there any way you could please help me out??

9 02 2011
Tim Zaman

Use the sample .m files from the matlab drivers, they will tell you roughly how it works and thenm you can play with that.




%d bloggers like this: