I bought a Kinect about a year ago with the idea of using it in some, albeit very experimental, robotic vision applications. At the time the only way to use the Kinect for something other than as game controller was to download and install an open source device driver and its associated software library. After several failed attempts I put my Kinect in the closet where it would have to wait for a better day. That day came last summer when Microsoft finally embraced the maker/hobbyist community and officially released the Kinect Software Development Kit (SDK) to the general public for non-commercial use. Although restricted to Windows 7 (no XP support) it provided a clean and reliable way for me to restart my experiments. From these experiments my K-See (pronounced Casey and is short for Kinect seeing) robot was born.
The K-See Platform
Before we explore the wondrous world of the Kinect letís take a look at the K-See platform. This robot was originally intended for different project but I re-purposed it for use in my Kinect experiments. Coincidently it ended up closely matching the ďstandardĒ robot configuration that Microsoft recommends for use with the Kinect and their Robotic Development Studio (MSRDS). The one major difference between K-See and the standard robot is the ability to pan the Kinect left or right by 90 degrees. To accomplish this, I mounted the Kinect on a Lynxmotion Base Rotate Kit. Being able to pan the Kinect greatly increases its field of view without the need to rotate the entire robot. As a side note, there are also a number of commercially available robots that follow this same standard with the most notable being the new Parallax Eddie.
As I mentioned, there is a good side and a dark side to using the Kinect. On the good side the Kinect requires a reasonably powerful laptop PC to operate. This PC can also be used to display a sophisticated user interface on its LCD display and allow interaction via its built-in mouse and keyboard. The PC also provides Wi-Fi, and Bluetooth communications to other devices and computers on your. You can even use Remote Desktop to control the robot from anywhere in the world via the internet. All these great features come for free when you use a PC with your robot. But there is a very dark side to all of this great stuff. It is the simple fact that itís a PC! This means you have all that baggage that comes with a PC. Not only must you consider its power requirements, size and weight but also all the other nasty little things like the long boot time, virus and spyware vulnerability, frequent software updates, as well as real-time performance issues. But donít let that discourage you. As we learned over the seemingly endless and tedious years of PC use, all these issues can be laboriously managed.
My Simple K-See Application
Since writing code in C# is part of my day job, I built my own application for K-See. It is written in C# and uses the Kinect SDK and a WPF user interface. It does not use the MSRDS. This application provides a basic navigation system and obstacle avoidance using a behavior-based programming architecture. This architecture uses a collection of simple behaviors that take inputs from one or more of the sensors (both the Kinect and the Pings) and then triggers an action that is sent to motor drive system. Each behavior is written as a single C# method that is called sequentially based on its priority. The priority determines when a behavior can get access to the motor drive system. K-See has five behaviors executed in the following order:
- See Glyph
- Go to Glyph
What is all this glyph stuff? A glyph is a special sign built from a 5x5 matrix of black and white squares. These optical glyphs can be reliably identified in the Kinect’s color stream (from its RGB camera) by using an open source software library called the Glyph Recognition and Tracking Framework (GRATF) which is built on top of a computer vision library called AForge.NET. Both these libraries are written in C# and can be easily integrated with the Kinect SDK. Once identified, the bearing to the glyph can be used to steer a robot towards it. This is the same idea behind my Beacon Navigation System but does not require any active beacons, just simple signs that you can create and print using any graphics program like Photoshop. The “See Glyph” behavior looks for the specified optical glyph and if one is found will let the remaining behaviors execute. The Blocked behavior uses the forward facing Ping and the Kinect depth data stream to determine if the path ahead is blocked. If so it stops the robot. The Move behavior simply instructs the motor drive to drive forward. The Avoid behavior uses a combination of the Pings and Kinect depth data to steer the robot away from potential obstacles. Finally the “Go to Glyph” behavior attempts to steer the robot to the desired glyph. This behavior has the addition capability to follow map directions and make turns based on distances traveled. When studying the code you will see that the behaviors will use two classes that you can reuse in your own applications. Using the depth data the CorridorFinder class will determine the best direction to steer based on the obstacles in its field of view. You can specify the height and width of the corridor you want. The GlyphFinder class will compute a bearing to the specified glyph using the RGB color stream. It wraps all the functionality of the GRATF library in an easy to use interface.