My previous post looked into my experimentation with using Visual Gesture Builder to create a gesture builder database (gbd) with a view to incorporating code into an application for detecting those trained gestures. I used the example of a military salute and will now describe the code for using this in a c#/xaml windows store app. I started by using some boilerplate code from a previous post which registers events to receive skeleton data (required for gesture detection) and colour data (to display the colour image from the RGB camera).
- _gestureDatabase = new VisualGestureBuilderDatabase(@"Gestures/salute.gbd");
- _gestureFrameSource = new VisualGestureBuilderFrameSource(_kinect, 0);
- // Add all gestures in the database to the framesource..
- _gestureFrameSource.AddGestures(_gestureDatabase.AvailableGestures);
- foreach (var gesture in _gestureDatabase.AvailableGestures)
- {
- if (gesture.Name == "salute")
- {
- _salute = gesture;
- }
- if (gesture.Name == "saluteProgress")
- {
- _saluteProgress = gesture;
- }
- }
- _gestureFrameReader = _gestureFrameSource.OpenReader();
- _gestureFrameReader.IsPaused = true;
- _gestureFrameReader.FrameArrived += reader_FrameArrived;
This snippet shows the initialisation code for importing the gesture database and registering to receive events when new data arrives.Note that I hold references to Gesture objects so that I can identify these when the data arrives.The two gestures I am interested in are the “salute” gesture which is a discrete gesture and “saluteProgress” which is a continuous gesture and will report progress for the salute. (Note. details of what, how and why for those different gesture types are in my previous post). This code follows the familiar pattern followed by the Kinect SDK of creating a frame source and opening a reader on that source which can be used to register for data events.Since we need a tracking id for gesture detection I have paused the reader until we have a valid one (this will be retrieved from the skeleton data, see code snippet below).
- if (body != null && body.IsTracked)
- {
- _gestureFrameSource.TrackingId = body.TrackingId;
- _gestureFrameReader.IsPaused = false;
- break;
- }
When the gesture data arrives, info about the current status of each gesture can be pulled out if the frame and in my sample I just display these on simple UI elements.
- void reader_FrameArrived(VisualGestureBuilderFrameReader sender, VisualGestureBuilderFrameArrivedEventArgs args)
- {
- using (var frame = args.FrameReference.AcquireFrame())
- {
- if (frame != null && frame.DiscreteGestureResults != null)
- {
- var result = frame.DiscreteGestureResults[_salute];
- if (result.Detected == true)
- {
- var progressResult = frame.ContinuousGestureResults[_saluteProgress];
- Progress.Value = progressResult.Progress;
- }
- else
- {
- Progress.Value = 0.0;
- }
- GestureText.Text = result.Detected ? "TRUE" : "FALSE";
- ConfidenceText.Text = result.Confidence.ToString();
- }
- }
- }
Here’s a screenshot of the sample in action:
The sample project is located here http://1drv.ms/1C4RQGU
Excuse me，When I calling my own . gbd file builded by VGB, it has an error:
“System. InvalidOperationException” type of exception happen in Microsoft.Kinect.VisualGestureBuilder.dll , but not handled in the user code
Other information: This API has returned an exception from the an HRESULT: 0 x80004005
I don’t know what’s wrong with it.. How to call my own .gbd in the code? Thank you very much!
If you go into your project properties and change the CPU type to x64 that should fix it.
Were you able to solve this problem?
I am facing the same difficulty while using my own .gbd file.
I build this project and the following hints appeared.
Error 1 Could not copy the file “E:\DetectVGBGesture\DetectVGBGesture\Gestures\DrumHit.gbd” because it was not found. DetectVGBGesture