Kinect

Visual Gesture Builder – Kinect 4 Windows v2 (code)

My previous post looked into my experimentation with using Visual Gesture Builder to create a gesture builder database (gbd) with a view to incorporating code into an application for detecting those trained gestures. I used the example of a military salute and will now describe the code for using this in a c#/xaml windows store app. I started by using some boilerplate code from a previous post which registers events to receive skeleton data (required for gesture detection) and colour data (to display the colour image from the RGB camera).

 

  1. _gestureDatabase = new VisualGestureBuilderDatabase(@"Gestures/salute.gbd");
  2. _gestureFrameSource = new VisualGestureBuilderFrameSource(_kinect, 0);
  3.  
  4. // Add all gestures in the database to the framesource..
  5. _gestureFrameSource.AddGestures(_gestureDatabase.AvailableGestures);
  6.  
  7. foreach (var gesture in _gestureDatabase.AvailableGestures)
  8. {
  9.     if (gesture.Name == "salute")
  10.     {
  11.         _salute = gesture;
  12.     }
  13.     if (gesture.Name == "saluteProgress")
  14.     {
  15.         _saluteProgress = gesture;
  16.     }
  17. }
  18.  
  19. _gestureFrameReader = _gestureFrameSource.OpenReader();
  20. _gestureFrameReader.IsPaused = true;
  21. _gestureFrameReader.FrameArrived += reader_FrameArrived;

 

This snippet shows the initialisation code for importing the gesture database and registering to receive events when new data arrives.Note that I hold references to Gesture objects so that I can identify these when the data arrives.The two gestures I am interested in are the “salute” gesture which is a discrete gesture and “saluteProgress” which is a continuous gesture and will report progress for the salute. (Note. details of what, how and why for those different gesture types are in my previous post). This code follows the familiar pattern followed by the Kinect SDK of creating a frame source and opening a reader on that source which can be used to register for data events.Since we need a tracking id for gesture detection I have paused the reader until we have a valid one (this will be retrieved from the skeleton data, see code snippet below).

 

  1. if (body != null && body.IsTracked)
  2. {
  3.     _gestureFrameSource.TrackingId = body.TrackingId;
  4.     _gestureFrameReader.IsPaused = false;
  5.     break;
  6. }

 

When the gesture data arrives, info about the current status of each gesture can be pulled out if the frame and in my sample I just display these on simple UI elements.

 

  1. void reader_FrameArrived(VisualGestureBuilderFrameReader sender, VisualGestureBuilderFrameArrivedEventArgs args)
  2. {
  3.     using (var frame = args.FrameReference.AcquireFrame())
  4.     {
  5.         if (frame != null && frame.DiscreteGestureResults != null)
  6.         {
  7.             var result = frame.DiscreteGestureResults[_salute];
  8.             
  9.             if (result.Detected == true)
  10.             {
  11.                 var progressResult = frame.ContinuousGestureResults[_saluteProgress];
  12.                 Progress.Value = progressResult.Progress;
  13.             }
  14.             else
  15.             {
  16.                 Progress.Value = 0.0;
  17.             }
  18.  
  19.             GestureText.Text = result.Detected ? "TRUE" : "FALSE";
  20.             ConfidenceText.Text = result.Confidence.ToString();
  21.         }
  22.     }
  23. }

 

Here’s a screenshot of the sample in action:

 

salutesmall

The sample project is located here http://1drv.ms/1C4RQGU

7 thoughts on “Visual Gesture Builder – Kinect 4 Windows v2 (code)

  1. Excuse me,When I calling my own . gbd file builded by VGB, it has an error:

    “System. InvalidOperationException” type of exception happen in Microsoft.Kinect.VisualGestureBuilder.dll , but not handled in the user code

    Other information: This API has returned an exception from the an HRESULT: 0 x80004005

    I don’t know what’s wrong with it.. How to call my own .gbd in the code? Thank you very much!

  2. I build this project and the following hints appeared.

    Error 1 Could not copy the file “E:\DetectVGBGesture\DetectVGBGesture\Gestures\DrumHit.gbd” because it was not found. DetectVGBGesture

Leave a Reply

Your email address will not be published. Required fields are marked *