c++, Kinect, Rx, WinRT

Face Tracking Native – Kinect 4 Windows v2

Well, with the managed version working in c# (see Face Tracking Kinect 4 Windows V2 – Managed) it should be a breeze to convert the code to a native c++ windows store xaml app :).

Converting an event-based c#/xaml app seemed like a gentle introduction to a more native environment which I hope to get into later. Also, I was asked recently whether face tracking worked in a native app.

There are a few things I’ve become accustomed to in the managed development environment; nuget for one thing and also I’ve been using Reactive extensions in my Kinect projects. A quick Bing-search later and I found this:

rx-nugetcppSo, typing Install-Package rxcpp into the Package Manager Console window caused a short flurry of output which I assume meant the packages were downloaded and referenced within my project. After some short experimentation I found that v2.0.0 doesn’t yet have bindings for WinRT so I rolled back to v1.0.2 with a ‘note-to-self’ to explore this more fully later. Including the headers rx.hpp and rx-winrt.hpp in the projects precompiled header and adding a using namespace rxcpp; enabled me to get started writing the code.

It took me a while to work out the correct syntax for configuring the Kinect event handlers using Rx. I ended up with this:

  1. _multiFrameReader = _kinect->OpenMultiSourceFrameReader(FrameSourceTypes::Body | FrameSourceTypes::Color);
  2. auto multiFrameObservable = from(rxrt::FromEventPattern<MultiFrameArrivedEventHandler>(
  3.     [this](MultiFrameArrivedEventHandler^ h)
  4.     {
  5.         return this->_multiFrameReader->MultiSourceFrameArrived += h;
  6.     },
  7.     [this](Windows::Foundation::EventRegistrationToken t)
  8.     {
  9.         this->_multiFrameReader->MultiSourceFrameArrived -= t;
  10.     }))
  11.     .observe_on(std::make_shared<rxcpp::NewThreadScheduler>());
  12.  
  13. _disposables.Add(from(multiFrameObservable)
  14. .subscribe([this](const rxrt::EventPattern<MultiSourceFrameReader^, MultiSourceFrameArrivedEventArgs^>& p){
  15.     this->OnMultiFrameArrived(p);
  16. }));

In the Kinect event handlers when I tried to call Close() on the Frames I got a compile error error C2039: ‘Close’ : is not a member of ‘WindowsPreview::Kinect::MultiSourceFrame’ so I used scoping to release the Frames instead.

  1. {
  2.     auto multiFrame = p.EventArgs()->FrameReference->AcquireFrame();
  3.     if (multiFrame != nullptr)
  4.     {
  5.     }
  6. }

Another difference I introduced to the native version was that I couldn’t find a way to schedule rx to deliver the Kinect events via the Thread Pool so I subscribed to the multi frame event from a Thread Pool thread and used the CurrentThreadScheduler. Aside from these minor differences the rest of the code looked pretty much the same. You can find the sample project here http://1drv.ms/1th57Lb.

2 thoughts on “Face Tracking Native – Kinect 4 Windows v2

Leave a Reply

Your email address will not be published. Required fields are marked *