wrnch CaptureStream

With wrnch CaptureStream, you can use the camera on your iOS device to capture full body human motion and real-time stream 3D motion data to host applications. For developers, you will want to take advantage of the local network streaming capabilities of CaptureStream, making it easy to ingest 3D motion capture data into your application. All you need to do is write IP networking code – no messy SDK or machine learning integrations involved!

For developers who need this 3D motion data right in their own iOS app, CaptureStream is designed to serve as a reference application for developing on iOS with the wrnch AI Engine. You can create 3D human pose-enabled applications by using wrnch’s groundbreaking real-time wrnch AI Engine to get accurate 3D poses and then create your own magic by building on ours.

HOW IT WORKS

CaptureStream performs markerless full body motion capture to record human movements from an iOS device without requiring the person to wear tracking sensors. Powerful AI algorithms are run against the live camera feed to identify skeletal joints, poses and movement. Real-time visualizations of the person in motion are displayed on the iOS device’s screen with an overlay of a 3D skeleton and avatar.

From the wrnch Developer Portal, developers can download wrnch ReceiveVisualize, an Unreal Engine sample application. Human pose metadata is streamed in real-time from the iOS device to the PC over a local network, allowing developers to visualize 3D character motion in real-time.

Both CaptureStream and ReceiveVisualize are available to download for free.

ReceiveVisualize is available as sample code so developers can start prototyping and create a complete vision system proof of concept.

USING THE APP

Using the app is easy. You can just start the app up, point it to someone dancing or performing an emote, and you will see your subject’s movements being mimicked by wrnchly, our avatar.

In order to best reproduce movement in true 3D, you can use Pro Mode, which will track you as you move around during your performance.

To enable Pro Mode, you toggle it on in the home screen and then perform a couple of initialization steps to maximize motion capture accuracy.

In Pro Mode, wrnchly will mimic your movements in virtual space in the ReceiveVisualize app as you perform them in front of CaptureStream. You can walk back and forth, side-to-side, in a circle, and wrnchly will imitate you in its virtual environment!