The model accurately fills out and completes the partial images captured by the camera.
Using a miniature camera and a customized deep neural network, Cornell University researchers have developed a wristband that tracks the entire body posture in 3D. Called BodyTrak, it is the first wearable to track the full body pose with a single camera. If integrated into future smartwatches, BodyTrak could be a game-changer in monitoring user body mechanics in physical activities where precision is critical.
“Since smartwatches already have a camera, technology like BodyTrak could understand the user’s pose and give real-time feedback,” said Cheng Zhang, PhD, assistant professor of information science at Cornell. “That’s handy, affordable, and does not limit the user’s moving area.”
The secret to BodyTrak is not only in the dime-sized camera on the wrist but the customized deep neural network behind it. This deep neural network–a method of artificial intelligence that trains computers to learn from mistakes–reads the camera’s rudimentary images of the user’s body in motion and virtually recreates 14 body poses in 3D and in real time.
“Our research shows that we don’t need our body frames to be fully within camera view for body sensing,” said Hyunchul Lim, a doctoral student in the field of information science “If we are able to capture just a part of our bodies, that is a lot of information to infer to reconstruct the full body.”
