Project Overview

Our goal is to build a system that allows a golfer to create a customized 3D model of his/her golf swing from the 2D video input of the golf swing itself. Having a motion capture session to record a golf swing is the most accurate way to obtain a 3D model of this motion, but few golfers have access to such expensive and uncommon mocap studios. To help a golfer realize some of the benefits of viewing his/her own swing as a 3D model, our system will import the swing video (hence the name) and try to reconstruct from the 2d images what the 3d motion looks like.

Because we are limiting the scope of this project to the motion of a golf swing, we will exploit the fact that any golf swing exhibits similarities. After we define the notion of a generic model, we will explore algorithms that identify key postures in the given 2D images and track these movements over time. Our intent is that restricting ourselves to the golf swing motion will avoid some of the difficulties of detecting arbitrary human body movement.

Although our long-term goal is to eventually extend the system for users to use their point-and-shoot cameras to produce their golf swing videos, our initial approach will contain a more ideal environment for video capture (in order to accomplish as much as we can). Because of the speed of a golf swing cannot be best represented with the typical point-and-shoot, we are planning to use cameras with frame rates on the order of 100 to 200 fps, and we will record in a room with a neutral, solid background color and neutral lighting.

We have not yet made the choice of processing monocular videos or videos corresponding to multiple angles of the same swing. Current work in vision-based motion capture has been done for both cases, and we need to further investigate this work before deciding. In the event that we choose to use multiple vantage points, we will then need to decide whether we will set up multiple cameras to record synchronously or if we will use alignment techniques to manually synchronize the cameras' videos.

Once the user has uploaded his video and the tracking algorithms have been run to identify the positions and angles of the golfer's skeleton, we will provide the user with a GUI that allows him/her to manually correct the skeleton. This marker phase will allow the user to overcome any shortcomings the tracking algorithms exhibit, and still allow a useful 3D model to be generated.

The resulting skeleton from the tracking/user-calibration phase will then be mapped on to a base 3D golf swing model, which we will either obtain through our own mocap recording sessions or by obtaining sample data from software companies that specialize in mocap systems. One such example can be found at http://www.tmplabs.com/.

Finally, we will design a metric for evaluating the performance of our system. One possibility is to compare the difference in specific joint angles from the 2D images with those from the 3D model over time. Another possibility is to compare the result of a 3D model produced by SwingImp with a model obtained of the same swing through a mocap session. This would present the complication of having to record the 2D videos and the mocap data simultaneously, however.

As you can see, we have a busy quarter ahead of us!