Converting 2D golf swing sequences into 3D models

Wednesday, March 12, 2008

End of Downswing

We haven't been considering the end of the downswing, so are results for that portion are expectedly erratic. But for a few cases, it does an okay job.






Demos

http://www.cs.ucsd.edu/~rchugh/swingimp/

Miscellaneous Video Tools

To create a video from images:

mencoder "mf://*.png" -vf scale=360:240 -mf fps=29 -o out.avi -ovc lavc -lavcopts vcodec=msmpeg4v2:vbitrate=800

To embed a movie directly in a webpage:

http://cit.ucsf.edu/embedmedia/step1.php

Monday, March 10, 2008

Oops

Extreme (part 2)

Over the weekend we re-wrote the early parts of our pipeline (up until Canny, right before Hough) to streamline them and make them a little faster. In the process, we introduced a bug where the masks that were being applied to each frame were offset by one. This led to a degradation in the quality of edge detection and consequently the Hough lines. Now that we've fixed that bug, the Hough lines are pretty much back to where they were before. And as a result, our extreme hypothesis checker shows better results.



Notice that for this video, the results are what we would like to always have: that the topmost point is at the highest point in the swing and that the bottommost point is the shadow of the club head at the bottom of the swing.

Sunday, March 9, 2008

Extreme Hypothesis Checking

In deciding whether or not to guide RANSAC towards fitting extreme points in all directions, we have examined the extreme points obtained on some of our sample videos. Out of the five videos we tried this on, in most cases the four extreme points corresponded to the clubhead. In one case, the topmost point was along the shaft not all the way to the clubhead.





Of course, to really validate that the extreme points are always good, we need to test on many more videos with varying conditions, which our sample videos do not have.

(face5, gw3)

Monday, March 3, 2008

Partial Downswing Fitting

We're experimenting a little with fitting the downswing as two separate parts. We're manually identifying the "crossover" frame, the point at which the club head passes 2*Pi (the club directly to the right of the reference point).

The first part of the downswing is being fit a little better.





The second part of the downswing is too gruesome to show. The curve is completely erratic. One likely source of error is that at the end of the swing, the club head is closer to the reference point than the hand, so we choose the wrong end of the line segment to be the club head. An easy way to solve this problem is to only track up to a certain point, such as Pi/2, where the club is directly over the reference point (and is still farther than the hand).

Saturday, March 1, 2008

Something Like Tomographic Reconstruction

The motion history masks have a lot of good signal that we want to take advantage to supply hints to the RANSAC fitting about where the boundaries of the swing trajectories are. We are using a projection slice approach to try and get a gift wrap fit around the border of the swing.

We first sum up the the number of white pixels in each column. Then we scan from left to right to compute two things:
  • The index of the leftmost column with at least THRESHOLD% pixels white.
  • The index of the rightmost column with at least THRESHOLD% pixels white.
These indices give us two lines representing the boundary of the object projected onto the x-axis.

We rotate the image and repeat this process, giving us similar projections along multiple directions. The polygon enclosed by these boundaries is an approximation of the boundary of the object.

The following are the results of running this process on the binary upswing motion history image from a previous post. The thresholds for the images below are 3%, 6%, and 9%, and we evaluate projections in 16 directions.







The following is the result for the corresponding upswing with a threshold of 5%.



Notice that the intensity of the mask is duller than the original. This is probably an artifact from the way we are manually rotating images, but it doesn't seem to affect the results.

Also note we have not yet computed the enclosing polygon; we are simply drawing the boundary lines from each projection.