In an attempt to fit more points along the trajectory, we tried a simple weighting scheme to favor points with larger magnitudes (distance from the origin). We define a set of concentric rings around the origin with radii of multiples of 50 pixels. Then, for a point in ring with radius n*50, we use n*THRESHOLD as the threshold for determining whether or not a point agrees with a model generated by RANSAC (where THRESHOLD is the threshold we were using for all points before).
The results using this scheme:
The model fit for the upswing is actually worse than it was before, but the downswing curve follows the actual trajectory better for longer.
In general, this doesn't seem like a promising approach. One related possibility is to take the extreme points (the left-most, bottom-most, etc) and force RANSAC to fit at least those points. But this does not sound like a robust approach either.
The following images correspond to the first video from the previous two posts (the one in which the golfer is wearing red).
In a previous phase of the pipeline, we've isolated the moving pixels in each frame as a binary mask. To generate these images, we iterate over the binary mask for each frame and count the number of white pixels at each location (i,j).
The first row contains binary motion histories for the up- and downswing, where a white pixel means at least one frame mask had that particular pixel set. For the images in the second row, we've normalized the intensity of white relative to the maximum number of votes any one pixel had.
The motion history images for the second video (not shown here) are similar in quality, with slightly less background noise.
The trajectory estimation computes magnitude as a function of angle. We have also now computed angle as a function of time (frames * seconds/frame). Following the Gehrig paper, we use a 3rd degree polar polynomial for the upswing and a 5th degree one for the downswing. The angles used to compute f(t) are those of the inliers from the best trajectory fitting.
These results are a decent start, especially for the upswing.
We are still trying to improve the results of the downswing curve fitting. A couple of fundamental things that still need work:
More accurate identification of transition
More reliably offsetting angles in the downswing by 2*Pi where appropriate
A more experimental possibility we may try stems from the following observation: none of the bad clubhead hypotheses lay outside of the actual trajectory. In fact, there are sometimes many good hypotheses on the actual trajectory that are not in the consensus set of the final model. For example:
Two things that might help the fitting process:
Weight the fitting towards points with larger magnitudes.
Weight the fitting towards points later in the swing, because these two images show that most inliers are early in the downswing.
We've worked on club detection to make it work for a better sample video of a full golf swing (in slow motion). We adjusted the threshold for the LO Hough transform, so that it still gets good signal around the club but not as much erroneous signal. We also realized that for the HI lines, we were getting a lot of good signal but we were pruning too much of it out. We've made the filtering process for HI a little better, but there are still ways to make it better to retain even more HI signal (e.g, working harder to merge parallel and disconnected segments).
We've also estimated the top of the backswing using a simple metric. For each frame, we consider the best single hypothesis for the club position. We then look for places where the 2nd derivative between 2 frames is 0, and choose these frames as possiblilities for the transition from upswing to backswing. We then use a weighting scale to assign each of these possibilities a score, based on distance from the middle of the sequence and the number of surrounding LO hypotheses. We then choose the frame with the lowest score.
The changes to Hough thresholding and detection of the transition work pretty well on the following video.
Upswing:
Downswing:
The hypotheses for the clubhead (marked by white circles) are fairly accurate, so we should be able to get a good polynomial approximation using least squares and RANSAC.
We also need to develop a library of test videos, with different conditions, so that we can be confident that the refinements we are making are robust.
There are 100 points around the line y=x, randomly perturbed a small amount. There are also 50 points scattered uniformly at random. The yellow line represents the line of best fit using least squares, and the blue line represents the line of best fit among consensuses of at least 70 points. The red points are those in consensus about the best model estimated by RANSAC.
