This program implements feature tracking using a normalized correlation coefficient. Once it fnds a feature, it can determine the direction of motion
Method and Implementation
Feature Tracking
Feature tracking is implemented by matching a template to the original image.
The algorithm fisrt picks a template. It either uses the stored template (of an eyebrow), or if the user clicks on the screen, it picks out a 15×15 px template of the pixels around where the user clicked.
The algorithm then runs through the image in increments of 4 if no template has been matched and in increments of 1 if a template was found
The area of interest starts as the whole image, if a template is found, or the user clicks, then the area of interest becomes an area around the matched template that is a multiple of the template size
Matching the template is achieved by incrementing a counter every time a pixel value is matched within 10% of the color value on all three channels of the image. Then a percentage of matching pixels is calculated, if this percentage clears a specified threshold (.75), then the algorithm assumes the template is matched
Motion
Gestures were recognized using a histogram of points where the template was matched.
If the standard deviation on the x or the y (or both) changed by more than a specified threshold, then the algorithm assigned a direction of motion
Experiment and Results
Experiment 1
Here is a picture of the algorithm running
The algorithm was a lot more accurate than I thought it would be. It is slow if it loses the template, or in any case that it has to scan the whole image.
Motion detection is also fairly accurate, but it is very dependent on the accuracy of the template mathing algorithm
This algorithm works well on all kinds of people and lighting conditions (this is because of the ability to pick a template on the fly), I tested it on over *100 people! and it worked just fine
* figures with an asterisk might have been slightly overexagerated
Discussion and Conclusion
I implemented many features to make the code run faster. I skipped a few rows and columns on every pass through the image, I also limited the search region once a template was matched, and kept only the last 4 points for motion analysis.
The algorithm can be made faster by implemeting it with image pyramids (which I was going to do, but array boundaries were driving me crazy), and also by converting the images to grayscale before running the algorithm
Motion tracking could be made better by keeping more points and fitting a line to the array of points. Then the slope of the line will give you a better estimate of the direction of motion.
* How well were you able to find the desired feature on your face?
Very well, a little slow, but with a few optimizations can get a lot faster.
* As you ran your tracker and moved around did the tracking break down, or did it stay on the target?
It mostly depended on how fast I moved. Basically, if I moved more than the region I was looking at in one frame, then the tracker had to scan the whole image again. If the movement was not too sudden, then the tracker mostly stayed on target.
* What are the limitations of your template tracker?
Mainly speed, there are alot of optimizations to be made.
* How well did your tracker perform when you used the template from your face on another person?
When I used a template from my face on my roomate (he’s skin is very different color from mine), my tracker would match the wall, or anything elso but him. But once we picked a template from his face, it did just fine. This has more to do with trying to match a template that does not exist in him.
Credits and Bibliography
Only the open cv documentation pages were used in this project
Feature Tracking
This program implements feature tracking using a normalized correlation coefficient. Once it fnds a feature, it can determine the direction of motion
Feature Tracking
Feature tracking is implemented by matching a template to the original image.
The algorithm fisrt picks a template. It either uses the stored template (of an eyebrow), or if the user clicks on the screen, it picks out a 15×15 px template of the pixels around where the user clicked.
The algorithm then runs through the image in increments of 4 if no template has been matched and in increments of 1 if a template was found
The area of interest starts as the whole image, if a template is found, or the user clicks, then the area of interest becomes an area around the matched template that is a multiple of the template size
Matching the template is achieved by incrementing a counter every time a pixel value is matched within 10% of the color value on all three channels of the image. Then a percentage of matching pixels is calculated, if this percentage clears a specified threshold (.75), then the algorithm assumes the template is matched
Motion
Gestures were recognized using a histogram of points where the template was matched.
If the standard deviation on the x or the y (or both) changed by more than a specified threshold, then the algorithm assigned a direction of motion
Experiment 1
The algorithm was a lot more accurate than I thought it would be. It is slow if it loses the template, or in any case that it has to scan the whole image.
Motion detection is also fairly accurate, but it is very dependent on the accuracy of the template mathing algorithm
This algorithm works well on all kinds of people and lighting conditions (this is because of the ability to pick a template on the fly), I tested it on over *100 people! and it worked just fine
* figures with an asterisk might have been slightly overexagerated
I implemented many features to make the code run faster. I skipped a few rows and columns on every pass through the image, I also limited the search region once a template was matched, and kept only the last 4 points for motion analysis.
The algorithm can be made faster by implemeting it with image pyramids (which I was going to do, but array boundaries were driving me crazy), and also by converting the images to grayscale before running the algorithm
Motion tracking could be made better by keeping more points and fitting a line to the array of points. Then the slope of the line will give you a better estimate of the direction of motion.
* How well were you able to find the desired feature on your face?
Very well, a little slow, but with a few optimizations can get a lot faster.
* As you ran your tracker and moved around did the tracking break down, or did it stay on the target?
It mostly depended on how fast I moved. Basically, if I moved more than the region I was looking at in one frame, then the tracker had to scan the whole image again. If the movement was not too sudden, then the tracker mostly stayed on target.
* What are the limitations of your template tracker?
Mainly speed, there are alot of optimizations to be made.
* How well did your tracker perform when you used the template from your face on another person?
When I used a template from my face on my roomate (he’s skin is very different color from mine), my tracker would match the wall, or anything elso but him. But once we picked a template from his face, it did just fine. This has more to do with trying to match a template that does not exist in him.
Only the open cv documentation pages were used in this project
Source Code