Trial ::: haarcascade18 :::

Training Options Used –

-npos 1898
-nneg 625
-nstages 20
-mode ALL

The number of positive images used to train this classifier was 1898, the number of negative images used was 625 and specified that the number of stages the trainer should try to complete is 20. Used the mode ALL switch, which not only includes the basic haar like features but an extended set of features, here is a link to the feature sets Haar-like features. During the training phase of this cascade it finished in stage 20, which created 139 total weak classifiers. In this stage it reached the desired minimum false alarm rate, or else the false alarm rate will be too high for real world use. This positive set had a few more images, most of the additions were from images that contained a cavity in Quadrant 3, and the image was rotated and saved as new image so that its cavity now resides in Quadrant 1.

NEG: 148 9.77509e-005

Used a modified version of the OpenCV haartrainer.

-data "F:\Thesis Research\AutomatedCMEDetector\HaarClassifier\haarcascade18" -vec "F:\Thesis Research\AutomatedCMEDetector\Test Sets\Positive Solar Cavities\positivesoutput.vec" -bg "F:\Thesis Research\AutomatedCMEDetector\Test Sets\Negative Solar Cavities\negatives.txt" -npos 1134 -nneg 625 -nstages 20 -mode ALL


Hit rate ≈ 56.7%

Tags: testing outcomes, trials

Training Options Used –

-npos 448
-nneg 587
-nonsym
-mode ALL

The number of positive images used to train this classifier was 448, the number of negative images used was 587 and did not specify the number of stages the trainer should try to complete, so by default it tries to achieve 14 stages. The “-nonsym” option is used when the object class does not have vertical (left-right) symmetry. The “-mode ALL” uses the extended sets of Haar-like features, the default is BASIC and it uses only upright features, while ALL flag uses the full set of upright and 45 degree rotated feature set. During the training phase of this cascade it finished in stage 14, which created 28 total weak classifiers.

NEG: 27 0.00258225

Hit rate ≈ 45.1%

Tags: testing outcomes, trials

Training Options Used –

-npos 448
-nneg 587
-nstages 20

The number of positive images used to train this classifier was 44, the number of negative images used was 587and specified that the number of stages the trainer should try to complete is 20.
With this cascade the trainer was able to reach its set goal of 20 stages, which created 51 total weak classifiers.

NEG: 27 0.000192682

Hit rate ≈ 29.8%

Tags: testing outcomes, trials

Training Options Used –

-npos 89
-nneg 479
-nstages 20

The number of positive images used to train this classifier was 89, the number of negative images used was 479 and specified that the number of stages the trainer should try to complete is 20.
During the training phase of this cascade it finished in stage 18, which created 98 total weak classifiers. In this stage it reached the desired minimum false alarm rate, or else the false alarm rate will be too high for real world use.

NEG: 365 2.52601e-006

Hit rate ≈ 33.1%

Tags: testing outcomes, trials

Training Options Used –

-npos 1134
-nneg 625
-nstages 20

The number of positive images used to train this classifier was 1134, the number of negative images used was 625 and specified that the number of stages the trainer should try to complete is 20.
With this cascade the trainer was able to reach its set goal of 20 stages, which created 133 total weak classifiers.

NEG: 146 0.000125991

Hit rate ≈ 57.0%

Tags: testing outcomes, trials

Training Options Used –

-npos 89
-nneg 479
-nonsym
-mode ALL

The number of positive images used to train this classifier was 89, the number of negative images used was 479 and did not specify the number of stages the trainer should try to complete, so by default it tries to achieve 14 stages. The “-nonsym” option is used when the object class does not have vertical (left-right) symmetry. The “-mode ALL” uses the extended sets of Haar-like features, the default is BASIC and it uses only upright features, while ALL flag uses the full set of upright and 45 degree rotated feature set. During the training phase of this cascade it finished in stage 13, which created 47 total weak classifiers. In this stage it reached the desired minimum false alarm rate, or else the false alarm rate will be too high for real world use.

NEG: 365 4.67794e-005

Hit rate ≈ 32.0%

Tags: testing outcomes, trials

During testing and attempting to find the perfect (well…most nearly perfect) classifier, below are the trial runs and their statistical summaries. The Trial numbers are clickable links to a more verbose detailed section of how the run was constructed and graphs of the data from the outputs.

 
The table represents the data When running each classifier against a defined validation set which contains 1134 positive images and in those images there are a total of 1770 solar cavities that exist.

Hits – The total number of correct identifications of solar cavities
Misses – The total number of missed solar cavities
Total – The total number of solar cavities in validation set
False Alarms – The total number of false detections made
Hits % – Hit percentage of the classifier [Hits/Total]

Below is a chart that shows the following specific data points for Hits, Misses, Number of Stages and Weak Classifiers for all of the current trial runs. This table is to aid in finding any trends that would help create better classifiers, by easily seeing how the data fits together.
 

 

The next chart shows the following specific data points for Hits, Misses, Number of Stages, Number of Weak Classifiers and False Alarms for all of the current trial runs, with graphical emphasis on the Hits versus False Alarms.
 

 

Figure showing the number of positive images used in training the classifier and how that corresponds to the number of Hits that classifier detected. It shows that typically with the more images you train your classifier against you will generally have a better Hits count. Although, there should be minimum number of the most distinct positives that you can train your classifier with, with maybe some distortions on the samples for variance so that marking thousands of images no longer needs to happen for a good classifier.
 

 

The follow graph shows the mapping of data for the hit count numbers based on the number of stages that classifier completed. Based on the graph of results I think its pretty fair to say that the number of stages does not directly correlate to a higher number of hits. I believe that there exists a boundary line that no matter if you set your stage count to something extremely high it will not necessarily increase your hit count as well as this is proved from the opencv_haartraining.exe where if the value of the false alarm rate becomes so low the program will exit.
Required leaf false alarm rate achieved. Branch training terminated.
 

Tags: testing outcomes, trials