|
|
 If you have questions about our technology or would like to join our mailing list click here. |
FACEngine ID Detector Accuracy
2D IMage Feature Analysis Accuracy Testing
The 2D Image Feature Analysis (2DIFA) module is a fundamental component of Animetrics' facial biometric system. The results of the analysis of the image plane drive the performance of all subsequent operations. If the image analysis is substandard, even a "perfect" matching algorithm will fail. Accurate and robust image analysis is the cornerstone of all facial biometric systems. Competing trackers obtain computational efficieny by examining a small number of features. Animetrics 2DIFA uses rapid tracking of the dense continuum of points in the projective geometry to obtain accurate results across a broader range of conditions and is robust to a greater variety of poses.
Animetrics evaluates 2DIFA accuracy by obtaining ground truth values of feature coordinates through the meticulous hand-landmarking of images, and the comparison of those values to the automatically generated values produced by the 2DIFA module. To calculate average deviation results, all images were manually hand featured for comparison. The mean-squared error deviation of each feature was calculated across the dataset. Shown in Figure 1 below are the standard deviation error ellipses in units of pixels shown to scale on the exploded head depicting the RMSE of the detected feature points. The RMSE values are computed at a relative scale of 64 pixels between the eyes.
|
|
||
|
|
 |
|
 |
||
 |
|
 |
Figure 1: Standard deviation error ellipses shown to scale on the exploded head depicting the RMSE of the detected feature. For each image of each dataset, the 2DIFA module was used to automatically locate feature points. Shown via the red ellipses are the RMSEs of the detected features in the image plane and compared to the manual features.
Controlled FRGC Dataset
The Controlled FRGC Data is high resolution, carefully collected data. Due to the high quality of the imagery and rigor of the collection process, tests against this data produced exceptional results. For the Controlled FRGC data, there were zero failures to detect the face. Further, with the exception of three (3) images out of 608, all images produced eye positions within 8 pixels of the ground truth metadata.
Table 1: Controlled FRGC Detailed Results
Dataset |
Failure |
Partially Degraded |
Success |
Success % |
Nonfailure % |
RMSE-Nonfailure |
Controlled (608) |
0 |
3 |
605 |
99.5% |
100% |
2.448 |
|
|
||
|
|
|
|
|
|
||
|
|
|
|
Figure 2: Sampling of images from FRGC controlled data. Red dots represent detected eye locations, while blue dots represent baseline coordinates.
Uncontrolled FRGC Dataset
The uncontrolled FRGC data is comprised of high resolution images taken with intentional variation in scale and in environments with a high degree of clutter and lighting variation. The uncontrolled FRGC data produced 16 failures to detect the face in the image out of 508 total images. This is mitigated by the fact that 13 of 16 failures were on heads which were within +/-10% of the designed scale range of the Coarse Pose Estimator.
The Animetrics coarse pose detector is designed with a 1:8 scale variability See Figure 3, below. It is currently designed to accommodate facial imagery where the distance between the eyes ranges from 50% down to 6.25% of the image width. This can be adjusted and it is expected that the lower scale range can be improved by adjusting the scale range to a lower percentage.
|
|
||
|
|
|
|
|
|
||
|
|
|
|
Figure 3: Example of scale variation in uncontrolled FRGC data. Two images of the same subject in the same hallway from the FRGC Uncontrolled data. The image on the right produces a successful detection whereas the image on the left produces a failure. Based on the image width of 2272 pixels, the left image is well below the 6.25% scale of 142 pixels.
Table 2: Uncontrolled FRGC Results
Dataset |
Failure |
Partially Degraded |
Success |
Success % |
Nonfailure % |
RMSE-Nonfailure |
Uncontrolled |
16 |
1 |
491 |
96.65% |
96.85% |
2.708 |
|
|
||
|
|
|
|
|
|
||
|
|
|
|
Figure 4: Sampling of images from the FRGC uncontrolled data. Red dots represent detected eye locations, while blue dots represent baseline coordinates.