|
|
 If you have questions about our technology or would like to join our mailing list click here. |
FACEngine ID Matching Accuracy
The ID validation testing has been organized into three categories: Controlled, Uncontrolled (lighting and expression) and Pose. There are also tests demonstrating the power of using multiple images for a target enrollment. To access the test results, please click on the following links:
|
|||||||||
| Figure 1: Sample FRGC Controlled (Left), and FRGC uncontrolled (Right) data used for Identification Testing. Uncontrolled samples show grouping of multigallery-multiprobe tests. | |||||||||
Accuracy Tests
Controlled Testing
The Controlled testing utilized the FRGC Conotrolled Dataset which includes high resolution, carefully collected data which, as expected, produced exceptional results. For the Controlled FRGC data, the identification system exhibited outstanding performance when using a single representative for each class (for a discussion of the effect of multiple gallery elements, please see the section The Advantage of Multiple Enrollment Images, below).
Table 1: FRGC Controlled Accuracy
Probe Elements |
Gallery |
Gallery Elements |
FRR @ FAR 1.0% |
FRR @ FAR 0.1% |
|
Controlled |
1 |
Controlled |
1 |
0% |
0.2% |
FRGC 1-1 Controlled Identification Results
|
|
||
 |
||
 |
||
 |
|
 |
2: FAR/FRR Graph showing the accuracy of FACEngine ID against the FRGC Controlled Dataset
Uncontrolled Testing - Lighting, Scale and Background
FRGC Uncontrolled Accuracy
The Uncontrolled FRGC Data is comprised of high resolution images which exhibit a wide range of environmental and scale variation. Although the pose was controlled in the sense of being frontal imagery, complex backgrounds, inconsistent lighting, strong shadows and scale variation provide for a challenging test for facial recognition systems (for a discussion of the effect of multiple gallery elements, please see the section The Advantage of Multiple Enrollment Images, below).
Table 2: FRGC Uncontrolled Accuracy
Probe Elements |
Gallery |
Gallery Elements |
FRR @ FAR 1% |
FRR @ FAR 0.1% |
|
Uncontrol |
1 |
Uncontrol |
1 |
11.5% |
29.8% |
FRGC 1-1 Uncontrolled Identification Results
|
|
||
|
|
||
|
|
||
|
|
|
|
Figure 3: FAR/FRR Graph showing accuracy of FACEngine ID against the FRGC Uncontrolled Dataset
FERET Uncontrolled Accuracy
The FERET Uncontrolled data consists of good quality low resolution (256x384px) images. The FERET Uncontrolled accuracy tests included controlled enrollment images, and gallery images with intentional variability of (a) Lighting, and (b) extreme expressions such as smiles and sneers. The FERET tests were performed to measure FACEngine ID's robustness to specific conditions in probe data, using a controlled gallery enrollment. All of the FERET tests utilized one member in each of the gallery and probe pairs.
As the results in Figure 4 and 5indicate, Animetrics lighting and expression normalization provides excellent accuracy in these lower resolution images.
Table 3: FERET Uncontrolled
Probe Elements |
Gallery |
Gallery Elements |
FRR @ FAR 1.0% |
FRR @ FAR 0.1% |
|
FERET |
1 |
Lighting |
1 |
2% |
4% |
FERET |
1 | Expressions |
1 |
2% |
2% |
FERET Lighting Identification Results
|
|
||
|
|
||
|
|
||
|
|
|
|
Figure 4: FAR/FRR Graph showing accuracy of FACEngine ID against the FERET Lighting Dataset
FERET Expressions Identification Results
|
|
||
|
|
||
|
|
||
|
|
|
|
Figure 5: FAR/FRR Graph showing accuracy of FACEngine ID against the FERET Expressions Dataset
Pose Testing
Pose testing analyzes the accuracy of Animetrics' pose normalization designed to maintain accuracy as faces rotate out of a frontal view in the image. The pose tests used FERET imagery. As with the FERET uncontrolled tests, we utilized a single controlled enrollment image, and single image probes which bore a yaw rotation of either 15, 25, or 45 degrees from front.
Table 4: FERET Pose Accuracy
Probe Elements |
Gallery |
Gallery Elements | FRR @ FAR 1.0% |
FRR @ FAR 0.1% |
|
FERET |
1 | 15 Deg. |
1 |
1% |
1.5% |
| FERET | 1 | 25 Deg. | 1 | 0.5% |
0.5% |
| FERET | 1 | 40 Deg. | 1 | 2% |
6.5% |
FERET 15 Degree Identification Results
|
|
||
|
|
||
|
|
||
|
|
|
|
Figure 6: FAR/FRR Graph showing accuracy of FACEngine ID against the FERET 15° Dataset
FERET 25 Degree Identification Results
|
|
||
|
|
||
|
|
||
|
|
|
|
Figure 7: FAR/FRR Graph showing accuracy of FACEngine ID against the FERET 25° Dataset
FERET 40 Degree Identification Results
|
|
||
|
|
|
|
|
|
|
|
Figure 8: FAR/FRR Graph showing accuracy of FACEngine ID against the FERET 40° Dataset
The Advantage of Multiple Enrollment Images
The use of multiple enrollment images provides a broader representation of the target. The tests using the FRGC controlled and uncontrolled images were performed using 1, 2, and 3 images for their enrollment. As can clearly be seen in the accuracy graphs below, the use of multiple representations can make reliable identification possible with very complex and difficult probe images.
Table 5: FRGC Controlled Accuracy
Probe Elements |
Gallery |
Gallery Elements |
FRR @ FAR 1.0% |
FRR @ FAR 0.1% |
|
Controlled |
1 |
Controlled |
1 |
0% |
0.2% |
Controlled |
1 |
Controlled |
2 |
0% |
0% |
Controlled |
1 |
Controlled |
3 |
0% |
0% |
Table 6: FRGC Uncontrolled Accuracy
Probe Elements |
Gallery |
Gallery Elements |
FRR @ FAR 1% |
FRR @ FAR 0.1% |
|
Uncontrol |
1 |
Uncontrol |
1 |
11.5% |
29.8% |
Uncontrol |
1 |
Uncontrol |
2 |
2.8% |
6.6% |
Uncontrol |
1 |
Uncontrol |
3 |
0.8% |
2.6% |
FRGC Multiple Member Gallery Controlled Identification Results
|
|
||
|
|
||
|
|
||
|
|
|
|
FIgure 9: FAR/FRR Graph showing the accuracy of FACEngine ID against the FRGC Controlled Dataset with 3 member galleries (left) and 2 member galleries (right).
FRGC Controlled 1, 2 and 3 Member Gallery Accuracy
|
|
||
|
|
||
|
|
||
|
|
|
|
Figure 10: FAR/FRR Summary Graph showing FACEngine ID accuracy against 1 member (red), 2 member (green) and 3 member (blue) galleries using FRGC controlled data.
FRGC Multiple Member Gallery Uncontrolled Identification Results
|
|
||
|
|
||
|
|
||
|
|
|
|
Figure 11: FAR/FRR Graph showing the accuracy of FACEngine ID against the FRGC Controlled Dataset with 3 member galleries (left) and 2 member galleries (right).
FRGC Uncontrolled 1, 2 and 3 Member Gallery Accuracy
|
|
||
|
|
||
|
|
||
|
|
|
|
Figure 12: FAR/FRR Summary Graph showing FACEngine ID accuracy against 1 member (red), 2 member (green) and 3 member (blue) galleries using FRGC uncontrolled data.