The ID validation testing has been organized into three categories: Controlled, Uncontrolled (lighting and expression) and Pose. There are also tests demonstrating the power of using multiple images for a target enrollment. To access the test results, please click on the following links:

• Controlled
• Uncontrolled
• Pose
• Multiple Enrollment

 

Figure 1: Sample FRGC Controlled (Left), and FRGC uncontrolled (Right) data used for Identification Testing. Uncontrolled samples show grouping of multigallery-multiprobe tests.

Accuracy Tests

Controlled Testing
The Controlled testing utilized the FRGC Controlled Dataset which includes high resolution, carefully collected data which, as expected, produced exceptional results. For the Controlled FRGC data, the identification system exhibited outstanding performance when using a single representative for each class (for a discussion of the effect of multiple gallery elements, please see the section The Advantage of Multiple Enrollment Images, below).

Table 1: FRGC Controlled Accuracy

>Probe>	Probe Elements	Gallery	Gallery Elements	FRR @ FAR 1.0%	FRR @ FAR 0.1%
Controlled	1	Controlled	1	0.5%	1.3%

FRGC 1-1 Controlled Identification Results

2: FAR/FRR Graph showing the accuracy of FACEngine ID against the FRGC Controlled Dataset

Uncontrolled Testing - Lighting, Scale and Background
 
FRGC Uncontrolled Accuracy
The Uncontrolled FRGC Data is comprised of high resolution images which exhibit a wide range of environmental and scale variation. Although the pose was controlled in the sense of being frontal imagery, complex backgrounds, inconsistent lighting, strong shadows and scale variation provide for a challenging test for facial recognition systems (for a discussion of the effect of multiple gallery elements, please see the section The Advantage of Multiple Enrollment Images, below).

  Table 2: FRGC Uncontrolled Accuracy

Probe	Probe Elements	Gallery	Gallery Elements	FRR @ FAR 1%	FRR @ FAR 0.1%
Uncontrol	1	Uncontrol	1	5.3%	12.8%

Figure 3: FAR/FRR Graph showing accuracy of FACEngine ID against the FRGC Uncontrolled Dataset

FERET Uncontrolled Accuracy
The FERET Uncontrolled data consists of good quality low resolution (256x384px) images. The FERET Uncontrolled accuracy tests included controlled enrollment images, and gallery images with intentional variability of (a) Lighting, and (b) extreme expressions such as smiles and sneers. The FERET tests were performed to measure FACEngine ID's robustness to specific conditions in probe data, using a controlled gallery enrollment. All of the FERET tests utilized one member in each of the gallery and probe pairs.
 
As the results in Figure 4 and 5indicate, Animetrics lighting and expression normalization provides excellent accuracy in these lower resolution images.

Table 3: FERET Uncontrolled

Probe	Probe Elements	Gallery	Gallery Elements	FRR @ FAR 1.0%	FRR @ FAR 0.1%
FERET	1	Lighting	1	1%	6%
FERET	1	Expressions	1	1%	3%

FERET Lighting Identification Results

Figure 4: FAR/FRR Graph showing accuracy of FACEngine ID against the FERET Lighting Dataset

FERET Expressions Identification Results

Figure 5: FAR/FRR Graph showing accuracy of FACEngine ID against the FERET Expressions Dataset

Pose Testing
Pose testing analyzes the accuracy of Animetrics' pose normalization designed to maintain accuracy as faces rotate out of a frontal view in the image. The pose tests used FERET imagery. As with the FERET uncontrolled tests, we utilized a single controlled enrollment image, and single image probes which bore a yaw rotation of either 15, 25, or 45 degrees from front.

Table 4: FERET Pose Accuracy

Probe	Probe Elements	Gallery	Gallery Elements	FRR @ FAR 1.0%	FRR @ FAR 0.1%
FERET	1	15 Deg.	1	1%	1%
FERET	1	25 Deg.	1	0%	1%
FERET	1	40 Deg.	1	2%	5.5%

 
FERET 15 Degree Identification Results

Figure 6: FAR/FRR Graph showing accuracy of FACEngine ID against the FERET 15° Dataset

FERET 25 Degree Identification Results

Figure 7: FAR/FRR Graph showing accuracy of FACEngine ID against the FERET 25° Dataset

FERET 40 Degree Identification Results

Figure 8: FAR/FRR Graph showing accuracy of FACEngine ID against the FERET 40° Dataset

The Advantage of Multiple Enrollment Images
The use of multiple enrollment images provides a broader representation of the target. The tests using the FRGC controlled and uncontrolled images were performed using 1, 2, and 3 images for their enrollment. As can clearly be seen in the accuracy graphs below, the use of multiple representations can make reliable identification possible with very complex and difficult probe images.
 
Table 5: FRGC Controlled Accuracy

Probe	Probe Elements	Gallery	Gallery Elements	FRR @ FAR 1.0%	FRR @ FAR 0.1%
Controlled	1	Controlled	1	0.5%	1.3%
Controlled	1	Controlled	2	0%	0.3%
Controlled	1	Controlled	3	0%	0%

Table 6: FRGC Uncontrolled Accuracy

Probe	Probe Elements	Gallery	Gallery Elements	FRR @ FAR 1%	FRR @ FAR 0.1%
Uncontrol	1	Uncontrol	1	5.3%	12.8%
Uncontrol	1	Uncontrol	2	1.2%	4.3%
Uncontrol	1	Uncontrol	3	1%	2.3%

FRGC Multiple Member Gallery Controlled Identification Results

FIgure 9: FAR/FRR Graph showing the accuracy of FACEngine ID against the FRGC Controlled Dataset with 3 member galleries (left) and 2 member galleries (right).

FRGC Controlled 1, 2 and 3 Member Gallery Accuracy

Figure 10: FAR/FRR Summary Graph showing FACEngine ID accuracy against 1 member (red), 2 member (green) and 3 member (blue) galleries using FRGC controlled data.

FRGC Multiple Member Gallery Uncontrolled Identification Results

Figure 11: FAR/FRR Graph showing the accuracy of FACEngine ID against the FRGC Controlled Dataset with 3 member galleries (left) and 2 member galleries (right).

FRGC Uncontrolled 1, 2 and 3 Member Gallery Accuracy

Figure 12: FAR/FRR Summary Graph showing FACEngine ID accuracy against 1 member (red), 2 member (green) and 3 member (blue) galleries using FRGC uncontrolled data.