１　簡介

OpenCV從版本2.4開始，加入了一個類FaceRecognizer，使用它可以方便的地進行人臉識別(源代碼，在OpenCV的opencv\modules\contrib\doc\facerec\src下)。

目前支持三種算法（BSD license）：

Eigenfaces特征臉：　createEigenFaceRecognizer()

Fisherfaces： createFisherFaceRecognizer()

LocalBinary Patterns Histograms（局部二值直方圖）：createLBPHFaceRecognizer()

２　人臉數(shù)據(jù)庫

１）　http://face-rec.org/databases/

２）　http://face-rec.org

　　

• AT&T Facedatabase The AT&T Facedatabase, sometimes also referred to asORL Database of Faces, contains ten different images of each of 40 distinct subjects. For some subjects, the images were taken at different times, varying the lighting, facial expressions (open / closed eyes, smiling / not smiling) and facial details (glasses / no glasses). All the images were taken against a dark homogeneous background with the subjects in an upright, frontal position (with tolerance for some side movement).

• Yale Facedatabase A, also known as Yalefaces. The AT&T Facedatabase is good for initial tests, but it’s a fairly easy database. The Eigenfaces method already has a 97% recognition rate on it, so you won’t see any great improvements with other algorithms. The Yale Facedatabase A (also known as Yalefaces) is a more appropriate dataset for initial experiments, because the recognition problem is harder. The database consists of 15 people (14 male, 1 female) each with 11 grayscale images sized ３２０Ｘ２４３ pixel. There are changes in the light conditions (center light, left light, right light), facial expressions (happy, normal, sad, sleepy, surprised, wink) and glasses (glasses, no-glasses).

  The original images are not cropped and aligned. Please look into the Appendix for a Python script, that does the job for you.

• Extended Yale Facedatabase B The Extended Yale Facedatabase B contains 2414 images of 38 different people in its cropped version. The focus of this database is set on extracting features that are robust to illumination, the images have almost no variation in emotion/occlusion/... . I personally think, that this dataset is too large for the experiments I perform in this document. You better use the AT&T Facedatabase for intial testing. A first version of the Yale Facedatabase B was used in[BHK97] to see how the Eigenfaces and Fisherfaces method perform under heavy illumination changes.[Lee05] used the same setup to take 16128 images of 28 people. The Extended Yale Facedatabase B is the merge of the two databases, which is now known as Extended Yalefacedatabase B.

３　數(shù)據(jù)準(zhǔn)備的格式和方法

用ＣＳＶ文件存儲下載的人臉數(shù)據(jù)的路徑和標(biāo)簽，路徑和標(biāo)簽用分號（；）隔開，格式如下：

　　

/path/to/image.ext;0
其中/path/to/image.ext為圖片的路徑，標(biāo)簽０為人的序號標(biāo)簽。

下載AT&T的數(shù)據(jù)庫和對應(yīng)的ＣＳＶ文件，格式是這樣的：

./at/s1/1.pgm;0 ./at/s1/2.pgm;0 ... ./at/s2/1.pgm;1 ./at/s2/2.pgm;1 ... ./at/s40/1.pgm;39 ./at/s40/2.pgm;39
ＣＳＶ文件的創(chuàng)建可以用ＯｐｅｎＣＶ提供的腳本 create_csv.py完成。

例如；數(shù)據(jù)的存儲路徑數(shù)如下：

philipp@mango:~/facerec/data/at$ tree . |-- s1 | |-- 1.pgm | |-- ... | |-- 10.pgm |-- s2 | |-- 1.pgm | |-- ... | |-- 10.pgm ... |-- s40 | |-- 1.pgm | |-- ... | |-- 10.pgm那么創(chuàng)建ＣＳＶ文件的方式如下：

philipp@mango:~/facerec/data$ python create_csv.py at/s13/2.pgm;0 at/s13/7.pgm;0 at/s13/6.pgm;0 at/s13/9.pgm;0 at/s13/5.pgm;0 at/s13/3.pgm;0 at/s13/4.pgm;0 at/s13/10.pgm;0 at/s13/8.pgm;0 at/s13/1.pgm;0 at/s17/2.pgm;1 at/s17/7.pgm;1 at/s17/6.pgm;1 at/s17/9.pgm;1 at/s17/5.pgm;1 at/s17/3.pgm;1 [...]

４　Eigenfaces

4.1 算法描述

令 ? 表示一個隨機特征，其中?.

????? 1) 計算均值向量

????? 2) 計算協(xié)方差矩陣?S
?????? 3) 計算S的特征值和對應(yīng)的特征向量??? ?
? ? ? ???
?????? 4)對特征值進行遞減排序，特征向量和它順序一致. K個主成分也就是k個最大的特征值對應(yīng)的特征向量。

?????

????? x的K個主成份:

? ? ? ?

?

?????? 其中? .

??????

?????? PCA基的重構(gòu):

? ? ? ? ? ? ? ? ??

?

?????? 其中??.

????

????? 然后特征臉通過下面的方式進行人臉識別：

???? A．? 把所有的訓(xùn)練數(shù)據(jù)投影到PCA子空間

???? B．? 把待識別圖像投影到PCA子空間

???? C．? 找到訓(xùn)練數(shù)據(jù)投影后的向量和待識別圖像投影后的向量最近的那個。

?仍然有一個問題有待解決。假設(shè)給定400張100*100像素大小的圖像，PCA需要解決協(xié)方差矩陣 的求解，而X的大小是10000*400，那么將會得到10000*10000大小的矩陣，大概0.8GB的內(nèi)存。解決這個問題不容易，所以我們需要另一個計策，就是轉(zhuǎn)置一下再求，特征向量不變化，描述如下：

　線性代數(shù)課程中講到。對于一個 的矩陣，如果 你只能得到 個非零的奇異值. So it’s possible to take the eigenvalue decomposition of size instead:　and get the original eigenvectors of with a left multiplication of the data matrix:

最終的結(jié)果奇異值向量是正交的, 要得到單位正交向量需要歸一化為單位長度.文獻?[Duda01]中有描述。

代碼：

/** Copyright (c) 2011. Philipp Wagner <bytefish[at]gmx[dot]de>.* Released to public domain under terms of the BSD Simplified license.** Redistribution and use in source and binary forms, with or without* modification, are permitted provided that the following conditions are met:* * Redistributions of source code must retain the above copyright* notice, this list of conditions and the following disclaimer.* * Redistributions in binary form must reproduce the above copyright* notice, this list of conditions and the following disclaimer in the* documentation and/or other materials provided with the distribution.* * Neither the name of the organization nor the names of its contributors* may be used to endorse or promote products derived from this software* without specific prior written permission.** See <http://www.opensource.org/licenses/bsd-license>*/#include "opencv2/core/core.hpp" #include "opencv2/contrib/contrib.hpp" #include "opencv2/highgui/highgui.hpp"#include <iostream> #include <fstream> #include <sstream>using namespace cv; using namespace std;static Mat norm_0_255(InputArray _src) {Mat src = _src.getMat();// Create and return normalized image:Mat dst;switch(src.channels()) {case 1:cv::normalize(_src, dst, 0, 255, NORM_MINMAX, CV_8UC1);break;case 3:cv::normalize(_src, dst, 0, 255, NORM_MINMAX, CV_8UC3);break;default:src.copyTo(dst);break;}return dst; }static void read_csv(const string& filename, vector<Mat>& images, vector<int>& labels, char separator = ';') {std::ifstream file(filename.c_str(), ifstream::in);if (!file) {string error_message = "No valid input file was given, please check the given filename.";CV_Error(CV_StsBadArg, error_message);}string line, path, classlabel;while (getline(file, line)) {stringstream liness(line);getline(liness, path, separator);getline(liness, classlabel);if(!path.empty() && !classlabel.empty()) {images.push_back(imread(path, 0));labels.push_back(atoi(classlabel.c_str()));}} }int main(int argc, const char *argv[]) {// Check for valid command line arguments, print usage// if no arguments were given.if (argc < 2) {cout << "usage: " << argv[0] << " <csv.ext> <output_folder> " << endl;exit(1);}string output_folder = ".";if (argc == 3) {output_folder = string(argv[2]);}// Get the path to your CSV.string fn_csv = string(argv[1]);// These vectors hold the images and corresponding labels.vector<Mat> images;vector<int> labels;// Read in the data. This can fail if no valid// input filename is given.try {read_csv(fn_csv, images, labels);} catch (cv::Exception& e) {cerr << "Error opening file \"" << fn_csv << "\". Reason: " << e.msg << endl;// nothing more we can doexit(1);}// Quit if there are not enough images for this demo.if(images.size() <= 1) {string error_message = "This demo needs at least 2 images to work. Please add more images to your data set!";CV_Error(CV_StsError, error_message);}// Get the height from the first image. We'll need this// later in code to reshape the images to their original// size:int height = images[0].rows;// The following lines simply get the last images from// your dataset and remove it from the vector. This is// done, so that the training data (which we learn the// cv::FaceRecognizer on) and the test data we test// the model with, do not overlap.Mat testSample = images[images.size() - 1];int testLabel = labels[labels.size() - 1];images.pop_back();labels.pop_back();// The following lines create an Eigenfaces model for// face recognition and train it with the images and// labels read from the given CSV file.// This here is a full PCA, if you just want to keep// 10 principal components (read Eigenfaces), then call// the factory method like this://// cv::createEigenFaceRecognizer(10);//// If you want to create a FaceRecognizer with a// confidence threshold (e.g. 123.0), call it with://// cv::createEigenFaceRecognizer(10, 123.0);//// If you want to use _all_ Eigenfaces and have a threshold,// then call the method like this://// cv::createEigenFaceRecognizer(0, 123.0);//Ptr<FaceRecognizer> model = createEigenFaceRecognizer();model->train(images, labels);// The following line predicts the label of a given// test image:int predictedLabel = model->predict(testSample);//// To get the confidence of a prediction call the model with://// int predictedLabel = -1;// double confidence = 0.0;// model->predict(testSample, predictedLabel, confidence);//string result_message = format("Predicted class = %d / Actual class = %d.", predictedLabel, testLabel);cout << result_message << endl;// Here is how to get the eigenvalues of this Eigenfaces model:Mat eigenvalues = model->getMat("eigenvalues");// And we can do the same to display the Eigenvectors (read Eigenfaces):Mat W = model->getMat("eigenvectors");// Get the sample mean from the training dataMat mean = model->getMat("mean");// Display or save:if(argc == 2) {imshow("mean", norm_0_255(mean.reshape(1, images[0].rows)));} else {imwrite(format("%s/mean.png", output_folder.c_str()), norm_0_255(mean.reshape(1, images[0].rows)));}// Display or save the Eigenfaces:for (int i = 0; i < min(10, W.cols); i++) {string msg = format("Eigenvalue #%d = %.5f", i, eigenvalues.at<double>(i));cout << msg << endl;// get eigenvector #iMat ev = W.col(i).clone();// Reshape to original size & normalize to [0...255] for imshow.Mat grayscale = norm_0_255(ev.reshape(1, height));// Show the image & apply a Jet colormap for better sensing.Mat cgrayscale;applyColorMap(grayscale, cgrayscale, COLORMAP_JET);// Display or save:if(argc == 2) {imshow(format("eigenface_%d", i), cgrayscale);} else {imwrite(format("%s/eigenface_%d.png", output_folder.c_str(), i), norm_0_255(cgrayscale));}}// Display or save the image reconstruction at some predefined steps:for(int num_components = min(W.cols, 10); num_components < min(W.cols, 300); num_components+=15) {// slice the eigenvectors from the modelMat evs = Mat(W, Range::all(), Range(0, num_components));Mat projection = subspaceProject(evs, mean, images[0].reshape(1,1));Mat reconstruction = subspaceReconstruct(evs, mean, projection);// Normalize the result:reconstruction = norm_0_255(reconstruction.reshape(1, images[0].rows));// Display or save:if(argc == 2) {imshow(format("eigenface_reconstruction_%d", num_components), reconstruction);} else {imwrite(format("%s/eigenface_reconstruction_%d.png", output_folder.c_str(), num_components), reconstruction);}}// Display if we are not writing to an output folder:if(argc == 2) {waitKey(0);}return 0; }

#include "opencv2/core/core.hpp" #include "opencv2/contrib/contrib.hpp" #include "opencv2/highgui/highgui.hpp"#include <iostream> #include <fstream> #include <sstream>usingnamespace cv; usingnamespace std;static Mat norm_0_255(InputArray _src) {Mat src = _src.getMat();// 創(chuàng)建和返回一個歸一化后的圖像矩陣:Mat dst;switch(src.channels()) {case1:cv::normalize(_src, dst, 0,255, NORM_MINMAX, CV_8UC1);break;case3:cv::normalize(_src, dst, 0,255, NORM_MINMAX, CV_8UC3);break;default:src.copyTo(dst);break;}return dst; } //使用CSV文件去讀圖像和標(biāo)簽，主要使用stringstream和getline方法 staticvoid read_csv(const string& filename, vector<Mat>& images, vector<int>& labels, char separator =';') {std::ifstream file(filename.c_str(), ifstream::in);if (!file) {string error_message ="No valid input file was given, please check the given filename.";CV_Error(CV_StsBadArg, error_message);}string line, path, classlabel;while (getline(file, line)) {stringstream liness(line);getline(liness, path, separator);getline(liness, classlabel);if(!path.empty()&&!classlabel.empty()) {images.push_back(imread(path, 0));labels.push_back(atoi(classlabel.c_str()));}} }int main(int argc, constchar*argv[]) {// 檢測合法的命令，顯示用法// 如果沒有參數(shù)輸入則退出！.if (argc <2) {cout <<"usage: "<< argv[0]<<" <csv.ext> <output_folder> "<< endl;exit(1);}string output_folder;if (argc ==3) {output_folder = string(argv[2]);}//讀取你的CSV文件路徑.string fn_csv = string(argv[1]);// 2個容器來存放圖像數(shù)據(jù)和對應(yīng)的標(biāo)簽vector<Mat> images;vector<int> labels;// 讀取數(shù)據(jù). 如果文件不合法就會出錯// 輸入的文件名已經(jīng)有了.try {read_csv(fn_csv, images, labels);} catch (cv::Exception& e) {cerr <<"Error opening file \""<< fn_csv <<"\". Reason: "<< e.msg << endl;// 文件有問題，我們啥也做不了了，退出了exit(1);}// 如果沒有讀取到足夠圖片，我們也得退出.if(images.size()<=1) {string error_message ="This demo needs at least 2 images to work. Please add more images to your data set!";CV_Error(CV_StsError, error_message);}// 得到第一張照片的高度. 在下面對圖像// 變形到他們原始大小時需要int height = images[0].rows;// 下面的幾行代碼僅僅是從你的數(shù)據(jù)集中移除最后一張圖片//[gm:自然這里需要根據(jù)自己的需要修改，他這里簡化了很多問題]Mat testSample = images[images.size() -1];int testLabel = labels[labels.size() -1];images.pop_back();labels.pop_back();// 下面幾行創(chuàng)建了一個特征臉模型用于人臉識別，// 通過CSV文件讀取的圖像和標(biāo)簽訓(xùn)練它。// T這里是一個完整的PCA變換//如果你只想保留10個主成分，使用如下代碼// cv::createEigenFaceRecognizer(10);//// 如果你還希望使用置信度閾值來初始化，使用以下語句：// cv::createEigenFaceRecognizer(10, 123.0);//// 如果你使用所有特征并且使用一個閾值，使用以下語句：// cv::createEigenFaceRecognizer(0, 123.0);//Ptr<FaceRecognizer> model = createEigenFaceRecognizer();model->train(images, labels);// 下面對測試圖像進行預(yù)測，predictedLabel是預(yù)測標(biāo)簽結(jié)果int predictedLabel = model->predict(testSample);//// 還有一種調(diào)用方式，可以獲取結(jié)果同時得到閾值:// int predictedLabel = -1;// double confidence = 0.0;// model->predict(testSample, predictedLabel, confidence);//string result_message = format("Predicted class = %d / Actual class = %d.", predictedLabel, testLabel);cout << result_message << endl;// 這里是如何獲取特征臉模型的特征值的例子，使用了getMat方法:Mat eigenvalues = model->getMat("eigenvalues");// 同樣可以獲取特征向量:Mat W = model->getMat("eigenvectors");// 得到訓(xùn)練圖像的均值向量Mat mean = model->getMat("mean");// 顯示還是保存:if(argc==2) {imshow("mean", norm_0_255(mean.reshape(1, images[0].rows)));} else {imwrite(format("%s/mean.png", output_folder.c_str()), norm_0_255(mean.reshape(1, images[0].rows)));}// 顯示還是保存特征臉:for (int i =0; i < min(10, W.cols); i++) {string msg = format("Eigenvalue #%d = %.5f", i, eigenvalues.at<double>(i));cout << msg << endl;// 得到第 #i個特征Mat ev = W.col(i).clone();//把它變成原始大小，為了把數(shù)據(jù)顯示歸一化到0~255.Mat grayscale = norm_0_255(ev.reshape(1, height));// 使用偽彩色來顯示結(jié)果，為了更好的感受.Mat cgrayscale;applyColorMap(grayscale, cgrayscale, COLORMAP_JET);// 顯示或者保存:if(argc==2) {imshow(format("eigenface_%d", i), cgrayscale);} else {imwrite(format("%s/eigenface_%d.png", output_folder.c_str(), i), norm_0_255(cgrayscale));}}// 在一些預(yù)測過程中，顯示還是保存重建后的圖像:for(int num_components =10; num_components <300; num_components+=15) {// 從模型中的特征向量截取一部分Mat evs = Mat(W, Range::all(), Range(0, num_components));Mat projection = subspaceProject(evs, mean, images[0].reshape(1,1));Mat reconstruction = subspaceReconstruct(evs, mean, projection);// 歸一化結(jié)果，為了顯示:reconstruction = norm_0_255(reconstruction.reshape(1, images[0].rows));// 顯示或者保存:if(argc==2) {imshow(format("eigenface_reconstruction_%d", num_components), reconstruction);} else {imwrite(format("%s/eigenface_reconstruction_%d.png", output_folder.c_str(), num_components), reconstruction);}}// 如果我們不是存放到文件中，就顯示他，這里使用了暫定等待鍵盤輸入:if(argc==2) {waitKey(0);}return0; }

OpenCV中代碼路徑：　src/facerec_eigenfaces.cpp

因為使用了偽彩色圖像（jet colormap），所以可以看到在特征臉中灰度值是如何分布的?？梢钥吹教卣髂槻坏珜θ四樚卣鬟M行編碼，還對圖像中的光照進行編碼。(看第四張圖像是左側(cè)的光照，而第五張是右側(cè)的光照)：

備注：PCA對光照變化圖像識別效果很差

We’ve already seen, that we can reconstruct a face from its lower dimensional approximation. So let’s see how many Eigenfaces are needed for a good reconstruction. I’ll do a subplot with Eigenfaces（可以利用低維近似來重構(gòu)人臉，對于一個好的重構(gòu)，看一下需要多少特征臉。將依次畫出10，30，。。310張?zhí)卣髂槙r的效果）：

// Display or save the image reconstruction at some predefined steps: for(int num_components = 10; num_components < 300; num_components+=15) {// slice the eigenvectors from the modelMat evs = Mat(W, Range::all(), Range(0, num_components));Mat projection = subspaceProject(evs, mean, images[0].reshape(1,1));Mat reconstruction = subspaceReconstruct(evs, mean, projection);// Normalize the result:reconstruction = norm_0_255(reconstruction.reshape(1, images[0].rows));// Display or save:if(argc == 2) {imshow(format("eigenface_reconstruction_%d", num_components), reconstruction);} else {imwrite(format("%s/eigenface_reconstruction_%d.png", output_folder.c_str(), num_components), reconstruction);} }
顯然10個特征向量（ 備注：　1個特征向量可以變形成一個特征臉，這里特征向量和特征臉概念有些近似）是不夠的，50個特征向量可以有效的編碼出重要的人臉特征。在AT&T數(shù)據(jù)庫中，當(dāng)使用300個特征向量時，可以獲取一個比較好的和重構(gòu)結(jié)果。 有定理可以給出出重構(gòu)需要選擇多少特征臉才合適，但它嚴重依賴于輸入數(shù)據(jù)。文獻 [Zhao03]是一個好的開始研究起點。

５　Fisherfaces

5.1 算法描述

令x是一個來自c個類中的隨機向量，

? ? ? ? ? ? ? ? ? ?

?

散度矩陣?? 和S_{W}如下計算:

? ? ? ? ? ? ? ? ??

?

, 其中 ??是全部數(shù)據(jù)的均值 ? ??:

? ? ? ? ? ? ??

?

而? 是某個類? 的均值:

? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

?

Fisher的分類算法可以看出一個投影矩陣? , 使得類的可分性最大:

? ? ? ? ? ? ? ? ??

?

根據(jù) [BHK97], 一個解決這個普通特征值優(yōu)化問題的方法被提出:

? ? ? ? ? ? ??? ? ?

?

???????? 還有一個問題要解決： Sw的秩最多為 （N-c）,? 包含N?個樣本和c個類別。在模式識別中，樣本數(shù)據(jù)個數(shù)N的一般小于輸入數(shù)據(jù)的維數(shù)（像素的數(shù)量）。 （例如。圖片數(shù)量N=400，而大小１００ｘ１００＝10000就是數(shù)據(jù)維數(shù)）。那么，散度矩陣Sw就是奇異的(文獻[RJ91])。在文獻[BHK97]中，使用PCA把數(shù)據(jù)投影到(N-c)維的子空間來解決，然后再使用線性鑒別分析（A Linear Discriminant Analysis），因為Sw不是奇異矩陣了(可逆矩陣)。

然后優(yōu)化問題可以寫成：

? ? ? ? ?

?

可以把樣本投影到(c-1)維的空間的變換矩陣W，可以表示為

? ? ? ? ? ? ? ??

?代碼：

/** Copyright (c) 2011. Philipp Wagner <bytefish[at]gmx[dot]de>.* Released to public domain under terms of the BSD Simplified license.** Redistribution and use in source and binary forms, with or without* modification, are permitted provided that the following conditions are met:* * Redistributions of source code must retain the above copyright* notice, this list of conditions and the following disclaimer.* * Redistributions in binary form must reproduce the above copyright* notice, this list of conditions and the following disclaimer in the* documentation and/or other materials provided with the distribution.* * Neither the name of the organization nor the names of its contributors* may be used to endorse or promote products derived from this software* without specific prior written permission.** See <http://www.opensource.org/licenses/bsd-license>*/#include "opencv2/core/core.hpp" #include "opencv2/contrib/contrib.hpp" #include "opencv2/highgui/highgui.hpp"#include <iostream> #include <fstream> #include <sstream>using namespace cv; using namespace std;static Mat norm_0_255(InputArray _src) {Mat src = _src.getMat();// Create and return normalized image:Mat dst;switch(src.channels()) {case 1:cv::normalize(_src, dst, 0, 255, NORM_MINMAX, CV_8UC1);break;case 3:cv::normalize(_src, dst, 0, 255, NORM_MINMAX, CV_8UC3);break;default:src.copyTo(dst);break;}return dst; }static void read_csv(const string& filename, vector<Mat>& images, vector<int>& labels, char separator = ';') {std::ifstream file(filename.c_str(), ifstream::in);if (!file) {string error_message = "No valid input file was given, please check the given filename.";CV_Error(CV_StsBadArg, error_message);}string line, path, classlabel;while (getline(file, line)) {stringstream liness(line);getline(liness, path, separator);getline(liness, classlabel);if(!path.empty() && !classlabel.empty()) {images.push_back(imread(path, 0));labels.push_back(atoi(classlabel.c_str()));}} }int main(int argc, const char *argv[]) {// Check for valid command line arguments, print usage// if no arguments were given.if (argc < 2) {cout << "usage: " << argv[0] << " <csv.ext> <output_folder> " << endl;exit(1);}string output_folder = ".";if (argc == 3) {output_folder = string(argv[2]);}// Get the path to your CSV.string fn_csv = string(argv[1]);// These vectors hold the images and corresponding labels.vector<Mat> images;vector<int> labels;// Read in the data. This can fail if no valid// input filename is given.try {read_csv(fn_csv, images, labels);} catch (cv::Exception& e) {cerr << "Error opening file \"" << fn_csv << "\". Reason: " << e.msg << endl;// nothing more we can doexit(1);}// Quit if there are not enough images for this demo.if(images.size() <= 1) {string error_message = "This demo needs at least 2 images to work. Please add more images to your data set!";CV_Error(CV_StsError, error_message);}// Get the height from the first image. We'll need this// later in code to reshape the images to their original// size:int height = images[0].rows;// The following lines simply get the last images from// your dataset and remove it from the vector. This is// done, so that the training data (which we learn the// cv::FaceRecognizer on) and the test data we test// the model with, do not overlap.Mat testSample = images[images.size() - 1];int testLabel = labels[labels.size() - 1];images.pop_back();labels.pop_back();// The following lines create an Fisherfaces model for// face recognition and train it with the images and// labels read from the given CSV file.// If you just want to keep 10 Fisherfaces, then call// the factory method like this://// cv::createFisherFaceRecognizer(10);//// However it is not useful to discard Fisherfaces! Please// always try to use _all_ available Fisherfaces for// classification.//// If you want to create a FaceRecognizer with a// confidence threshold (e.g. 123.0) and use _all_// Fisherfaces, then call it with://// cv::createFisherFaceRecognizer(0, 123.0);//Ptr<FaceRecognizer> model = createFisherFaceRecognizer();model->train(images, labels);// The following line predicts the label of a given// test image:int predictedLabel = model->predict(testSample);//// To get the confidence of a prediction call the model with://// int predictedLabel = -1;// double confidence = 0.0;// model->predict(testSample, predictedLabel, confidence);//string result_message = format("Predicted class = %d / Actual class = %d.", predictedLabel, testLabel);cout << result_message << endl;// Here is how to get the eigenvalues of this Eigenfaces model:Mat eigenvalues = model->getMat("eigenvalues");// And we can do the same to display the Eigenvectors (read Eigenfaces):Mat W = model->getMat("eigenvectors");// Get the sample mean from the training dataMat mean = model->getMat("mean");// Display or save:if(argc == 2) {imshow("mean", norm_0_255(mean.reshape(1, images[0].rows)));} else {imwrite(format("%s/mean.png", output_folder.c_str()), norm_0_255(mean.reshape(1, images[0].rows)));}// Display or save the first, at most 16 Fisherfaces:for (int i = 0; i < min(16, W.cols); i++) {string msg = format("Eigenvalue #%d = %.5f", i, eigenvalues.at<double>(i));cout << msg << endl;// get eigenvector #iMat ev = W.col(i).clone();// Reshape to original size & normalize to [0...255] for imshow.Mat grayscale = norm_0_255(ev.reshape(1, height));// Show the image & apply a Bone colormap for better sensing.Mat cgrayscale;applyColorMap(grayscale, cgrayscale, COLORMAP_BONE);// Display or save:if(argc == 2) {imshow(format("fisherface_%d", i), cgrayscale);} else {imwrite(format("%s/fisherface_%d.png", output_folder.c_str(), i), norm_0_255(cgrayscale));}}// Display or save the image reconstruction at some predefined steps:for(int num_component = 0; num_component < min(16, W.cols); num_component++) {// Slice the Fisherface from the model:Mat ev = W.col(num_component);Mat projection = subspaceProject(ev, mean, images[0].reshape(1,1));Mat reconstruction = subspaceReconstruct(ev, mean, projection);// Normalize the result:reconstruction = norm_0_255(reconstruction.reshape(1, images[0].rows));// Display or save:if(argc == 2) {imshow(format("fisherface_reconstruction_%d", num_component), reconstruction);} else {imwrite(format("%s/fisherface_reconstruction_%d.png", output_folder.c_str(), num_component), reconstruction);}}// Display if we are not writing to an output folder:if(argc == 2) {waitKey(0);}return 0; }

#include "opencv2/core/core.hpp"#include "opencv2/contrib/contrib.hpp"#include "opencv2/highgui/highgui.hpp"#include <iostream>#include <fstream>#include <sstream>usingnamespace cv;usingnamespace std;static Mat norm_0_255(InputArray _src) {Mat src = _src.getMat();// 創(chuàng)建和返回歸一化的圖像:Mat dst;switch(src.channels()) {case1:cv::normalize(_src, dst, 0,255, NORM_MINMAX, CV_8UC1);break;case3:cv::normalize(_src, dst, 0,255, NORM_MINMAX, CV_8UC3);break;default:src.copyTo(dst);break;}return dst;}staticvoid read_csv(const string& filename, vector<Mat>& images, vector<int>& labels, char separator =';') {std::ifstream file(filename.c_str(), ifstream::in);if (!file) {string error_message ="No valid input file was given, please check the given filename.";CV_Error(CV_StsBadArg, error_message);}string line, path, classlabel;while (getline(file, line)) {stringstream liness(line);getline(liness, path, separator);getline(liness, classlabel);if(!path.empty()&&!classlabel.empty()) {images.push_back(imread(path, 0));labels.push_back(atoi(classlabel.c_str()));}}}int main(int argc, constchar*argv[]) {// 判斷輸入命令是否有效，輸出用法// 如果沒有輸入?yún)?shù).if (argc <2) {cout <<"usage: "<< argv[0]<<" <csv.ext> <output_folder> "<< endl;exit(1);}string output_folder;if (argc ==3) {output_folder = string(argv[2]);}// 獲取CSV文件的路徑.string fn_csv = string(argv[1]);// 這些容器存放圖片和標(biāo)簽.vector<Mat> images;vector<int> labels;// 載入數(shù)據(jù).如果不合理，會出錯// 輸入文件名fn_csv已經(jīng)有了.try {read_csv(fn_csv, images, labels);} catch (cv::Exception& e) {cerr <<"Error opening file \""<< fn_csv <<"\". Reason: "<< e.msg << endl;// 什么也不能做了exit(1);}// 如果沒有足夠圖像就退出掉.if(images.size()<=1) {string error_message ="This demo needs at least 2 images to work. Please add more images to your data set!";CV_Error(CV_StsError, error_message);}int height = images[0].rows;Mat testSample = images[images.size() -1];int testLabel = labels[labels.size() -1];images.pop_back();labels.pop_back();// 如果想保存10個fisherfaces// cv::createFisherFaceRecognizer(10);//// 如果要以123.0作為置信閾值// cv::createFisherFaceRecognizer(0, 123.0);//Ptr<FaceRecognizer> model = createFisherFaceRecognizer();model->train(images, labels);int predictedLabel = model->predict(testSample);//// model->predict(testSample, predictedLabel, confidence);//string result_message = format("Predicted class = %d / Actual class = %d.", predictedLabel, testLabel);cout << result_message << endl;Mat eigenvalues = model->getMat("eigenvalues");Mat W = model->getMat("eigenvectors");Mat mean = model->getMat("mean");if(argc==2) {imshow("mean", norm_0_255(mean.reshape(1, images[0].rows)));} else {imwrite(format("%s/mean.png", output_folder.c_str()), norm_0_255(mean.reshape(1, images[0].rows)));}//顯示還是保存, 最多16 Fisherfaces:for (int i =0; i < min(16, W.cols); i++) {string msg = format("Eigenvalue #%d = %.5f", i, eigenvalues.at<double>(i));cout << msg << endl;Mat ev = W.col(i).clone();Mat grayscale = norm_0_255(ev.reshape(1, height));// 使用Bone偽彩色圖像來顯示.Mat cgrayscale;applyColorMap(grayscale, cgrayscale, COLORMAP_BONE);if(argc==2) {imshow(format("fisherface_%d", i), cgrayscale);} else {imwrite(format("%s/fisherface_%d.png", output_folder.c_str(), i), norm_0_255(cgrayscale));}}for(int num_component =0; num_component < min(16, W.cols); num_component++) {Mat ev = W.col(num_component);Mat projection = subspaceProject(ev, mean, images[0].reshape(1,1));Mat reconstruction = subspaceReconstruct(ev, mean, projection);reconstruction = norm_0_255(reconstruction.reshape(1, images[0].rows));if(argc==2) {imshow(format("fisherface_reconstruction_%d", num_component), reconstruction);} else {imwrite(format("%s/fisherface_reconstruction_%d.png", output_folder.c_str(), num_component), reconstruction);}}if(argc==2) {waitKey(0);}return0;}

OpenCV中代碼的路徑： src/facerec_fisherfaces.cpp

為顯示更好些，在這個例子使用了YaleA人臉數(shù)據(jù)庫。每一個Fisherface都和原始圖像有同樣長度，因此它可以被排列顯示在一幅圖像上。下面顯示了16張Fisherfaces圖像。

? Fisherfaces方法學(xué)習(xí)一個正對標(biāo)簽的轉(zhuǎn)換矩陣（class-specific transformation matrix），所依它不會如特征臉那樣capture illumination那樣明顯。鑒別分析是尋找可以區(qū)分人的面部特征。需要特別說明的是，Fisherfaces的性能嚴重依賴于輸入數(shù)據(jù)。實際上，如果用光照好的圖片訓(xùn)練學(xué)習(xí)Fisherfaces，對光照不好的圖片進行識別，那么他可能會找到錯誤的主元（因為在光照不好的圖片上，這些特征不優(yōu)越）。這是符合邏輯的，因為這個方法沒有機會去學(xué)習(xí)光照。(備注：那么采集圖像時就要考慮光照變化，訓(xùn)練時考慮所有光照情況，數(shù)據(jù)庫multi-pie就考慮很多種光照）。

???????? 和特征臉一樣，Fisherfaces允許對投影圖像進行重建。由于我們僅僅使用這些特征來區(qū)分不同的類別，所以無法期待對原圖像有一個好的重建效果。（備注：也就是特征臉把每個圖片看成一個個體，重建時效果也有保證，而Fisherfaces把一個人的照片看成一個整體，那么重建時重建的效果則不是很好）。對于Fisherfaces方法我們將把樣本圖像逐個投影到Fisherfaces上。因此你可以獲得一個好的可視效果，每個Fisherfaces特征可以被描述為：

// Display or save the image reconstruction at some predefined steps: for(int num_component = 0; num_component < min(16, W.cols); num_component++) {// Slice the Fisherface from the model:Mat ev = W.col(num_component);Mat projection = subspaceProject(ev, mean, images[0].reshape(1,1));Mat reconstruction = subspaceReconstruct(ev, mean, projection);// Normalize the result:reconstruction = norm_0_255(reconstruction.reshape(1, images[0].rows));// Display or save:if(argc == 2) {imshow(format("fisherface_reconstruction_%d", num_component), reconstruction);} else {imwrite(format("%s/fisherface_reconstruction_%d.png", output_folder.c_str(), num_component), reconstruction);} }
對于人眼來說，差異比較微妙，但是任然可以看到一些差異的。

６　Local Binary Patterns Histograms

Eigenfaces和Fisherfaces使用整體方法來進行人臉識別[gm:直接使用所有的像素]。你把你的數(shù)據(jù)當(dāng)作圖像空間的高維向量。我們都知道高維數(shù)據(jù)是糟糕的，所以一個低維子空間被確定，對于信息保存可能很好。Eigenfaces是最大化總的散度，這樣可能導(dǎo)致，當(dāng)方差由外部條件產(chǎn)生時，最大方差的主成分不適合用來分類。所以為使用一些鑒別分析，我們使用了LDA方法來優(yōu)化。Fisherfaces方法可以很好的運作，至少在我們假設(shè)的模型的有限情況下。

???????? 現(xiàn)實生活是不完美的。你無法保證在你的圖像中光照條件是完美的，或者說1個人的10張照片。所以，如果每人僅僅只有一張照片呢？我們的子空間的協(xié)方差估計方法可能完全錯誤，所以識別也可能錯誤。是否記得Eigenfaces在AT&T數(shù)據(jù)庫上達到了96%的識別率？對于這樣有效的估計，我們需要多少張訓(xùn)練圖像呢？下圖是Eigenfaces和Fisherfaces方法在AT&T數(shù)據(jù)庫上的首選識別率，這是一個簡單的數(shù)據(jù)庫：

?

???????? 因此，若你想得到好的識別率，你大約需要每個人有8(7~9)張圖像，而Fisherfaces在這里并沒有好的幫助。以上的實驗是10個圖像的交叉驗證結(jié)果，使用了facerec框架：?https://github.com/bytefish/facerec。這不是一個刊物，所以我不會用高深的數(shù)學(xué)分析來證明這個圖像。?當(dāng)遇到小的訓(xùn)練數(shù)據(jù)集時，可以看一下文獻[KM01]，了解二種方法的細節(jié)分析。

???????? 一些研究專注于圖像局部特征的提取。主意是我們不把整個圖像看成一個高維向量，僅僅用局部特征來描述一個物體。通過這種方式提取特征，你將獲得一個低維隱式。一個好主意！但是你很快發(fā)現(xiàn)這種圖像表示方法不僅僅遭受光照變化。你想想圖像中的尺度變化、形變、旋轉(zhuǎn)—我們的局部表示方式起碼對這些情況比較穩(wěn)健。正如SIFT，LBP方法在2D紋理分析中舉足輕重。LBP的基本思想是對圖像的像素和它局部周圍像素進行對比后的結(jié)果進行求和。把這個像素作為中心，對相鄰像素進行閾值比較。如果中心像素的亮度大于等于他的相鄰像素，把他標(biāo)記為1，否則標(biāo)記為0。你會用二進制數(shù)字來表示每個像素，比如11001111。因此，由于周圍相鄰8個像素，你最終可能獲取2^8個可能組合，被稱為局部二值模式，有時被稱為LBP碼。第一個在文獻中描述的LBP算子實際使用的是3*3的鄰域。

6.1 算法描述

一個更加正式的LBP操作可以被定義為

? ? ? ? ? ? ? ? ? ??

?

其中 是中心像素，亮度是 ；而則是相鄰像素的亮度。s是一個符號函數(shù)：

?

?? ? ??

這種描述方法使得你可以很好的捕捉到圖像中的細節(jié)。實際上，研究者們可以用它在紋理分類上得到最先進的水平。正如剛才描述的方法被提出后，固定的近鄰區(qū)域?qū)τ诔叨茸兓木幋a失效。所以，使用一個變量的擴展方法，在文獻[AHP04]中有描述。主意是使用可變半徑的圓對近鄰像素進行編碼，這樣可以捕捉到如下的近鄰：

? ? ? ? ? ??

對一個給定的點? ?，他的近鄰點?可以由如下計算：

? ? ? ??

其中，R是圓的半徑，而P是樣本點的個數(shù)。

這個操作是對原始LBP算子的擴展，所以有時被稱為擴展LBP(又稱為圓形LBP)。如果一個在圓上的點不在圖像坐標(biāo)上，我們使用他的內(nèi)插點。計算機科學(xué)有一堆聰明的插值方法，而OpenCV使用雙線性插值。

? ? ? ? ?

LBP算子，對于灰度的單調(diào)變化很穩(wěn)健。我們可以看到手工改變后的圖像的LBP圖像(你可以看到LBP圖像是什么樣子的！)

? ? ? ? ? ?

?

???????? 那么剩下來的就是如何合并空間信息用于人臉識別模型。Ahonenet. Al在文獻?[AHP04]中提出表示方法，對LBP圖像成m個塊，每個塊提取直方圖。通過連接局部特直方圖(而不是合并)然后就能得到空間增強的特征向量。這些直方圖被稱為局部二值模式直方圖。

代碼：

/** Copyright (c) 2011. Philipp Wagner <bytefish[at]gmx[dot]de>.* Released to public domain under terms of the BSD Simplified license.** Redistribution and use in source and binary forms, with or without* modification, are permitted provided that the following conditions are met:* * Redistributions of source code must retain the above copyright* notice, this list of conditions and the following disclaimer.* * Redistributions in binary form must reproduce the above copyright* notice, this list of conditions and the following disclaimer in the* documentation and/or other materials provided with the distribution.* * Neither the name of the organization nor the names of its contributors* may be used to endorse or promote products derived from this software* without specific prior written permission.** See <http://www.opensource.org/licenses/bsd-license>*/#include "opencv2/core/core.hpp" #include "opencv2/contrib/contrib.hpp" #include "opencv2/highgui/highgui.hpp"#include <iostream> #include <fstream> #include <sstream>using namespace cv; using namespace std;static void read_csv(const string& filename, vector<Mat>& images, vector<int>& labels, char separator = ';') {std::ifstream file(filename.c_str(), ifstream::in);if (!file) {string error_message = "No valid input file was given, please check the given filename.";CV_Error(CV_StsBadArg, error_message);}string line, path, classlabel;while (getline(file, line)) {stringstream liness(line);getline(liness, path, separator);getline(liness, classlabel);if(!path.empty() && !classlabel.empty()) {images.push_back(imread(path, 0));labels.push_back(atoi(classlabel.c_str()));}} }int main(int argc, const char *argv[]) {// Check for valid command line arguments, print usage// if no arguments were given.if (argc != 2) {cout << "usage: " << argv[0] << " <csv.ext>" << endl;exit(1);}// Get the path to your CSV.string fn_csv = string(argv[1]);// These vectors hold the images and corresponding labels.vector<Mat> images;vector<int> labels;// Read in the data. This can fail if no valid// input filename is given.try {read_csv(fn_csv, images, labels);} catch (cv::Exception& e) {cerr << "Error opening file \"" << fn_csv << "\". Reason: " << e.msg << endl;// nothing more we can doexit(1);}// Quit if there are not enough images for this demo.if(images.size() <= 1) {string error_message = "This demo needs at least 2 images to work. Please add more images to your data set!";CV_Error(CV_StsError, error_message);}// Get the height from the first image. We'll need this// later in code to reshape the images to their original// size:int height = images[0].rows;// The following lines simply get the last images from// your dataset and remove it from the vector. This is// done, so that the training data (which we learn the// cv::FaceRecognizer on) and the test data we test// the model with, do not overlap.Mat testSample = images[images.size() - 1];int testLabel = labels[labels.size() - 1];images.pop_back();labels.pop_back();// The following lines create an LBPH model for// face recognition and train it with the images and// labels read from the given CSV file.//// The LBPHFaceRecognizer uses Extended Local Binary Patterns// (it's probably configurable with other operators at a later// point), and has the following default values//// radius = 1// neighbors = 8// grid_x = 8// grid_y = 8//// So if you want a LBPH FaceRecognizer using a radius of// 2 and 16 neighbors, call the factory method with://// cv::createLBPHFaceRecognizer(2, 16);//// And if you want a threshold (e.g. 123.0) call it with its default values://// cv::createLBPHFaceRecognizer(1,8,8,8,123.0)//Ptr<FaceRecognizer> model = createLBPHFaceRecognizer();model->train(images, labels);// The following line predicts the label of a given// test image:int predictedLabel = model->predict(testSample);//// To get the confidence of a prediction call the model with://// int predictedLabel = -1;// double confidence = 0.0;// model->predict(testSample, predictedLabel, confidence);//string result_message = format("Predicted class = %d / Actual class = %d.", predictedLabel, testLabel);cout << result_message << endl;// Sometimes you'll need to get/set internal model data,// which isn't exposed by the public cv::FaceRecognizer.// Since each cv::FaceRecognizer is derived from a// cv::Algorithm, you can query the data.//// First we'll use it to set the threshold of the FaceRecognizer// to 0.0 without retraining the model. This can be useful if// you are evaluating the model://model->set("threshold", 0.0);// Now the threshold of this model is set to 0.0. A prediction// now returns -1, as it's impossible to have a distance below// itpredictedLabel = model->predict(testSample);cout << "Predicted class = " << predictedLabel << endl;// Show some informations about the model, as there's no cool// Model data to display as in Eigenfaces/Fisherfaces.// Due to efficiency reasons the LBP images are not stored// within the model:cout << "Model Information:" << endl;string model_info = format("\tLBPH(radius=%i, neighbors=%i, grid_x=%i, grid_y=%i, threshold=%.2f)",model->getInt("radius"),model->getInt("neighbors"),model->getInt("grid_x"),model->getInt("grid_y"),model->getDouble("threshold"));cout << model_info << endl;// We could get the histograms for example:vector<Mat> histograms = model->getMatVector("histograms");// But should I really visualize it? Probably the length is interesting:cout << "Size of the histograms: " << histograms[0].total() << endl;return 0; }
#include "opencv2/core/core.hpp"#include "opencv2/contrib/contrib.hpp"#include "opencv2/highgui/highgui.hpp"#include <iostream>#include <fstream>#include <sstream>usingnamespace cv;usingnamespace std;staticvoid read_csv(const string& filename, vector<Mat>& images, vector<int>& labels, char separator =';') {std::ifstream file(filename.c_str(), ifstream::in);if (!file) {string error_message ="No valid input file was given, please check the given filename.";CV_Error(CV_StsBadArg, error_message);}string line, path, classlabel;while (getline(file, line)) {stringstream liness(line);getline(liness, path, separator);getline(liness, classlabel);if(!path.empty()&&!classlabel.empty()) {images.push_back(imread(path, 0));labels.push_back(atoi(classlabel.c_str()));}}}int main(int argc, constchar*argv[]) {if (argc !=2) {cout <<"usage: "<< argv[0]<<" <csv.ext>"<< endl;exit(1);}string fn_csv = string(argv[1]);vector<Mat> images;vector<int> labels;try {read_csv(fn_csv, images, labels);} catch (cv::Exception& e) {cerr <<"Error opening file \""<< fn_csv <<"\". Reason: "<< e.msg << endl;// nothing more we can doexit(1);}if(images.size()<=1) {string error_message ="This demo needs at least 2 images to work. Please add more images to your data set!";CV_Error(CV_StsError, error_message);}int height = images[0].rows;Mat testSample = images[images.size() -1];int testLabel = labels[labels.size() -1];images.pop_back();labels.pop_back();// TLBPHFaceRecognizer 使用了擴展的LBP// 在其他的算子中他可能很容易被擴展// 下面是默認參數(shù)// radius = 1// neighbors = 8// grid_x = 8// grid_y = 8//// 如果你要創(chuàng)建 LBPH FaceRecognizer 半徑是2，16個鄰域// cv::createLBPHFaceRecognizer(2, 16);//// 如果你需要一個閾值，并且使用默認參數(shù):// cv::createLBPHFaceRecognizer(1,8,8,8,123.0)//Ptr<FaceRecognizer> model = createLBPHFaceRecognizer();model->train(images, labels);int predictedLabel = model->predict(testSample);// int predictedLabel = -1;// double confidence = 0.0;// model->predict(testSample, predictedLabel, confidence);//string result_message = format("Predicted class = %d / Actual class = %d.", predictedLabel, testLabel);cout << result_message << endl;// 有時你需要設(shè)置或者獲取內(nèi)部數(shù)據(jù)模型,// 他不能被暴露在 cv::FaceRecognizer類中.//// 首先我們對FaceRecognizer的閾值設(shè)置到0.0，而不是重寫訓(xùn)練模型// 當(dāng)你重新估計模型時很重要 //model->set("threshold",0.0);predictedLabel = model->predict(testSample);cout <<"Predicted class = "<< predictedLabel << endl;// 由于確保高效率，LBP圖沒有被存儲在模型里面。Dcout <<"Model Information:"<< endl;string model_info = format("\tLBPH(radius=%i, neighbors=%i, grid_x=%i, grid_y=%i, threshold=%.2f)",model->getInt("radius"),model->getInt("neighbors"),model->getInt("grid_x"),model->getInt("grid_y"),model->getDouble("threshold"));cout << model_info << endl;// 我們可以獲取樣本的直方圖:vector<Mat> histograms = model->getMatVector("histograms");// 我需要現(xiàn)實它嗎? 或許它的長度才是我們感興趣的:cout <<"Size of the histograms: "<< histograms[0].total()<< endl;return0;}

??? 參考文獻：　http://docs.opencv.org/2.4/modules/contrib/doc/facerec/facerec_tutorial.html

　　　http://www.cnblogs.com/guoming0000/archive/2012/09/27/2706019.html

總結(jié)

以上是生活随笔為你收集整理的使用OpenCV进行人脸识别的三种方法的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。