Dr. Margarita Osadchy: System for Secure Face Identification (SCIFI)
Inventors: Dr. Margarita Osadchy, Dr.Binyamin Pinkas, : Dr. Ayman Jarrous, : Dr.Boaz Moskovich,
A method for privacy-retaining face identification comprising dividing at least one input face into a first set of patches, providing an assortment of patch profiles for each patch in the first set of patches, each patch profile in the assortment being associated with a unique index thereby to define an assortment of unique indices, for each individual patch in the input face, finding at least one patch profile within the assortment of patch profiles which most resembles the individual patch, thereby to define a first set of indices from among the assortment of unique indices whose corresponding patch profiles most resemble the input face's patches respectively, and using a privacy preserving computation to generate a physical output indicative of a comparison of the input face and at least one target face by comparing first and second functions of the first set of indices and of a second set of indices respectively, the second set of indices corresponding to patch profiles which most resemble a second set of patches into which the target face has been divided.
New Invention :
Certain embodiments of SCiFI, a system for Secure Computation of Face Identification, are now described. The term “SCiFI” as used herein refers to any or all of such embodiments. Certain embodiments of the system perform face identification which compares faces of subjects with a database of registered faces. The identification is done in a secure way which protects both the privacy of the subjects and the confidentiality of the database. A specific application of SCiFI is reducing the privacy impact of camera based surveillance. In that scenario, SCiFI would be used in a setting which contains a server which has a set of faces of suspects, and client machines which might be cameras acquiring images in public places. The system runs a secure computation of a face recognition method, which identifies if an image acquired by a client matches one of the suspects, but otherwise reveals no information to neither of the parties.
Certain embodiments of the present invention seek to provide a new face identification method which is suitable for usage in secure computation. Nonetheless, the method has face recognition performance comparable to that of state of the art algorithms. Experiments show the method to be robust to different viewing conditions, such as illumination, occlusions, and changes in appearance, like wearing glasses.
Certain embodiments of the present invention seek to provide a secure protocol for computing the new face recognition method. In addition, it is preferable to optimize the protocol and minimize its online latency.
Certain embodiments of the present invention seek to provide a SCiFI system which implements a secure computation of the face identification protocol.
Experiments show that the entire system can run in near real-time: The secure computation protocol performs a preprocessing of all public-key cryptographic operations. Its online performance therefore typically depends mainly on the speed of data communication, and experiments performed show it to be extremely efficient.
Video and camera based surveillance is very common, and is found to be useful for fighting crime. On the other hand, the ubiquity of such surveillance is a major concern for the public, that feels that its privacy is being violated e.g. as per the ACLU's report on this subject . Face recognition systems can automatically identify if some known suspects appear in a large set of images. Such systems can be useful, for example, for automatically searching for suspects in a stream of images coming from public places. On the other hand, these systems can be misused to track people regardless of suspicion, and a rogue operator can even combine it with a universal database linking faces to identities, such as a database of drivers' license photos.
Optionally, each camera may be provided with the list of images of suspects, perform the identification task locally at the camera, and report to the operator only in the rare cases where a match is found. This solution might not be acceptable, though, if the list of suspects is confidential, as is often the case. There is therefore a need for a solution which protects both the privacy of the public and the confidentiality of the data held by authorities. Certain embodiments of the SCiFI system match images taken by a client camera to a list of images e.g. of potential suspects which are held by a server. Face identification in SCiFI is based on a novel face recognition method that performs very well in terms of applicability to real-life images and robustness to unseen conditions e.g., images taken under different illumination conditions. The matching is done in a privacy preserving way, using efficient methods of secure computation, and does not reveal any information to the parties, except for whether a match was found. Furthermore, to further protect the database of suspects it is possible to distribute the operation of the server to run on several machines, such that each of these machines knows only part of each suspect's face. Denote this system as dSCiFI. The overhead of dSCiFI is only negligibly higher than that of certain basic SCiFI systems.
SCiFI typically resides in a server, which stores a set of face representations of subjects from a confidential list, and a client, whose input is a single face. In a typical setting the server might have a list of faces of suspected persons, while the client might be a camera which takes photos of passersby. The recognition module of the client typically receives images of faces preprocessed for face recognition. Methods for face detection or localization that perform this preprocessing are known in the art. The face detection methods are known in conventional digital cameras, and compute a bounding box containing a face, given an image of a face. The system is operative to find out if the face held by the client matches any of the faces in the server's list. As with any biometric data, it is unlikely that there will be an exact match between the image acquired by the client and the image of the same person that exists in the list. Therefore a fuzzy matching face identification method is typically used. The matching is done in a privacy preserving manner in that the server and client typically do not learn any information except for whether there is a match between the client's input and a face in the server's list.
The face recognition methods shown and described herein have good performance in terms of robustness of recognition, and can also support secure computation protocols. Face recognition algorithms typically use data representations over the Real numbers, whereas secure protocols operate over finite fields, and a simple conversion of existing face recognition methods to finite field's results in degradation of recognition. Secure protocols, based on homomorphic encryption and oblivious transfer, computing the face recognition method are described herein. Experiments were conducted showing the accuracy of the face recognition method and the nearly real-time performance of the secure protocols.
Applications : For fast and private identification of customers