Some thoughts on Mobile Face Recognition (part 4 - Embedded processing)

Being able to embed the biometric processing in the mobile devices has gained a lot of interest for face recognition systems. Some of the advantages of the algorithms being embedded in the device are:

• Reduced volume of data exchanged over the network, since sending the video stream to a server is not needed. This allows an off-line use of the system and also response time is reduced.
• The privacy of the users is better preserved, since their biometric data stay in their own devices.
• Scalability: the computational power of the server (for biometric template extraction) does not need to grow with the number of users as much as if the processing were performed in the server.
• Some recent interoperability standards for online identification, like the one proposed by the FIDO Alliance, require a secure unlock operation (biometric or not) to release the cryptographic keys. This is accomplished through a safe action, such as the use of biometrics, but the biometric information is required to never leave the user device, so embedded biometric processing is mandatory.

However, an essential challenge remains on mobile face recognition scenario. The duration of the battery remains to be one the biggest weaknesses in mobile devices. Since energy efficiency is not a problem for traditional server-based face recognition systems, it is usually overlooked (at least, more than it would be desired for mobile scenarios). Nevertheless, it is a key issue in mobile embedded face recognition systems, so a broad study on more efficient algorithms, parallel computing optimization and exploitation of the hardware resources need to be done, as recent works point out.

An interesting topic related to the above point is the implementation of the recent Deep Neural Network paradigms for face recognition into mobile devices, taking advantage of the embedded GPU and exploiting its capabilities for energy-optimized real-time processing. Some questions to solve are how to design a proper net architecture for mobile computing or the effects of feature representation and net dimensionality on mobile face recognition accuracy.

Some thoughts on Mobile Face Recognition (part 3 - Template Protection)

Continuing with another security threat in biometric systems, one of the main concerns both from user and service provider sides is what happens if someone steals the biometric templates. A hacker might directly access system databases, obtaining the biometric templates from the users. A recent example can be found in the US government data breach in December 2014, when 5.6 million fingerprints were stolen. With them, the hacker could get improper access to the system, to other systems, and even track users in different systems. This is a big threat for the privacy of the users and the security of the system. Besides, another question arises: will the stolen biometric traits be persistently invalidated?

This threat motivates the need of protected biometric templates. The industry and the scientific community are now making big efforts for researching, standardising and extending the use of protection mechanisms, since we are aware of the problems related to the use of unprotected biometric templates. As defined in the standard ISO/IEC 24745 for biometric information protection, protected templates are required to comply with some requirements, namely:

1. Irreversibility: property of a transform that creates a biometric reference from biometric samples or features such that knowledge of the transformed biometric reference cannot be used to determine any information about the original biometric samples or features.
2. Renewability: property of a transform or process to create multiple, independent transformed biometric references derived from one or more biometric samples obtained from the same data subject and which can be used to recognize the individual while not revealing information about the original reference.
3. Revocability: ability to prevent future successful verification of a specific biometric reference and the corresponding identity reference.

The use of template protection schemes is not as extended in mobile face recognition systems as it is in other biometrics (e.g. fingerprint), so we believe it is one of the keystones to be developed shortly, in order to achieve the desired levels of privacy and security. Some of the problems to solve are to properly characterise the output signals from the different face recognition algorithms and to get the amount of entropy required for the template protection schemes to achieve a good performance in terms of recognition rates, response time and, at the same time, comply with privacy requirements.

Some thoughts on Mobile Face Recognition (part 2 - Anti-Spoofing)

Some biometric traits might be easily captured by an attacker. This is the case of faces, since almost everyone has photos publicly available in social networks like LinkedIN or Facebook. This problem motivates the recent efforts in liveness detection for a secure use of face biometrics. Anti-spoofing methods go from simple ones, for example those based on blink detection, to more complex algorithms for analysing the texture or the light in the scene.

As shown in different publications, these machine learning-based anti-spoofing methods tend to be strongly dependent on the dataset used for training the model. This means that the robustness of the liveness analysis depends on the training dataset (genuine accesses and attacks) and the technology used for face presentation and acquisition, so several concerns appear. Can their behaviour be predicted in the presence of a new attack which has not been taken into account in the training set? Can a single anti-spoofing method be enough to guarantee the security of the system?

Given the cross-dataset analysis in recent publications and real scenario tests it does not seem a good idea entrusting the security of the system to a single anti-spoofing method. This is why we believe the use of a single non-collaborative liveness detection method is not enough for guaranteeing the security of the system in real scenarios, now and in the future, since their robustness is dependent on the presentation technology used by the attacker (video quality measures, light reflectance analysis, etc.).

Alternatively, to counteract presentation attacks, a more robust solution would be the combination of several methods working together and combining automatic analysis tools with user interaction. If the system is able to provoke a reaction in the user and then analyse this reaction, fake attempts using photos or videos from the genuine users could be detected and avoided. Unfortunately, interaction can be a time consuming operation and it could reduce the usability, so the challenge here is to achieve a proper balance between security and convenience. The less perceptible the interaction is, the more usable and difficult to spoof the system will be. Current methods rely on asking the user to perform some action, but we think the future points to unconsciously action-reaction interaction analysis in order to increase both security and usability.

Some thoughts on Mobile Face Recognition (part 1)

Information access from smartphones and tablets has become mainstream both in business and personal environments over the last years. The use of these devices for accessing services like social networks, email or electronic commerce and banking has surpassed the access from traditional computers, turning mobile devices into essential tools in our everyday life. Mobility and ubiquity work are powerful tools for increasing efficiency and productivity in business (and also in personal life). However, without the proper usage, companies and users may
be exposed to security risks and threats.

Security in the access to information is one of the most important issues to consider in mobility scenarios. Passwords have been the usual mechanism for user authentication for many years. However, there are many usability and security concerns that compromise their effectiveness. People use simple passwords, they reuse them on different accounts and services, passwords can be shared and cracked, etc. The amount of different accounts and passwords we deal with these days contributes in making harder the proper usage and maintenance. As a result, we often see news and reports that alert of stolen accounts and passwords. This problem becomes critical in mobile devices, since they can be easily lost or stolen. Nevertheless, mobile devices can also become part of the solution, providing increased levels of security due to their new authentication options and capabilities.

The use of biometrics brings a more secure and convenient authentication method than traditional passwords. In the 2015 Biometrics Institute Industry Survey the use of biometrics for mobile access control has been established as the most significant development in the biometrics world over the last year. In addition, the survey points to other new applications for biometrics in mobile devices, such as mobile payments or law enforcement.

There are different biometric modalities that can be integrated in mobile devices: face, speaker, iris, fingerprint, etc. All of them have advantages and disadvantages, but one of the main benefits of face recognition (together with speaker recognition) is that, since smartphones already have integrated cameras, no additional hardware is required. Regardless of which biometric modality is used, for achieving a really effective system the following requirements must be accomplished:

1. Usability: Ease of use is a key factor for achieving low false rejection rates.
2. Security: It is important to avoid impostors to get access to the system (i.e. low false acceptance rate).
3. Availability: The verification method should be usable anywhere and at any time.

Face recognition meets these requirements and brings a powerful biometric authentication solution for mobile devices since:

1. It is easy to use and user friendly, since the user is already familiar with using the camera on the phone.
2. Current face recognition systems achieve high recognition rates, suitable for secure authentication; and
3. As stated before, face recognition does not need any additional hardware on the mobile devices. It takes advantage of the integrated camera so it is available in most smartphones.

However, there are some relevant issues for face recognition on mobile devices that remain unsolved or not enough studied. These concerns need to be addressed shortly for face recognition to be a leading contender in mobile device authentication. I will review some of these issues in future entries in this blog: liveness detection anti-spoofing methods, template protection, power consumption, availability under changing scenarios and adverse conditions or inter-device performance.

(The above text is extracted from this article, previously published in Image and Vision Computing Journal.)