Face Recognition in JAVA

Face Recognition in JAVA

This post was inspired by a YouTube video I published on my channel some time ago. We will do Face Recognition in JAVA.

We are partnering up with TechInDeep to bring you a full course on Deep Learning in JAVA.

Before continuing with this post make sure to check out some other articles as well:

If you want to learn how to build Neural Networks from scratch you can check:

Demonstration Video

Source Code Download

Programming Language:

  • JAVA

What libraries are we going to use:

  • OpenCV
  • ND4J
  • DL4J

Who is this article for:

  • A developer who already is somewhat familiar with the Face Recognition problem. For all of you that are struggling with this solution, I will be posting a very detailed tutorial on each workflow step described bellow backed up with code files and many examples. For the more experienced ones we are still not done. This application can be improved by a lot.

Additional Questions:

  • I will be answering all additional questions here.


  • Detect Face (Phase I)
  • Extract face feature vector (Phase II)
  • Find the most similar face from our data set (Phase III)

So let’s get started…

Face Detection

In order to create a face detector I used OpenCV. This library provides us with a great real time face detection algorithm using Haar Cascades. OpenCV for JAVA has a class called CascadeClassifier that implements the algorithm. At object creation time of the class, you will need to provide the location of the xml file that CascadeClassifier needs in order to detect faces on the image. The xml file that I actually use is a pre-trained file that is dispatched with OpenCV. The file is called “haarcascade_frontalface_alt.xml” and you will find it included in the project. Detecting faces is as simple as calling the function:


If you want to know more about this function I suggest to read the following documentation on OpenCV. More in depth on this problem and OpenCV API will be covered in another tutorial.

The output from detectMultiScale is a MatOfRect type that contains an array of type Rect. Now this array actually contains the X and Y coordinates, as well as the Width and Height of the area where the face is located on the image. Now given that rectangle we can actually extract the faces individually from the image. Since the image is loaded in a Mat, we can extract the face pretty easy just by calling the submat function and passing the face Rect as a input parameter. Next what I do is I save the face as an image. And that is pretty much it.

The whole process can be seen in ExtractFacesFromImageTask.java file.

With this we finish the task of Face Detection.

As I said more on this topic in another tutorial…

Face Recognition in Java

Once we have extracted the faces we can continue with the next item defined in our workflow and that is face recognition. Since the goal is to create an app that can recognized undefined number of faces, we need to split this section in two. The first part will explain how to obtain facial features and the second one would measure the similarity between the extracted facial features and the features we have in a data set.

Data Set in our project is a file that contains list of face feature vectors and face labels. So every single face feature vector would contain a label as well.

Creating a facial features vector

In order to recognize faces we would need to extract face features from a given image. We can either train our own neural network to get good quality facial features or we can use a pre-trained one. In this project I do use a pre-trained one.

Pre-trained network is just a model that was already trained and has calculated weights in it. That’s all…

Usually people like using the first couple of layers of such network because they tend to generalize better. The main advantage is that I don’t have to train the network myself. Sometimes I don’t have big enough training set or I don’t have the necessary computing power to do the training. So I opt out for using these pre-trained models.

Creating the facial features vector boils down to using a pre-trained neural network model that we can obtain from the DL4J library. In order to do that we would need to load the “VGGFace” Computational Graph. The idea is to pass the face image into the “VGGFace” computational graph to process it and we would use the output of layer “pool4” as our input into the classification method we will use later. Since this is a pre-trained model it would extract excellent features for the classifier. I chose to get the outputs out of the layer named “pool4” which I think it is OK for this demo.

So why this approach? 

In order to create a face recognition in java, that can be trained on unknown amount of faces, I could not use the whole pre-trained network, since I don’t know how many faces I would enter.

Instead I decided to take the output from the neural network and write it as a n-dimensional vector. The logic behind this is the following: Since similar faces would yield similar vectors they will be grouped in a cluster. Or simply put, similar faces would end up having similar vector elements.

So if I want to classify a new face, I would run the same procedure, get the output vector from the neural network and calculate the distance between the vectors that are already in my data set. My data set contains a vector and a label, so If I find which vector is most similar to the one I am processing at the moment I will also find the label. I do that using Euclidean Distance but even better solution would be KNN algorithm.

Face Recognition implementation workflow

So the code starts with loading in the weights of “VGGFace” into our Computational Graph. We read in the image using NativeImageLoader  to transform the image into the correct size for the “VGGFace” neural network. Then we use the function Featurize from our TransferLearningHelper object. Next we would like to “flatten” the matrix, into a vector, so we use the function reshape from ND4J library.

Finally we have our vector that represents the facial features from the person and we have a label. We combine them and put them in a data set. When a new face comes along, we use the same procedure as described above to get the feature vector, flatten it and compare it to the rest of our data set. The vector that has the closest distance to it (most similar) wins and we display the label (the name of the person).

If the presented label is wrong we can correct it by entering the new label. Please note that we already have the feature vector for the face so all we need to do is insert this record into our data set. We use this method to correct miss classifications and increase the accuracy.

Final Words

A basic Face Recognition in Java technique is explained. This is it for now…

We will cover every topic described here in more detail very soon. Also please note that this is not the most optimal and best solution for this problem. So how to improve accuracy and optimize for speed will be covered in another post.

Complete source code: Face Recognition in Java Project

14 thoughts on “Face Recognition in JAVA

  1. Reply
    July 9, 2019 at 7:12 am

    Hello sir,
    Can we develop a face recognition system using java???? but without using any libraries.

    1. Reply
      July 10, 2019 at 8:53 am

      Yes you can. And it is not even that hard. All you need to implement is a Convolutional Neural Network and K-Means. But at least use some fast and reliable Linear Algebra library for the matrix computation. I don’t recommend you to write the linear algebra operations yourself because there is a good chance of ending up with a very slow performance code.
      Good luck on the project 🙂

  2. Reply
    July 17, 2019 at 10:41 am

    Error ı’m getting what ı should ı do?

    Exception in Application start method
    Exception in thread “main” java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:567)

    1. Reply
      July 25, 2019 at 10:33 am

      Please check if you have correct libraries to run the application.

  3. Reply
    July 17, 2019 at 1:11 pm

    What is a Problem?

    Exception in Application start method
    Exception in thread “main” java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:567)
    at java.base/sun.launcher.LauncherHelper$FXHelper.main(LauncherHelper.java:1051)
    Caused by: java.lang.RuntimeException: Exception in Application start method
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplication1(LauncherImpl.java:900)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.lambda$launchApplication$2(LauncherImpl.java:195)
    at java.base/java.lang.Thread.run(Thread.java:835)
    Caused by: java.lang.IllegalAccessError: superclass access check failed: class com.sun.javafx.scene.control.ControlHelper (in unnamed module @0x698004e4) cannot access class com.sun.javafx.scene.layout.RegionHelper (in module javafx.graphics) because module javafx.graphics does not export com.sun.javafx.scene.layout to unnamed module @0x698004e4
    at java.base/java.lang.ClassLoader.defineClass1(Native Method)
    at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1016)
    at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:151)
    at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:802)
    at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:700)
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:623)
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
    at javafx.scene.control.Control.(Control.java:86)
    at View.MainView.(MainView.java:36)
    at View.ApplicationController.showMainView(ApplicationController.java:29)
    at App.start(App.java:10)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.lambda$launchApplication1$9(LauncherImpl.java:846)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runAndWait$12(PlatformImpl.java:455)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$10(PlatformImpl.java:428)
    at java.base/java.security.AccessController.doPrivileged(AccessController.java:389)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$11(PlatformImpl.java:427)
    at javafx.graphics/com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:96)
    at javafx.graphics/com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
    at javafx.graphics/com.sun.glass.ui.win.WinApplication.lambda$runLoop$3(WinApplication.java:174)
    … 1 more

  4. Reply
    Tony Nguru
    August 14, 2019 at 8:34 pm

    Hi Vanco.

    Thanx a lot for this project. I did manage to have it run without using maven. Had to compile opencv 4.0 and cuda 10 though and went through a few challenges especially coz of using different versions of bytedeco-javacv and bytedeco-javacpp but i finally figured it out in a normal netbeans project.

    Now for the next step am trying to see of i can save the trained data into a new haar cascade model or some other supported standard using DL4J. I saw you save only at runtime in a trainedList. Which disappear after the program terminates. Is it possible you also try it out once you get some time.

    1. Reply
      August 16, 2019 at 6:50 am

      Hi Tony,
      Yes it is possible to save it. In my original project instead of working with memory I work with HDF5 file type. It behaves as my face database. So inside I would save the INDArray I get from the Neural Netowork. That would create a cluster of similar faces. All you have to do is assign centroids to those clusters in order to describe them with one central point.
      Good job Tony,
      Thank you for your comment.

  5. Reply
    October 15, 2019 at 3:28 pm

    Exception in thread “main” java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at sun.launcher.LauncherHelper$FXHelper.main(LauncherHelper.java:767)
    Caused by: java.lang.RuntimeException: Exception in Application start method
    at com.sun.javafx.application.LauncherImpl.launchApplication1(LauncherImpl.java:917)
    at com.sun.javafx.application.LauncherImpl.lambda$launchApplication$159(LauncherImpl.java:182)
    at java.lang.Thread.run(Thread.java:748)
    Caused by: java.lang.IllegalStateException: Pretrained model file failed checksum.

    1. Reply
      October 30, 2019 at 6:59 am

      OK, so from the error I can see that the pretrained model I use has not downloaded successfully. The first time you run the application there is a big file for the VGG model that the app automatically downloads. Please wait for the app to properly finish.

  6. Reply
    October 15, 2019 at 3:30 pm

    How to deal with the problem I met above,thanks!

  7. Reply
    Nora Kovacs
    February 17, 2021 at 7:48 pm

    hi can not download source code the link does not work. is it still available?

    many thanks,

    1. Reply
      February 17, 2021 at 8:30 pm

      Hi Nora,
      I just checked the link and it is working as expected. Let me know if you still experience download issues

  8. Reply
    February 23, 2021 at 7:16 am

    Hi Vanco.

    So, honestly I’m pretty much a novice here. Although I’ve been doing java programming. Question is, OpenCV, ND4J, DKL4J, are they just downloadable?? Also, the pretrained model….is it supposed to download the first time I run the program? Or I can download it outside the program?

    1. Reply
      February 23, 2021 at 9:06 am

      Hi Simran,
      OpenCV, ND4J and DL4J are packages that you can include in your project. Those are dependencies that you must include in your project. They are downloaded and included. The pre-trained model is also downloaded automatically at first run. So you have to wait (depending on your connection) for maybe 20 mins to download the model. On the dependencies and how to include them please check out the Quick Start for Java using Maven: https://deeplearning4j.konduit.ai/getting-started/quickstart
      Good luck

Leave a Reply

Your email address will not be published. Required fields are marked *