This post was inspired by a YouTube video I published on my channel some time ago. We will do Face Recognition in JAVA.
We are partnering up with TechInDeep to bring you a full course on Deep Learning in JAVA.
Before continuing with this post make sure to check out some other articles as well:
If you want to learn how to build Neural Networks from scratch you can check:
Demonstration Video
Source Code Download
Programming Language:
- JAVA
What libraries are we going to use:
- OpenCV
- ND4J
- DL4J
Who is this article for:
- A developer who already is somewhat familiar with the Face Recognition problem. For all of you that are struggling with this solution, I will be posting a very detailed tutorial on each workflow step described bellow backed up with code files and many examples. For the more experienced ones we are still not done. This application can be improved by a lot.
Additional Questions:
- I will be answering all additional questions here.
Workflow
- Detect Face (Phase I)
- Extract face feature vector (Phase II)
- Find the most similar face from our data set (Phase III)
So let’s get started…
Face Detection
In order to create a face detector I used OpenCV. This library provides us with a great real time face detection algorithm using Haar Cascades. OpenCV for JAVA has a class called CascadeClassifier that implements the algorithm. At object creation time of the class, you will need to provide the location of the xml file that CascadeClassifier needs in order to detect faces on the image. The xml file that I actually use is a pre-trained file that is dispatched with OpenCV. The file is called “haarcascade_frontalface_alt.xml” and you will find it included in the project. Detecting faces is as simple as calling the function:
detectMultiScale(image,matFaceList,_scaleFactor,_minNeighbours,0,_minFaceSize,_maxFaceSize);
If you want to know more about this function I suggest to read the following documentation on OpenCV. More in depth on this problem and OpenCV API will be covered in another tutorial.
The output from detectMultiScale is a MatOfRect type that contains an array of type Rect. Now this array actually contains the X and Y coordinates, as well as the Width and Height of the area where the face is located on the image. Now given that rectangle we can actually extract the faces individually from the image. Since the image is loaded in a Mat, we can extract the face pretty easy just by calling the submat function and passing the face Rect as a input parameter. Next what I do is I save the face as an image. And that is pretty much it.
The whole process can be seen in ExtractFacesFromImageTask.java file.
With this we finish the task of Face Detection.
As I said more on this topic in another tutorial…
Face Recognition in Java
Once we have extracted the faces we can continue with the next item defined in our workflow and that is face recognition. Since the goal is to create an app that can recognized undefined number of faces, we need to split this section in two. The first part will explain how to obtain facial features and the second one would measure the similarity between the extracted facial features and the features we have in a data set.
Data Set in our project is a file that contains list of face feature vectors and face labels. So every single face feature vector would contain a label as well.
Creating a facial features vector
In order to recognize faces we would need to extract face features from a given image. We can either train our own neural network to get good quality facial features or we can use a pre-trained one. In this project I do use a pre-trained one.
Pre-trained network is just a model that was already trained and has calculated weights in it. That’s all…
Usually people like using the first couple of layers of such network because they tend to generalize better. The main advantage is that I don’t have to train the network myself. Sometimes I don’t have big enough training set or I don’t have the necessary computing power to do the training. So I opt out for using these pre-trained models.
Creating the facial features vector boils down to using a pre-trained neural network model that we can obtain from the DL4J library. In order to do that we would need to load the “VGGFace” Computational Graph. The idea is to pass the face image into the “VGGFace” computational graph to process it and we would use the output of layer “pool4” as our input into the classification method we will use later. Since this is a pre-trained model it would extract excellent features for the classifier. I chose to get the outputs out of the layer named “pool4” which I think it is OK for this demo.
So why this approach?
In order to create a face recognition in java, that can be trained on unknown amount of faces, I could not use the whole pre-trained network, since I don’t know how many faces I would enter.
Instead I decided to take the output from the neural network and write it as a n-dimensional vector. The logic behind this is the following: Since similar faces would yield similar vectors they will be grouped in a cluster. Or simply put, similar faces would end up having similar vector elements.
So if I want to classify a new face, I would run the same procedure, get the output vector from the neural network and calculate the distance between the vectors that are already in my data set. My data set contains a vector and a label, so If I find which vector is most similar to the one I am processing at the moment I will also find the label. I do that using Euclidean Distance but even better solution would be KNN algorithm.
Face Recognition implementation workflow
So the code starts with loading in the weights of “VGGFace” into our Computational Graph. We read in the image using NativeImageLoader to transform the image into the correct size for the “VGGFace” neural network. Then we use the function Featurize from our TransferLearningHelper object. Next we would like to “flatten” the matrix, into a vector, so we use the function reshape from ND4J library.
Finally we have our vector that represents the facial features from the person and we have a label. We combine them and put them in a data set. When a new face comes along, we use the same procedure as described above to get the feature vector, flatten it and compare it to the rest of our data set. The vector that has the closest distance to it (most similar) wins and we display the label (the name of the person).
If the presented label is wrong we can correct it by entering the new label. Please note that we already have the feature vector for the face so all we need to do is insert this record into our data set. We use this method to correct miss classifications and increase the accuracy.
Final Words
A basic Face Recognition in Java technique is explained. This is it for now…
We will cover every topic described here in more detail very soon. Also please note that this is not the most optimal and best solution for this problem. So how to improve accuracy and optimize for speed will be covered in another post.
Complete source code: Face Recognition in Java Project
kamila
July 9, 2019 at 7:12 amHello sir,
Can we develop a face recognition system using java???? but without using any libraries.
vanco
July 10, 2019 at 8:53 amYes you can. And it is not even that hard. All you need to implement is a Convolutional Neural Network and K-Means. But at least use some fast and reliable Linear Algebra library for the matrix computation. I don’t recommend you to write the linear algebra operations yourself because there is a good chance of ending up with a very slow performance code.
Good luck on the project 🙂
hakan
July 17, 2019 at 10:41 amError ı’m getting what ı should ı do?
Exception in Application start method
Exception in thread “main” java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
vanco
July 25, 2019 at 10:33 amPlease check if you have correct libraries to run the application.
hakan
July 17, 2019 at 1:11 pmWhat is a Problem?
Exception in Application start method
Exception in thread “main” java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at java.base/sun.launcher.LauncherHelper$FXHelper.main(LauncherHelper.java:1051)
Caused by: java.lang.RuntimeException: Exception in Application start method
at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplication1(LauncherImpl.java:900)
at javafx.graphics/com.sun.javafx.application.LauncherImpl.lambda$launchApplication$2(LauncherImpl.java:195)
at java.base/java.lang.Thread.run(Thread.java:835)
Caused by: java.lang.IllegalAccessError: superclass access check failed: class com.sun.javafx.scene.control.ControlHelper (in unnamed module @0x698004e4) cannot access class com.sun.javafx.scene.layout.RegionHelper (in module javafx.graphics) because module javafx.graphics does not export com.sun.javafx.scene.layout to unnamed module @0x698004e4
at java.base/java.lang.ClassLoader.defineClass1(Native Method)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1016)
at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:151)
at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:802)
at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:700)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:623)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
at javafx.scene.control.Control.(Control.java:86)
at View.MainView.(MainView.java:36)
at View.ApplicationController.showMainView(ApplicationController.java:29)
at App.start(App.java:10)
at javafx.graphics/com.sun.javafx.application.LauncherImpl.lambda$launchApplication1$9(LauncherImpl.java:846)
at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runAndWait$12(PlatformImpl.java:455)
at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$10(PlatformImpl.java:428)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:389)
at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$11(PlatformImpl.java:427)
at javafx.graphics/com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:96)
at javafx.graphics/com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
at javafx.graphics/com.sun.glass.ui.win.WinApplication.lambda$runLoop$3(WinApplication.java:174)
… 1 more
Tony Nguru
August 14, 2019 at 8:34 pmHi Vanco.
Thanx a lot for this project. I did manage to have it run without using maven. Had to compile opencv 4.0 and cuda 10 though and went through a few challenges especially coz of using different versions of bytedeco-javacv and bytedeco-javacpp but i finally figured it out in a normal netbeans project.
Now for the next step am trying to see of i can save the trained data into a new haar cascade model or some other supported standard using DL4J. I saw you save only at runtime in a trainedList. Which disappear after the program terminates. Is it possible you also try it out once you get some time.
vanco
August 16, 2019 at 6:50 amHi Tony,
Yes it is possible to save it. In my original project instead of working with memory I work with HDF5 file type. It behaves as my face database. So inside I would save the INDArray I get from the Neural Netowork. That would create a cluster of similar faces. All you have to do is assign centroids to those clusters in order to describe them with one central point.
Good job Tony,
Thank you for your comment.
zie
October 15, 2019 at 3:28 pmException in thread “main” java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.launcher.LauncherHelper$FXHelper.main(LauncherHelper.java:767)
Caused by: java.lang.RuntimeException: Exception in Application start method
at com.sun.javafx.application.LauncherImpl.launchApplication1(LauncherImpl.java:917)
at com.sun.javafx.application.LauncherImpl.lambda$launchApplication$159(LauncherImpl.java:182)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Pretrained model file failed checksum.
vanco
October 30, 2019 at 6:59 amOK, so from the error I can see that the pretrained model I use has not downloaded successfully. The first time you run the application there is a big file for the VGG model that the app automatically downloads. Please wait for the app to properly finish.
zie
October 15, 2019 at 3:30 pmHow to deal with the problem I met above,thanks!
Nora Kovacs
February 17, 2021 at 7:48 pmhi can not download source code the link does not work. is it still available?
many thanks,
Nora
vanco
February 17, 2021 at 8:30 pmHi Nora,
I just checked the link and it is working as expected. Let me know if you still experience download issues
Simran
February 23, 2021 at 7:16 amHi Vanco.
So, honestly I’m pretty much a novice here. Although I’ve been doing java programming. Question is, OpenCV, ND4J, DKL4J, are they just downloadable?? Also, the pretrained model….is it supposed to download the first time I run the program? Or I can download it outside the program?
vanco
February 23, 2021 at 9:06 amHi Simran,
OpenCV, ND4J and DL4J are packages that you can include in your project. Those are dependencies that you must include in your project. They are downloaded and included. The pre-trained model is also downloaded automatically at first run. So you have to wait (depending on your connection) for maybe 20 mins to download the model. On the dependencies and how to include them please check out the Quick Start for Java using Maven: https://deeplearning4j.konduit.ai/getting-started/quickstart
Good luck
vpn special code
March 31, 2024 at 9:14 pmThis is a topic which is near to my heart… Best wishes!
Where are your contact details though?
Also visit my webpage … vpn special code
vpn coupon 2024
March 31, 2024 at 10:55 pmHi there, I want to subscribe for this weblog to take most recent updates, therefore where can i do it please assist.
Feel free to visit my homepage vpn coupon 2024
vpn special coupon
April 1, 2024 at 5:33 pmThe other day, while I was at work, my sister stole
my apple ipad and tested to see if it can survive a twenty five
foot drop, just so she can be a youtube sensation. My iPad is now broken and she has 83 views.
I know this is totally off topic but I had to share it with someone!
Feel free to visit my blog post … vpn special coupon
vpn special coupon code 2024
April 7, 2024 at 11:34 amSuperb site you have here but I was curious about if you knew of any community forums that cover the same topics
discussed in this article? I’d really love to be a part
of group where I can get comments from other experienced individuals
that share the same interest. If you have any recommendations,
please let me know. Appreciate it!
my web-site :: vpn special coupon code 2024
vpn special code
April 9, 2024 at 10:07 pmVery great post. I just stumbled upon your blog and wanted
to mention that I have really enjoyed surfing around your weblog posts.
In any case I will be subscribing to your rss feed and I’m hoping
you write once more soon!
My web site: vpn special code
Haircuts
April 19, 2024 at 11:24 pmThank you sharing these kind of wonderful articles. In addition, the perfect travel as well as medical insurance approach can often ease those worries that come with travelling abroad. Any medical emergency can soon become costly and that’s likely to quickly impose a financial impediment on the family finances. Setting up in place the perfect travel insurance offer prior to setting off is well worth the time and effort. Thank you
Beauty Fashion
April 21, 2024 at 3:49 amHi there this is kind of of off topic but I was wanting to know if blogs use WYSIWYG editors or if you have to manually code with HTML. I’m starting a blog soon but have no coding knowledge so I wanted to get advice from someone with experience. Any help would be greatly appreciated!
Hottest Hairstyles
April 24, 2024 at 3:29 amYou are a very smart person!
Black Hairstyles
April 29, 2024 at 1:55 amI discovered more new things on this losing weight issue. One particular issue is a good nutrition is extremely vital any time dieting. An enormous reduction in bad foods, sugary food items, fried foods, sweet foods, red meat, and bright flour products could be necessary. Possessing wastes organisms, and contaminants may prevent ambitions for fat loss. While specific drugs momentarily solve the situation, the bad side effects are not worth it, and so they never offer more than a momentary solution. This is a known fact that 95 of fad diet plans fail. Many thanks sharing your ideas on this weblog.
Amrit
April 29, 2024 at 7:15 pmHi there! I could have sworn I’ve been to this blog before but after reading through some of the post I realized
it’s new to me. Anyhow, I’m definitely glad I found it and I’ll be bookmarking
and checking back often!
Fashion
April 30, 2024 at 5:26 amThank you for some other great article. Where else may just anybody get that type of info in such a perfect means of writing? I’ve a presentation subsequent week, and I am on the search for such information.
winning303
April 30, 2024 at 7:49 amWhen someone writes an post he/she retains the thought of a user in his/her brain that
how a user can understand it. Thus that’s why
this piece of writing is outstdanding. Thanks!
Barclay
April 30, 2024 at 12:31 pmDo you have a spam problem on this website; I also am a blogger, and I was
wanting to know your situation; many of us have created some nice methods and we are looking to exchange methods with other
folks, be sure to shoot me an e-mail if interested.
Sachi
April 30, 2024 at 12:35 pmHi! Do you know if they make any plugins to protect against hackers?
I’m kinda paranoid about losing everything I’ve worked hard on. Any tips?
Vana
April 30, 2024 at 12:36 pmHello There. I found your blog using msn. This is a very well written article.
I will be sure to bookmark it and return to read more of your useful info.
Thanks for the post. I’ll definitely return.
Anup
April 30, 2024 at 12:41 pmThis piece of writing will help the internet people for setting up new webpage
or even a blog from start to end.
Corban
April 30, 2024 at 12:50 pmVery energetic article, I enjoyed that bit. Will there be a part 2?
Patricie
April 30, 2024 at 1:11 pmHi to all, the contents existing at this website are genuinely remarkable for people knowledge, well,
keep up the good work fellows.
Reo
April 30, 2024 at 1:16 pmI was able to find good advice from your blog articles.
Nhut
April 30, 2024 at 1:30 pmMy spouse and I stumbled over here different
web page and thought I should check things out.
I like what I see so now i’m following you. Look forward to looking
over your web page repeatedly.
Situs Judi Slot303
April 30, 2024 at 4:27 pmHello there, You have done a fantastic job. I’ll definitely
digg it and personally recommend to my friends.
I’m sure they’ll be benefited from this site.
winning303
April 30, 2024 at 4:48 pmWOW just what I was searching for. Came here by searching for winning303
Corena
April 30, 2024 at 5:56 pmI am truly delighted to glance at this web site posts which contains plenty of
useful information, thanks for providing such data.
Kristofer
April 30, 2024 at 5:58 pmNice post. I was checking continuously this blog and I’m impressed!
Very helpful info specially the last part 🙂 I care for such information a lot.
I was looking for this certain information for a very long time.
Thank you and good luck.
Fortunato
April 30, 2024 at 6:09 pmThis is my first time pay a quick visit at here and i am actually pleassant to read all at
single place.
Rodrick
April 30, 2024 at 6:14 pmGreat post.
naga169
April 30, 2024 at 6:14 pmHeya i’m for the first time here. I came across this board and
I in finding It truly useful & it helped me out much.
I hope to provide something again and aid others like you aided me.
Imogene
April 30, 2024 at 6:14 pmI blog often and I seriously appreciate your information. The article has truly peaked my interest.
I’m going to book mark your website and keep checking for
new information about once a week. I opted in for your Feed
too.
Loredana
April 30, 2024 at 6:30 pmI do not even know the way I finished up right here, however I believed this publish was once good.
I do not realize who you are however definitely you’re going to a famous
blogger in the event you aren’t already. Cheers!
Cythina
April 30, 2024 at 6:31 pmAttractive section of content. I just stumbled upon your site and in accession capital to
assert that I get in fact enjoyed account your blog posts.
Any way I will be subscribing to your feeds and even I achievement
you access consistently fast.
naga169
April 30, 2024 at 6:32 pmWay cool! Some extremely valid points! I appreciate you writing
this write-up plus the rest of the website is extremely good.
Chanel
April 30, 2024 at 6:34 pmIt’s genuinely very difficult in this full of activity life to listen news on Television, therefore I simply use the web for that reason, and
get the hottest news.
Nohealani
April 30, 2024 at 6:46 pmI like the helpful information you provide in your articles.
I’ll bookmark your blog and check again here frequently.
I’m quite sure I will learn plenty of new stuff right
here! Best of luck for the next!
Hokicoy login
April 30, 2024 at 9:12 pmIt’s hard to find educated people on this topic, however, you seem like
you know what you’re talking about! Thanks
warung makan
April 30, 2024 at 10:22 pmHeya i am for the primary time here. I found this board and I in finding
It really useful & it helped me out a lot.
I hope to provide one thing back and help others such as you aided me.
moschino belts on sale
May 1, 2024 at 12:13 amI do not even know how I finished up here, but I believed this post was good.
I do not recognise who you are however definitely you’re going to a well-known blogger if
you happen to are not already. Cheers!
adu ayam sv388
May 1, 2024 at 2:12 amHey I know this is off topic but I was wondering if you knew of any widgets I could add to my blog that automatically tweet my newest twitter updates.
I’ve been looking for a plug-in like this for quite some time and
was hoping maybe you would have some experience with something like this.
Please let me know if you run into anything. I truly enjoy reading your blog and I look forward to your new updates.
yupoo
May 1, 2024 at 3:51 amHey there would you mind sharing which blog platform you’re working with?
I’m looking to start my own blog soon but I’m having a tough time selecting between BlogEngine/Wordpress/B2evolution and Drupal.
The reason I ask is because your layout seems different then most blogs and I’m looking for something unique.
P.S Apologies for being off-topic but I had to ask!
cara deposit pulsa joker123
May 1, 2024 at 4:33 amWhat’s up, just wanted to mention, I enjoyed this blog post.
It was funny. Keep on posting!
Yupoo Down Jacket
May 1, 2024 at 6:44 amI am truly thankful to the holder of this website who has shared this great
piece of writing at here.
bbq
May 1, 2024 at 11:16 amI’ll right away grasp your rss feed as I can’t to find your e-mail subscription link
or e-newsletter service. Do you have any?
Please allow me recognise so that I may just subscribe. Thanks.
bbq
May 1, 2024 at 11:17 amHey there this is somewhat of off topic but I was wanting to know if blogs use WYSIWYG editors or if
you have to manually code with HTML. I’m starting a blog soon but have no
coding skills so I wanted to get advice from someone with experience.
Any help would be enormously appreciated!
Celebrities
May 1, 2024 at 11:39 amI just like the helpful info you provide for your articles. I will bookmark your blog and take a look at once more right here frequently. I’m moderately sure I抣l learn a lot of new stuff right here! Best of luck for the next!
naga169
May 1, 2024 at 2:36 pmAppreciate this post. Will try it out.
sv388
May 1, 2024 at 4:51 pmYou’re so awesome! I don’t believe I’ve read something like this before.
So nice to find somebody with genuine thoughts on this topic.
Seriously.. many thanks for starting this
up. This site is something that is needed on the internet,
someone with some originality!
Hokicoy Gacor
May 1, 2024 at 8:50 pmThis is a topic which is near to my heart… Thank you!
Exactly where are your contact details though?
agen sv388
May 1, 2024 at 9:02 pmHello would you mind letting me know which hosting company you’re utilizing?
I’ve loaded your blog in 3 completely different
internet browsers and I must say this blog loads a lot faster then most.
Can you recommend a good web hosting provider at a honest price?
Cheers, I appreciate it!