Facial Recognition Technology Takes Center Stage

This is Part 4 of our series on the goings on in the BlueFletch IoT Lab. The Internet of Things is the next big revolution in technology and we’ve got our fingers on the pulse. Stay tuned for exciting projects, both personal and professional coming out of the Lab.

In our last post we discussed how the Sigfox network has furthered the projects in our lab, including our secure entry system. In this installment I’ll expand on the facial recognition component of that system. Andres took the lead, using the Facebook recognition system as a starting point. We’re still working out the kinks, but our first pass had some interesting results I’d like to share with you here.

When contemplating adding facial recognition technology to our access system, Andres was inspired by this article describing how to reverse engineer the system Facebook uses when we tag our friends. From this starting point, Andres used the Raspberry PI, OpenCV, the Facebook API, Python, and nodeJS to build a face recognition system. Under his system, the Raspberry Pi can be utilized for any number of functions, including sending a signal to the GPIO pins to “Open Doors,” sending a message via Sigfox to record the name of the person that was recognized or performing a calculation based on its’ findings.

IoT, BlueFletch IoT LabHere’s How it Works:

Step 1: Using OpenCV in the Raspberry Pi, Andres was able to take a picture with the Raspberry Pi camera and go through a routine, or function, that checks if there are any faces on the image taken. If there is indeed a face, it goes to the next step –  

Step 2: Using the Facebook API, Andres was able to authenticate to Facebook, obtaining an access Token and using the GraphAPI to push the image to his Facebook profile, keeping it private. The API returned the image ID, which is used in step # 3 –  

Step 3: The Facebook recognition system works by calling a URL with a POST body that includes the photo ID (the one we received in the previous step). Andres needed to create a nodeJS server that would expect an image ID and then send a request to the Facebook recognition URL endpoint using authenticated cookies.

He used this github project as a guide to build the NodeJS server and adopted the recognition method to fit our needs. Before getting cookies from the browser he had to authenticate through the Facebook web portal and then use Chrome Developer Tools to get the Facebook authentication cookies.

Step 4: Once the recognition request is made it will return any friend names that were detected on the image taken. The Raspberry Pi will parse those names out and display their names back to the client or send the names to Sigfox if required.

Challenges We Faced:

Recognition

The Facebook recognition API was not super reliable. We assume that their recognition system uses repeated tags of a particular user to build a database for future recognition. If a particular user doesn’t get tagged often, the recognition system probably is not going to recognize that face.

Security

When using Facebook as a system to recognize a user, you need to expose your AppId, access token, and App Secret, which could open doors to attackers or someone to manipulate your Facebook page if they get access to the code or this info.

Ease of Use

In order for Andres to hit the Facebook recognition URL he needed to have an active Facebook session and pass the authentication cookies with his request every time. This makes it a nightmare because anyone can access and manipulate his Facebook if they have access to this info, but also if the session ended he would have to go to the browser authenticate and re create/ add the cookies to the server so his request would be valid, a very time consuming process with big security risks.

Next Steps

In Phase 2 of our project we plan to build our own database. Our platform will ask the user if he/she wants to be recognized and added to the system. If the user agrees to be added, it will take 20 images of said user in about 2 – 3 mins, then it will process the images using the eigen decomposition algorithm and add the image with a name to our new database. After the process finishes, the user can request the platform to identify them and open the lab door.

Previous posts in this series include:
Lab Happenings
Sigfox in the IoT Lab
Halloween Hack

Related Posts

Lauren Lynn

Lauren Lynn

Operations and Marketing Manager - Lauren is responsible for managing event scheduling, marketing, and anything related to keeping the lights on. Lauren has a background managing Art Galleries in New York and Charleston and holds a BA in Art History from Wofford College.