Robust Person Tracking

For an intership I did during my master AI, I created a robust algorithm for the tracking of one or more people in a videostream.

It works by combining face recognition for identifying the person, with object detection for tracking the body, and finally a tracking algorithm to match the face and body together and track people over time. The advantage of this is that when the face is out of view, the person turns around, or the face is too small to identify, the object detection can still detect the person and identify them.

To make it even more robust the algorithm also remembered the last position of a person, so when the camera is obscured for some reason it could still track you when you came back into view. For instance, when someone walked in front of the camera or a frame was corrupted for some reason, the algorithm will look for the person for a couple seconds before completely forgetting them. In this way it could still track you when you put on a mask (face is covered but body still seen) and subsequently the camera is blocked for a second (algorithm tries to find person for a couple seconds) before being uncovered again.

Combining face recognition and object tracking for more robust person tracking.

Combining face recognition and object tracking for more robust person tracking.

For the implementation I used the open source face_recognition library, and YOLO V3 for the person detection. For the tracking I used the centroid algorithm from DLIB and was inspired by this nice blogpost. The entire thing was programmed in Python3 did require quite a beefy PC because of the large object detection model.

The result worked quite well! The face recognition model required only a small number (1-3) images to already work quite reliably, and the tracking was also quite robust because of the persistent body-face tracking. Check the result to see an example of my awkward test walk.