Social Distancing Detection

Hi everyone, In this project, I going to explain the two methods for social distancing detection. Recently I had seen an SDD which was created by landing AI Company. So I plan to mimic the SDD project because in recent times I have been working on a volume estimation project using a stereo camera in that I got some knowledge of Depth and camera perspective, due to that I started the SDD the project immediately and in one day I got some awesome result, So I want to share it with you, RightNow I stopped the project. This project is work but not in all cases because I didn’t use calibration.

I going to explain the project shortly because there many good resources, So In this blog, I just interconnect all required opensource code to get the social distancing project.

Output

Note: This blog content is difficult to digest because these topics are more about visual content rather than text content. So while reading it, simultaneously view a video of the below topics whatever I covered.

In the first stage, we have to detect the person in a live feed or video. So I used detectron2 to detect the person, which is a Facebook object detection. It is very robust and also we can add some custom code easily without getting a headache.

Source from github facebookresearch:detectron2

Detectron2 Installation is already well documented, So go to the installation page and install the detectron2 and check it.

facebookresearch/detectron2

To check it

facebookresearch/detectron2

and clone the repositories of detectron2.

In the detectron2 folder, we going to modify and add some of the code in two python files.

• Prediction.py which is in Detectron2/demo/prediction.py
• video_visualizer.py

Note: If you installed the detectron2 with precompiled source code which is pip install. we have to edit the video_visualizer.py file in an installed folder(/usr/local/lib/python3.7/site-packages/detectron2/utils/video_visualizer.py note: this path will change according to your system python configuration).

optional: If you want to change in the GitHub folder (detectron2/detectron2/utils/video_visualizer.py) and then, recompile it and install the detectron2.

The second stage is we need object track to track the person moves in a video frame, fortunately, detectron2 is an instance segment object detection which means it has an inbuild object tracker and that object tracker id is assigned as unique colors. To make use of object tracker id color we have to send the colors to the prediction.py file from the video_visualizer.py.

In video_visualizer, add colors with return frame_visualizer.ouput.

In the third stage, we have to calculate the distance between the two objects or persons in a frame using a bounding box.

• the detectron2 gives all the object bounding box. But we want only a person bounding box, So using person label id (0) we filter the bounding box.
• After that, we sort the bounding box and color array, then we find the distance between each of the bounding boxes using Euclidean distance.

then, filter out the nearby people If the distance between the two people is lower than 100, then the people are nearer to each other.

Note: 100 is a threshold value, please change this value to get the proper result according to your sceneries

Measuring distance between objects in an image with OpenCV — PyImageSearch

If the camera is from the top view, Upto this we can see the proper result, because, in top view, we did not get any object size illusion, so we can measure both the vertical and horizontal distance properly.

but If the camera is a perfect side view in which the camera is near to the ground, we get the wrong distance estimation using Euclidean distance.

As we can see above example, we get the wrong distancing estimation.

Sorry for the above violent picture. I didn't get any other proper example pictures.

So If we measure the Euclidean distance between the person who is too long from the camera and a person who is near to the camera we get wrong distance estimation between the two-person, but in top-view, we won't get this problem if the surface flat for minimum threshold value otherwise, the top view can also suffer this problem.

Note: Estimation means I am not talking about real-world distances numbers like meter or centimeter. Even In top view if you want real distance value like meter you have to use the camera calibration parameter and formula to calculate. But In this problems, we do not need real distance value like meter. We just want a number which whether a person is near to another person.

but if we have a camera side view in which the camera distance between the ground is larger we get somewhat correct estimation using Euclidean distance.

To solve this problem, there are two methods:

• One is converting side view into the top view perspective using IPM(Inverse Perspective Mapping), Grid, etc or

transforming perspective view to a top view

• Find the distance between the person and the camera. If we are using a calibrated stereo camera we can get the almost real-world distance measurement like meter, inch or centimeter, etc using a different type of formula depends on what known parameter of the camera we are having.
• For example triangulation, similar triangle, etc,
• we can also use a single camera, but real-world distance measurement accuracy is less compared to the single-camera estimation.

Depth Estimation

Both have some advantage and disadvantage that too depends on the sceneries and algorithm.

But If you want real-world distance measurement to go with a stereo camera.

In this problem, we do not need real-world distance, we just need near or far estimation.

So Initially I tried with side view to the top view, But It didn’t get the proper result, because some the boundary get cropped or getting small image after the perspective change, because I did not have video camera parameter to make the proper transformation or I should read more about this topic.

Please share some ideas, if anyone has experience to convert side view to top view.

Then I used a single-camera depth estimation algorithm due to all the surveillance cameras are single camera.

Find distance from camera to object using Python and OpenCV

Measuring size of objects in an image with OpenCV - PyImageSearch

But I didn’t use the whole single-camera formula. Instead, I used the width of the bounding box to get the somewhat correct estimation by adding width to the y-axis, after that, I had some other works to do, So stopped improving the project.

If you want please improve the project and let me know.

Please download below given two files and override the file in detectron2.

Github code :

vigneshgig/Social-Distancing-Detection

Thanks, If you like this blog, please give a clap and share it.