Embedded Real-Time Wide Area Vehicle Detection and Tracking System for Traffic Safety & Monitoring Using Deep Learning

An application of a fish-eye camera on embedded real-time vehicle detection and tracking tasks has risen in recent years due to its wide coverage area compared to a traditional camera. However, there is a lack of a) sufficient fish-eye camera dataset and b) an efficient and accurate vehicle detection model that can detect small vehicles (<16 pixels in extent) for an embedded fish-eye camera-based real-time traffic monitoring system. The existing state-of-the-art (SoTA) object detection models, such as YOLOv3, RetinaNet, RefineDet, achieve high accuracy for middle to large-sized objects (>32 pixels in extent) on PC with Graphical Processing Unit. Thus, these SoTA models are not suitable for embedded real-time systems that have insufficient computational power. To fill the gap, we propose to a) collect fish-eye videos and create UAE featured fish-eye traffic dataset, and b) Recursively Fused Feature Networks (RFFN) for small vehicle detection using CNNs. RFFN consists of non-feature deteriorating operations unlike the state-of-the-art models, which causes small objects’ features disappear at the end of the feature extraction.

Figure 1. The proposed system framework.

Based on the dataset and proposed RFFN we will be able to implement an efficient and accurate embedded real-time vehicle detection and tracking system on an affordable deep learning module Jetson TX2, that can be used for traffic flow estimation and improvement of traffic safety within UAE. Expected results would be: 1 patent, 2 international conference papers and 1-2 international SCI-indexed journal papers, open UAE fish-eye traffic dataset which will be a big contribution in computer vision and deep learning research field. Moreover, vehicle tracking will be done by employing DeepSORT model, then we can estimate traffic flow, accidents, violation. Furthermore, we can predict pending accidents of vehicles and pedestrians crossing the road and can prevent by controlling the traffic signal or alarming the pedestrians.

Figure 2. Proposed deep learning object detection model RFFN.

Figure 3: Final feature pyramid generating flowchart for Scale n consisting of CORE block and FE. However, for the Scale0, feature pyramid scale is generated with SPP and FE