Automatic License Plate Recognition
(Nvidia TAO + Deepstream Application + Python Bindings)
Mobia Latin America (Start-up)
- Oscar Guarnizo Mobia Latam
- Leduin Cuenca Mobia Latam
- Diego Suntaxi Digevo
Description
This project contains an entire process for developing a multistream Automatic License Plate Recognition (ALPR) system
using mainly Nvidia TAO Toolkit and Nvidia Deepstream SDK using the Python Bindings. The computer vision pipeline
consists of 3+ deep learning models and plugins to treat and periodically send metadata streams to a Kafka Apache
cluster for further use-cases.
Skills:
Python, C/C++, Nvidia Deepstream, Nvidia Tao Toolkit, TensorFlow, and Kafka Apache (Connectivity Skills).
Video Streaming Pipeline
The pipeline was developed using Nvidia Deepstream and allows us to perform the following actions:
- Accept multi-sources inputs of two types: videos (in mp4) and video streaming.
- Contains 4 deep learning models (trained with Nvidia TAO Toolkit) working in a cascade style.
- Accept two sink outputs (at the same time): video MP4 sink and streaming RTSP sink.
- Send message payload to a topic in Kafka broker by using two additional plugins (nvdsmsgconv , nvdsmsgbroker), which can be customized.
Cascade Style
The cascade style means the output detection of a previous model is used as input to the next model. Particularly in this project we use the following models and cascade style:
- tcnet detects cars.
- lpdnet detects plates on the previously detected car.
- lprnet recognizes characters on the previously detected plate.
- colornet recognizes the car color on the previous car detection (from tcnet).
Car and Plate Detection
The primary model is called TrafficCamNet (tcnet), which helps us to detect cars. Then, the bounding box detections are accepted as input to a LicensePlateDetectionNet (lpdnet).
Character Recognition
After getting the plate detection, we will use them as input to LicensePlateRecognition (lprnet), which returns the characters in the plate.