Car_License_Plate_Recognition(Realtime)

Car License Plate numbers and characters Recognition by YOLOv3 and Pytesseract

Recognize the numbers and characters of license plates based on Hong Kong.

Support Hong Kong license plate (one&two-line).

Enviroment and package

• Python 3.6.8
• Labelimg
• Python virtual environments (venv)
• OpenCV
• pytesseract
• matplotlib
• flask

Project idea

1. Use the YOLOv3 custom model to recognize the position of the car plate.
2. Use OpenCV to process the position of the car plate. Based on Hong Kong car plates there have two types so need concern.
3. Dependence the position to do the OCR process by using the pytesseract to recognize those car codes and characters.

Annotating Images using LabelImg

Collect the car license plate data set of more than 150 images of each class(one & two line car plates). I found the Hong Kong license plate from Google, Flickr which is free.

The labeling IMG software will save a file within the classes containing the boxes for each class. to one of its image classes to save the list class.txt

In yolov3, the values are fed into the system which has a specified format and the data is in txt format which contains the classes and some values. These values look like below. The order of yolo format txt files follows class, x, y, w, h

x = Absolute x / width of total image

y = Absolute y / height of total image

w = Absolute width / width of total image

h = Absolute height / height of total image

Model training

Step 1, in a new colab notebook go to Runtime Change runtime type, and select GPU

Step 2 Mount Google Drive, In Google Drive, create a backup folder. I’ve named mine yolo-license-plates. That’s where model weights and configuration will get stored.

In the first cell, execute the following code to mount Google Drive:

from google.colab import drive 
drive.mount('/content/gdrive')

!ln -s /content/gdrive/My\ Drive/ /mydrive

Step 3 Download and configure Darknet,Darknet is an open-source neural network framework that features a YOLO object detection system. To download it, execute this line from a code cell:

!git clone https://github.com/AlexeyAB/darknet

Step 4 Configure settings files,To know how to set up the YOLO configuration file, you need to know how many classes there are. Next, you need to calculate the number of batches and the number of filters. Here are the formulas:

Batches = number of classes * 2000

Filters = (number of classes + 5) * 3

Custom YOLOv3 Model Example

In the source code, use the cv.dnn.readNet to read the custom model file and use detection program to find the license plate detected position(x,y,w,h) and use cv.VideoCapture(0)open the system camera to do the object tracking.

The crop pure car code use self.img[y:y + h, x:x + w] Then use openCV ser the image from RGB to graygray = cv2.cvtColor(roi, cv2.COLOR_BGR2GRAY) gray = cv2.threshold(gray, 230, 255,cv2.THRESH_BINARY | cv2.THRESH_OTSU)[1] kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (10,10)) thresh = cv2.morphologyEx(gray, cv2.MORPH_CLOSE, kernel)

For the twoline car plate the pytesseract can't detect two row characters so that need to use cut at half use Getting the height and wegith half = h//2&half2 = w//2 img[:,half]

Second, use the pytesseract to do the OCR. Set the config that means here is the pytesseract OCR white list for each characters.custom_config = r'-c tessedit_char_whitelist=ABCDEFGHIJKLMNPQRSTUVWXYZ1234567890 --psm 7' text = pytesseract.image_to_string(top, config=custom_config) text2 = pytesseract.image_to_string(bottom, config=custom_config)

Lastly use flask to build the front-end web page

Reference

https://towardsdatascience.com/how-to-detect-license-plates-with-python-and-yolo-8842aa6d25f7