Vehicle-Number-Plate-Detector

Vehicle Number Plate Detection and Recognition Techniques

 

1. Introduction Vehicle Number Plate Recognition (NPR) or License Plate Recognition (LPR) or Registration Plate Recognition (RPR) is an enhanced computer vision technology that connects vehicles without direct human connection through their number plates [1- 3]. Day by day, the number of vehicles on the road is continuing to grow. For this reason, the news spread almost every day about the vehicle being filched from the parking garage or any other place in the city or having an accident and fleeing. To recognize these vehicles [4, 5], authorities should therefore install a number plate detection and recognition device on CCTV at every street corner in every region. This system enhances the police’s ability to track illegal activities involving the use of vehicles. NPR systems are effectively used by provincial establishments and manufacturing groups in all facets of safety, inspection, traffic management applications [6, 7]. The number plates vary from country to country. There are some rules and regulations for vehicle number plates. Number plate consists of (1) 2 letters (these refer to the region in the country where the vehicle was first registered) (2) 2 numbers (when it was issued) (3) 3 letters chosen at random. Some basic information about vehicle number plates like dimension, styles, and characters of number plates fitted after 1st September 2001 is shown in Figure 1. Some variations are often seen on the vehicle number plates. The difference between American and European number plates is that American vehicle number plates have more things than identification numbers, sometimes little pictures, different color text but in European vehicle plates are used just for identification. Front number plates must show black characters on a white background and the rear number plate must have black letters on a yellow reflective background [8]. The number plates dimension of the car and motorcycle in the UK (United Kingdom) is shown in the form of Table 1. For nearly half a century, vehicle number plate detection, as well as recognition, has been a topic of interest. This technique in the field has opened new challenges. In terms of consistency, color, number plate shape, and type of vehicle, the major challenges of vehicle number plate detection as well as recognition are focused on the various categories of features and are related to changing illumination level, the geometry of visualization, and background [9,10]. In Figure 2, typical samples of vehicle number plates [11] are shown. Number plate recognition procedure is divided into three key functions: Identification of Plate Area, Segmentation of Plate Character, and Recognition of Character [12-16]. In terms of traffic management, traffic optimization, traffic law enforcement, vehicle access control, automated collection of tolls, traffic speed control, automatic parking, monitoring of stolen cars, and tracking of possible acts of terrorism, each of these aspects plays a crucial role [6, 7, 14, 17, 18]. Figure 3 shows common vehicle number plate detection and recognition method based on the edge detection method. At first, the vehicle registration plate detection as well as the recognition system capture the image using the camera and then apply some image processing techniques for pre-processing the image such as input image to grayscale image conversion, filtering technique to eliminate noise. Next, to extract the license plate area, apply the canny edge detection technique. After that, apply the appropriate detection method to detect the vehicle registration plate effectively, and apply the segmentation technique to segment the characters of the registration plate. Finally, the appropriate character recognition method is used to recognize each of the characters separately. Due to the lighting conditions, the noisy image captured, fastmoving vehicles, are always a difficult task in vehicle number plate identification as well as recognition. Several researchers have been working on vehicle number plate recognition and are still working in this field. They have adopted several image processing techniques and presented some of their development strategies for vehicle number plate detection. As much research has been done so far in this paper on vehicle number plate detection as well as recognition and their success behind their proposed method and exactly what caused their proposed method to fail is discussed here. And this paper explores how to resolve their limitations or what more can be achieved in this area in the future. Vehicle number plate detection studies, as well as recognition techniques, have been categorized into three sections in this review paper: (1) Related Works on Vehicle Number Plate Detection Techniques (2) Related Works on Vehicle Number Plate Recognition Techniques (3) Related Works on Vehicle Number Plate Detection as well as Recognition Techniques. The residual of the paper is arranged in a structured way. The number plate detection strategies are demonstrated in section 2. Techniques for number plate recognition are discussed in section 3. In section 4, techniques of vehicle number plate detection, as well as recognition, are illustrated. Finally, section 5 states the conclusions. 2. Related Works on Vehicle Number Plate Detection Techniques Number plate detection (NPD) is a technology that uses certain image features to understand vehicle registration plates to assess location data for vehicles [14,19]. To determine a location going to the next frame, NPD identifies a region of the vehicle number plate with similar structures. The consecutive frame fixes the area of detection in the prior frames with the observed area of the vehicle [20]. During the identification of the registration plate of the vehicle, various difficulties of the surrounding environment were observed. In addition to these, several vehicle number plate considerations are concise in Table 2. Input Image Convert Gray Scale Filtering Canny Edge Detection Segment the Characters Number Plate Region Detection Pre-Processing Figure 3: Flow diagram of common number plate detection and recognition method Character Recognition K P C 1 3 1 3 ………… 

Different researchers have talked about their proposed techniques for identifying vehicle number plates at different times and still a lot of work is being done following their proposed method. Many image processing techniques are existing to detect vehicle number plates such as segmentation, edge detection, color code-based techniques, feature-based techniques, and machine learning techniques. This section discusses different strategies associated with the identification of vehicle number plates. Centered on various methods, the following section is split into several sub-sections.

2.1. Edge Detection In image processing, it is possible to recognize the edges of the image through different edge detection techniques, such as Sobel, Prewitt, Laplacian, and Canny edge band detectors. The Sobel edge detector effectively transforms a compact, detachable, and numeral valued filter to the image in a horizontal and vertical direction. Prewitt is used in frames to detect vertical and horizontal edges. Hence Sobel and Prewitt are kind of similar. Canny edge detector probably the most effective method for complex edge detection. Below, discussed the previous literature of vehicle number plate detection techniques based on edge detection. In [22], an algorithm has been suggested for vehicle number plate detection in practical situations by Wazalwar. To define the region of interest (ROI), they used the Euler number of a binary image and for edge detection, they used the Mexican hat operator. They have claimed that a license plate had been successfully identified through their suggested technique and their success rate was about 94-99% and the average accuracy was about 96.17%. Yet there is a situation during their prosperity where they have suffered. The edge detection system fails to properly recognize the edge if the license plate is black. In [23], a license plate detection system founded on an enhanced Prewitt arithmetic operator has been suggested by Chen under various backgrounds and lighting conditions. The projection method was also carried out horizontally and vertically to change the top and bottom edge areas along the edge to get the vehicle number position. They have achieved 96.75% precision in their proposed technique, and they have stated that their proposed system meets efficiency in real-time. An innovative technique for vehicle number plate detection using the special technique of edge detection [24] has been introduced by Tejas. They have used the Sobel edge detection technique to obtain accurate boundaries of the number plate in the image. The system scanned the connected component and then fill them with holes. Thereafter, the system searches the rectangular region that is filled with holes which is probably the size of the license plate and then extracts it. Their proposed system is based on the Internet of Things (IoT). Therefore, online databases have been developed and regularly updated. They have also estimated that the accuracy of their acquisition is around 96.842%. In Figure 4, their suggested technique is shown. 2.2. Morphological Operation Morphological Operations in image processing attempts to remove these imperfections by considering the image’s shape and structure. To reduce noise or to brighten the frame, morphological operations are essentially applied to grayscale images. Morphological operations are referred to as a blend of erosion, dilation, and basic set-theoretical functions, such as a binary image supplement [25]. The corresponding study on morphological operation-based vehicle number plate detection techniques has been discussed below. In [26], an existing system used for license plate location on a Raspberry Pi has been improved by the Yepez. Their improved morphological algorithm that reduces computational complexity is based on morphological operations. The strength of this strategy is that the emerging LPR algorithm can operate with the computer as well as low processing power on portable devices. They have also claimed that their enhanced algorithm can detect license plates effectively and have achieved a high precision is about 98.45%. In Figure 5, the flowchart of their proposed method is shown. 2.3. Convolutional Neural Networks (CNNs) A convolutional neural network (CNN) based framework for the detection of vehicle number plate was proposed by the authors in [27]. They have enhanced the existing blurred and obscure image method. They believed that their suggested method effectively detects the number plate of the vehicle under various lighting conditions. The accuracy obtained by their proposed method is around 100%. 2.4. Machine Learning (ML) based Approaches Machine Learning (ML) likewise means that by providing a collection of training data, the machine is trained to do something in image processing. Machine learning has models/architectures, functions of loss, and many methods that can be used to decide which will provide better processing of images. For image enhancement, this approach is commonly applied. The corresponding work of machine learning-based vehicle number plate detection techniques is given below. In [5], a new technique to detect a vehicle authorization plate has been developed in the Miyata study. The license plate detection technique detects only the edge vertical parts and the candidate license plates that use the contours acquired by dilation and erosion processing and area fill processing. The SVM (Support Vector Machine) has applied to decide whether a license plate is a candidate region or not, and eventually recognizes the location of the license plate. They have claimed that the suggested method efficiently detects license plates and achieved the rate of detection is 90%.

In [28], an innovative method has been presented for detecting and locating a vehicle’s license plate in color images by Yaseen. AdaBoost, a multi-boosting model based on HOG features, is part of the development process. They have claimed that the accuracy achieved by their proposed method is around 89.66%. Figure 6 displays the flow chart of their suggested system. In [29], a vehicle registration plate detection method in the natural image by AdaBoost using the Modified Census Transform (MCT) feature has been suggested by Ban. There are too many noises in the natural image, so detecting the number plate in natural images is too difficult. MCT features, which are robust to illumination change, and AdaBoost for the feature selection to overcome this restriction. They have also reported that the high detection rate achieved by the proposed technique is about 98.7%. In [30], a systematic style for vehicle registration plate detection applying boosting and part-based models was proposed by Molina-Moreno, which is an algorithm for boosting. They used two datasets and stated that better performance on these datasets was obtained 97.52% accuracy by their proposed method. With several algorithms, they have also compared their proposed method. In [31], a novel vehicle number plate detection system has been suggested to improve identification in low lights and over corrosive environments by Babbar. For the extraction of license plates, they used CCA (Connected Component Analysis) and Ratio Analysis (RA). Some OCR strategies have also been used, for example, LR+RF, SVC+KNN, Extra Trees, SVC (Linear, Poly, Rbf, Linear.svc). They stated that the car localization achieved by the developed system is 92.7% and the segmented characters’ accuracy is about 97.1%. In [32], a KNN (K-nearest Neighbor) machine learning system for automatic vehicle license plate detection was developed by Akshay lepcha. The KNN classifier has been used according to the aspects of the license to retrieve the registration plate from the image. They have also stated that a license plate is correctly identified through their suggested method and achieved an accuracy higher than 90%. Table 3 provides an overview of the strategies for detecting the vehicle number plate. This table has been sorted based on the year and accuracy. 3. Related Works on Vehicle Number Plate Recognition Techniques Vehicle Automatic Number Plate Recognition (ANPR) is a technology applied for the observation as well as recognition of vehicle number plate characters from static and moving vehicle images [14, 28, 32, 33]. Due to its effect on the rapid development of traffic monitoring and surveillance [15, 22, 34, 35], vehicle number plate recognition has become a key research field in recent years. For the identification of number plates, several methods are used, such as machine learning, neural networks, BAM (Bidirectional Associative Memories) [35]. Various researchers have given their useful ideas on their proposed vehicle number plate recognition method at various times. In this review paper, various vehicle number plate recognition techniques have been explored. Vehicle number plate recognition techniques have been categorized into some subsections based on distinct approaches in the following section. 3.1. Neural Network (NN) based Approaches Image recognition algorithms in neural networks (NN) can recognize anything, from text to images, audio files, and videos. Neural networks are an interlinked set of neurons or perceptron’s called nodes. Each node uses a single input data, generally a single pixel of the image, and uses a simple calculation called an activation function which produces results and each neuron has a numerical score that determines its outcome I. Artificial Neural Networks (ANNs) In [15], a high-performance-based system for vehicle number plate recognition has been introduced by Türkyılmaz. They have applied edge-based image processing techniques for registration plate detection and have also used a three-layer feedforward artificial neural network for vehicle number plate character recognition using a learning algorithm for back-propagation. The feedforward ANN model for three layers is shown in Figure 7. The input layer receives information from the external environment and transmits it to the nodes (processing units) of the hidden layer without any modification. Network outputs are calculated by processing information in hidden layers and output layers. The most well-known back-propagation learning algorithms are used efficiently at the training stage of this ANN. The authors have verified that the number plate has been successfully identified and recognized by their developed system and their performance rate

https://github.com/raihandevelopers/Vechicle-Number-Plate-Detector.git

import cv2
	import pytesseract
	# Read the image file
	image = cv2.imread('car0.JPG')
	# Convert to Grayscale Image
	gray_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
	#Canny Edge Detection
	canny_edge = cv2.Canny(gray_image, 170, 200)
	# Find contours based on Edges
	contours, new = cv2.findContours(canny_edge.copy(), cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
	contours=sorted(contours, key = cv2.contourArea, reverse = True)[:30]
	# Initialize license Plate contour and x,y coordinates
	contour_with_license_plate = None
	license_plate = None
	x = None
	y = None
	w = None
	h = None
	# Find the contour with 4 potential corners and creat ROI around it
	for contour in contours:
	# Find Perimeter of contour and it should be a closed contour
	perimeter = cv2.arcLength(contour, True)
	approx = cv2.approxPolyDP(contour, 0.01 * perimeter, True)
	if len(approx) == 4: #see whether it is a Rect
	contour_with_license_plate = approx
	x, y, w, h = cv2.boundingRect(contour)
	license_plate = gray_image[y:y + h, x:x + w]
	break
	# Removing Noise from the detected image, before sending to Tesseract
	license_plate = cv2.bilateralFilter(license_plate, 11, 17, 17)
	(thresh, license_plate) = cv2.threshold(license_plate, 150, 180, cv2.THRESH_BINARY)
	#Text Recognition
	text = pytesseract.image_to_string(license_plate)
	#Draw License Plate and write the Text
	image = cv2.rectangle(image, (x,y), (x+w,y+h), (0,0,255), 3)
	image = cv2.putText(image, text, (x-100,y-50), cv2.FONT_HERSHEY_SIMPLEX, 3, (0,255,0), 6, cv2.LINE_AA)
	print("License Plate :", text)
	cv2.imshow("License Plate Detection",image)
	cv2.waitKey(0) Links - https://github.com/raihandevelopers/Vechicle-Number-Plate-Detector.git