A Deep Learning Based Spontaneous Retail Product Identification

The concept of self-service retail store has been successfully adopted around the world since its inception in 1916 United States of America. People carry their required retail products in a cart, hand-basket and make their purchase by standing in the queue to make payment. With advancements in electronics and computer sciences, this project aims to ease the “traditional purchase experience” at the same time maintaining the conventional “Hold-Observe-Purchase” experience by attempting to automate the billing procedure. Using deep learning algorithms and Image processing principles, the product images are recognized and its pertaining information is used to generate the transactional ledger as the products are added. This phase of the project aims at training our model to recognize custom images of Indian Retail products more specifically “Indian Consumable Retail products”. Furthermore, the trained model is deployed on a “Single Board Computer (SBC)” such as “Raspberry Pi 4” to recognize the product images by taking its picture with the Raspberry Pi camera attached to the SBC and running the image through the trained model to identify it and generate the transaction ledger.

Objectives: Primary objective to develop this project, was to eliminate the need of long payment queues especially in weekends in retail stores by automating the payment procedure. Implementing a complex principle of image recognition to promote “Digital payment revolution in India under atmanirbhar bharat” as predicted results are used to generate the transactional ledger. To learn and create a custom image data set of the retail products using data augmentation and image processing principles which is comprised of internet search results and photos captured by smart phone. To deploy a pre-trained image recognition algorithm (trained on Laptop computer) on a SBC such as raspberry pi attached with a camera which can capture image and can conveniently execute the pre trained model bypassing the need of advanced computation power.

Table of Contents

1.1 Deep Learning

1.2 Perceptron

1.3 Convolutional Neural Networks (CNN):

1.3.1 Convolutional Neural Network Layers:

1.4 The Tools

1.5 Creating the training data set

1.5.1 Importance of the dataset

1.6 Images

1.7 Data Augmentation

1.8 Canny Edge Detection

1.9 Data Extraction

1.10 Training the model

10.1.1 Preparing the data for training

1.11 The CNN Model

1.12 Trained model output

1.13 Output

1.14 Hardware flow diagram

1.15 Raspberry Pi

1.15.1 Raspberry pi 4 mode B 4GB

1.16 Raspberry Pi setup

1.17 Raspberry PI output

1.18 Test results merits and demerits

1.19 Conclusion & future enhancements

Objectives and Scope

This project aims to modernize the traditional retail purchase experience by automating the billing process through deep learning-based image recognition. By utilizing a Raspberry Pi and a camera, the system recognizes consumable retail products in real-time, thereby eliminating the need for long payment queues.

• Automate retail billing procedures to reduce waiting times.
• Implement image recognition to support the digital payment revolution.
• Develop a custom image dataset using advanced data augmentation and processing techniques.
• Deploy pre-trained neural network models on single-board computers (SBCs) for cost-effective, independent operation.

Excerpt from the book

1.8 Canny Edge Detection

Deep learning algorithms are specifically used in computer vision based problems, where the data is mostly in the form of image or a video or real time video recording. So naturally the features of the image are going to be the pixel values that determine the results of the prediction. Also change in pixel values alter the entire prediction.

Primary features of the image are edges, texture, shape and size, but many times different objects having similar size and shape are often not recognized by the learning algorithm and as a result they are predicted inaccurately. This is a major problem recognized for Indian retail products, the amount of information present in an image of a single product is more than it’s shape and size. In order to extract those pixel values, canny edge detection filter is used which highlight the texture of images thereby highlighting the information present in the image along with it’s shape and size.

Summary of Chapters

1.1 Deep Learning: Defines deep learning as a machine learning subfield that mimics human brain neurons to drive supervised learning models.

1.2 Perceptron: Introduces the perceptron as the fundamental algorithmic building block of deep learning models.

1.3 Convolutional Neural Networks (CNN): Explains the complex architecture of CNNs, detailing how hidden layers extract features for more accurate predictions compared to simple perceptrons.

1.4 The Tools: Outlines the software stack used, including OpenCV, Pandas, Keras, and Tensorflow.

1.5 Creating the training data set: Discusses the necessity of high-quality, abundant data for training accurate machine learning models.

1.6 Images: Displays the specific Indian consumable retail products utilized for training the model.

1.7 Data Augmentation: Describes techniques to expand dataset volume by rotating, zooming, and shifting images.

1.8 Canny Edge Detection: Details the filter used to highlight critical product textures and features to improve recognition accuracy.

1.9 Data Extraction: Illustrates the process of importing and preparing image data for inclusion in the training dataset.

1.10 Training the model: Covers data preparation steps, including labeling and normalizing data for CNN ingestion.

1.11 The CNN Model: Defines the specific architectural structure, including layer stack-up and activation functions.

1.12 Trained model output: Shows the test methodology for validating the trained model using external input.

1.13 Output: Presents the practical result of product recognition via the developed system.

1.14 Hardware flow diagram: Provides a schematic overview of the integration between the Raspberry Pi, camera, and processing unit.

1.15 Raspberry Pi: Details the hardware specifications and computational benefits of the Raspberry Pi 4 model B.

1.16 Raspberry Pi setup: Explains the physical configuration and command-line execution for the image capture hardware.

1.17 Raspberry PI output: Demonstrates the successful execution of the model on the edge device.

1.18 Test results merits and demerits: Analyzes the current limitations, such as distance constraints and color-based inaccuracies.

1.19 Conclusion & future enhancements: Reflects on the project as a functional prototype and suggests future integration with centralized retail servers.

Keywords

Deep Learning, CNN, Raspberry Pi, Image Recognition, Canny Edge Detection, Perceptron, Keras, Tensorflow, Data Augmentation, Supervised Learning, Retail Automation, Computer Vision, Consumable Products, Neural Networks, Softmax Activation

Frequently Asked Questions

What is the core purpose of this research project?

The project aims to automate the retail billing experience by creating a device capable of spontaneous product identification, thereby removing the need for manual scanning and reducing wait times.

What are the primary fields of study involved?

The work primarily integrates deep learning, computer vision, specialized image processing, and hardware deployment on edge computing devices.

What is the main research question the authors address?

The main question is how to effectively deploy a deep learning-based image recognition model for Indian consumable retail products on a power-efficient single-board computer.

Which scientific methods are applied?

The authors employ Convolutional Neural Networks (CNNs), data augmentation, Canny edge detection, and label-based supervised learning classification techniques.

What content is covered in the main body?

The main body covers the theoretical background of neurons and perceptrons, the architectural design of a CNN, the tools required for implementation, training data preparation, and finally, hardware deployment on a Raspberry Pi.

Which keywords best describe this study?

Key terms include Deep Learning, CNN, Raspberry Pi, Image Recognition, Computer Vision, and Supervised Learning.

Why is Canny edge detection used for these specific products?

It is used to highlight the texture and edges of products, which helps the model distinguish between items that may have similar shapes but different surface details.

What are the performance limitations identified by the authors?

The model struggles with adversarial color similarities (e.g., Lays green vs. Kurkure green) and is most accurate when items are placed between 15 to 17 centimeters from the camera.

How does the model handle multi-class classification?

The model utilizes a "softmax" activation function in its final fully connected layer to output probability vectors for each of the eight product categories.

What role does the Raspberry Pi play in the system architecture?

The Raspberry Pi serves as the edge computation node that captures images via its camera, runs the pre-trained model independently, and identifies products to generate a transactional ledger.

Details

• Autoren: Dr. Upesh Patel (Autor:in), Sachi Joshi (Autor:in)
• Erscheinungsjahr: 2023
• Seiten: 28
• Katalognummer: V1416457
• ISBN (eBook): 9783346967343
• ISBN (Buch): 9783346967350
• Sprache: Englisch
• Schlagworte: deep learning based spontaneous retail product identification