A project led by Dr. Nguyễn Ngọc Thảo focused on developing a system to recognise drug names and prescription information using Convolutional Neural Networks (CNNs). The project aimed to improve accuracy and processing speed in extracting data from prescriptions, addressing limitations of existing methods, and successfully achieved notable results, including higher accuracy and faster processing times.

On 15th February, the VNUHCM Scientific and Technological Project Evaluation Council held a meeting to assess and evaluate the project titled “Research and Development of a System for Recognising Drug Names and Prescription Information from Printed Documents Using Convolutional Neural Networks” at VNUHCM-University of Science. The project was led by Dr. Nguyễn Ngọc Thảo, project Leader and a lecturer at the Faculty of Information Technology, alongside Dr. Lê Ngọc Thành (team member).

In the healthcare sector, recognising and extracting information from prescriptions is a critical requirement for effective medical data management and ensuring patient safety. The accurate extraction of prescription details is essential to reduce errors in treatment and improve the overall quality of healthcare. However, current methods for prescription recognition still face significant limitations. These systems often struggle with issues such as low accuracy and slow processing speeds, particularly when dealing with prescriptions captured on smartphones. Images taken in this way are frequently skewed, poorly aligned, or have inconsistent resolution, which significantly hampers the system’s ability to process the data effectively.

These challenges have led to a growing need for more sophisticated approaches that can not only handle such variations in image quality but also provide faster and more reliable results. As the demand for efficient and accurate medical data management increases, it has become clear that the current solutions are insufficient for meeting these practical healthcare needs. Therefore, developing a system that can overcome these limitations is crucial to improving the efficiency and safety of prescription data management in medical environments.

Research Content and Methodology

The primary aim of the project was to develop a system capable of recognising and extracting information from prescriptions to facilitate the management of medical data and ensure patient safety. By improving the accuracy of this process, the system aimed to minimise treatment errors and assist both pharmacies and hospitals in storing and utilising drug information more effectively. The ability to accurately extract prescription details from various sources would significantly enhance the efficiency of healthcare systems and contribute to better patient outcomes.

Previous research in this area had explored the use of traditional optical character recognition (OCR) models, as well as some deep learning approaches such as Convolutional Neural Networks (CNNs), to recognise and interpret text on prescriptions. While these models showed promise, they faced significant challenges when processing low-quality images, misaligned or skewed text, and inconsistent resolutions. As a result, the performance of these models was often hindered, leading to lower accuracy rates and slower processing speeds. Recognising these limitations, the research team proposed a novel solution by combining Temporal Convolutional Networks (TCN) with heuristic rules to improve both the accuracy and efficiency of the system.

The newly developed system integrated a number of advanced technologies to address these challenges. These included CRAFT (Character Region Awareness for Text detection), which was used to detect text regions in prescription images; VietOCR, a tool for text recognition; and regular expressions combined with post-processing OCR techniques for extracting drug names. Additionally, a TCN model was employed to classify and validate the extracted drug information, enhancing the overall reliability of the system. By incorporating these state-of-the-art technologies, the team aimed to overcome the limitations of traditional methods and provide a more robust and accurate solution for the recognition of prescription data.

This multi-faceted approach demonstrated a significant improvement over previous models, not only in terms of speed and accuracy but also in its ability to handle challenging prescription images, making it a practical tool for real-world healthcare applications. Through this research, the team was able to offer a substantial advancement in the field of medical data management, contributing to both the scientific community and the healthcare industry.

Research Results and Applications

The MEP system demonstrated a significant improvement with an accuracy rate of 0.94, surpassing previous methods. The average processing time per prescription was reduced to just 6.67 seconds, a notable enhancement compared to the previous model’s 17.81 seconds. This improvement was achieved through a combination of advanced techniques that allowed for better text recognition, particularly in the case of prescription images that were of poor quality or taken under suboptimal conditions, such as skewed angles or inconsistent resolution.

These enhancements not only contributed to the higher accuracy in recognising and extracting drug names but also addressed the practical demands of the healthcare industry. The system’s ability to process prescriptions more quickly and efficiently ensures that it is well-suited for real-world applications, where time and accuracy are crucial in managing medical information and ensuring patient safety. The improvements also make the system a more viable tool for pharmacies and hospitals in maintaining accurate drug records and reducing treatment errors.

In addition to these impressive research outcomes, the team achieved several significant milestones. They successfully published a paper at a B-tier conference, as recognised by CORE2023, further cementing the project’s academic contribution. The team also had the opportunity to mentor and supervise six students who defended their final-year project internships, five of whom went on to graduate. These achievements reflect not only the technical success of the project but also the impact it has had on fostering the next generation of researchers and professionals in the field.

Minh Tâm _ Translated by ℙ𝕄ℕ