| Titre : | AIoT-Based Assistive Technology for Real-Time Text Detection, Recognition, and Text-to-Speech Conversion for Visually Impaired Users |
| Auteurs : | Torkia SALEM, Auteur ; Laïd Kahloul, Directeur de thèse |
| Editeur : | Biskra [Algérie] : Faculté des Sciences Exactes et des Sciences de la Nature et de la Vie, Université Mohamed Khider, 2023 |
| Format : | 1 vol. (69 p.) / ill., couv. ill. en coul / 30 cm |
| Langues: | Français |
| Mots-clés: | Visually impaired, Text detection, Assistive technology, OCR, YOLOv8. |
| Résumé : |
As Artificial Intelligence and Internet of Things technology become more widely available, there is a chance to develop creative solutions that improve accessibility for the visually impaired. This study investigates the potential of Artificial Intelligence of Things (AIoT) based systems for real-time text detection, recognition, and text-to-speech conversion, allowing visually impaired users to receive textual information via audio feedback. This study intends to increase the accuracy, efficiency, and usefulness of assistive devices for the visually impaired community by utilising deep learning methodologies. To achieve this aim, we developed a complete assistive technology system that integrates powerful AI algorithms, deep learning models, and IoT devices. For accurate text detection in images, the system employs a deep learning model based on the YOLOv8 architecture, whose results demonstrated exceptional accuracy, with a mean Average Precision (mAP) of 99.3% achieved on the ICDAR2013 dataset and 82.2% on the challenging ICDAR2017-MLT dataset. The system can recognise the detected text information by integrating optical character recognition (OCR) libraries. In addition, we have used state-of-the-art text-to-speech technology to turn the recognised text into vocal output. We designed and implemented the RingReader device, a wearable device with a camera, to improve usability and mobility. The RingReader captures images and provides real-time aural feedback on detected and recognised text, helping visually impaired people more successfully read textual information. |
| Sommaire : |
Acknowledgements . . . . . . . . . . . . . . . . i Dedication . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . ii Papers Submitted . . . . . . . . . . . . . . . . . . . . . . . . iii Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv Résumé . . . . . . . . . . . . . . . . . . . . . . . v List of Figures xii List of Tables 0 1 General introduction 1 2 Background 3 2.1 Introduction . . . . . . . . . . . . . . 3 2.2 Assistive Technology . . . . . . .. . . . . 4 2.2.1 Definitions . . . . . . . . . . . . . . . . . . . . . 4 2.2.2 Assistive Technology Stages . . . . . . . . . . . . . . . . . .. 4 2.3 Artificial intelligence and Assistive Technology . . . . . . . . 5 2.4 Artificial Intelligence of Things (AIoT) . . . . . . . . . . . . . . . . . 7 2.5 Machine Learning . . . . . . . . . . . . . . . . . . . . . . 7 2.5.1 Types of Machine learning . . . . . . . . . . . . . 8 2.6 Deep learning . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . 12 2.6.1 Artificial neural networks . . . . . . .. . 12 2.7 Convolutional neural networks . . . . . . . . . . 19 2.7.1 CNN definition . . . . . . . . . . . . . . . . . . . . . . 20 2.7.2 CNN blocks . . . . . . . . . . . . . . . . . . . . . 20 2.7.3 CNN layers types according to tasks . . . .. 21 2.7.4 Object detection . . . . . . . . . . . . . . . . 2.8 Transfer learning . . . . . . . . . . . . . . . . . . 27 2.8.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . 27 2.8.2 Main benefits of Transfer learning . . . . . . . . . . . . . . . . . 27 2.9 Internet of Things . . . . . . . . . . . . . . . . . . . . . . . . 27 2.9.1 Definition of IoT . . . . . . . . . . . . . . . . . . . . . . 28 2.9.2 Six Any in IoT . . . . . . . . . . . . . . .. . . . 29 2.9.3 Features of IoT Technology . . . . . . . . . . . . . . . . . . . 30 2.9.4 Architecture of IoT Technology . . . . . . . . . . . . . . 31 2.10 Visually Impaired . . . . . . . . . . . . . . . . . . . . . . . 31 2.10.1 Challenges faced by Visually Impaired . . . . . . . 32 2.10.2 Assistive technologies and interventions . . . . . . . . . . . . . 32 2.11 Related Works . . . . . . . . . . . . . . . . . . . . . . .. . . 33 2.12 Conclusion . . . . . . . . . . . . . . 34 3 Design and implementation 35 3.1 Introduction . . . . . . . . . . . . . . . . 35 3.2 System Architecture . . . . . . . . . . . .. . . . 35 3.3 Implementation of a DL Model for Text Detection . . . . . . . . . 36 3.3.1 Data set description . . . . . . . . . . . . . . . . . . . . . . . 37 3.3.2 Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 3.3.3 YOLOv8 Model structure . . . . . . . . . . . . .. . 39 3.3.4 Training . . . . . . . . . . . . 43 3.3.5 Prediction . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.3.6 Evaluation of YOLOv8 Model . . . . . . . . . . . . . . 44 3.4 Post-processing . . . . . . . . . . . . . . . . . . . . . . . . 45 3.5 Text Recognition . . . . . . . . . . . . . . . . . . . . . . . . .. . 45 3.5.1 Text Extraction . . . . . . . . . . . . . . . . 45 3.5.2 OCR Libraries . . . . . . . . . . . . . . . . . . . . . . . . . 46 3.6 Text-To-Speech . . . . . . . . . . . . . . . . . . . . . . . . . 47 3.7 IoT Integration . . . . . . . . . . . . . . . . . . . . . . . . 47 3.8 User Interface : . . . . . . . . . . . . . . . . .. . . . 52 3.9 Frameworks, tools and libraries . . . .. . 55 3.10 Conclusion . . . . . . . . . . . . . . . . . . 57 4 Experimentation and Results 59 4.1 Introduction . . . . . . . . . . . . . . . 59 4.2 First Dataset . . . . . . . . . . . .. . . 59 4.2.1 Parameters initialization . . . . . . . 60 4.2.2 Experimentation . . . . . . . . . . . . . . . . . 60 4.2.3 Saving Model . . . . . . . . . . . . . . . . . . . . . 62 4.2.4 Results of test phase . . . . . . . . . . . . . . 62 4.3 Second Dataset . . . . . . . . . . . . . . .. . . . 63 4.3.1 Parameters initialization . . . . . . . . . 63 4.3.2 Experimentation . . . . . . . . . . . . . . . . . . . . . . . . 64 4.3.3 Saving Model . . . . . . . . . . . . . . . . 66 4.3.4 Results of test phase . . . . . . . . . . . . . . . 66 4.4 Comparison of our work with previous works . . . . . 67 4.5 Conclusion . . . . . . . . . . . . . 67 5 Achievements and Conclusions 68 5.1 Contributions . . . . . . . . . . 68 5.2 Future Directions . |
Disponibilité (1)
| Cote | Support | Localisation | Statut |
|---|---|---|---|
| MINF/814 | Mémoire master | bibliothèque sciences exactes | Consultable |
