Indoor sign recognition for the blind

Dumisani Kunene; Hima Vadapalli; Jaco Cronje

doi:10.1145/2987491.2987530

Indoor sign recognition for the blind

Dumisani Kunene, Hima Vadapalli, Jaco Cronje

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

4 Citations (Scopus)

Abstract

Blind people face diculties when navigating unfamiliar environments. The information displayed on indoor signs and notice boards is of no use to them. In order to assist them with this challenge, we propose a real time system that can recognise a selection of indoor navigational signs placed over clear backgrounds. The selection of signs will consist of common samples from several di erent types of indoor signs. Given a captured image, the approach is to use image processing techniques to find the region of interest(ROI) that contains the sign and then extract this region for classification. Using sliding windows for searching the ROI can be time consuming and can lead to many false classifications, hence we used a more explicit approach that is faster and more reliable. We first segment the signs by colour, and then by shape recognition. The sign-type classification is done using a tree search structure that enables the use of iterative contour descriptors like the speeded-up-robustfeatures( SURF). Once a sign has been detected, this information is communicated to the user via stereo headsets. To evaluate the system's performance, several random pictures with and without signs were used to determine the system's detection rate. The user-feedback performance was evaluated by testing the system's usability score with volunteers.

Original language	English
Title of host publication	SAICSIT 2016 - Proceedings of the 2016 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists
Editors	Dustin T van der Haar, Duncan A. Coulter, Marijke Coetzee, Elize M. Ehlers, Carl Marnewick, Frans F. Blauw, Wai Sze Leung
Publisher	Association for Computing Machinery
ISBN (Electronic)	9781450348058
DOIs	https://doi.org/10.1145/2987491.2987530
Publication status	Published - 26 Sept 2016
Externally published	Yes
Event	2016 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists, SAICSIT 2016 - Johannesburg, South Africa Duration: 26 Sept 2016 → 28 Sept 2016

Publication series

Name	ACM International Conference Proceeding Series
Volume	26-28-September-2016

Conference

Conference	2016 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists, SAICSIT 2016
Country/Territory	South Africa
City	Johannesburg
Period	26/09/16 → 28/09/16

Keywords

Colour-segmentation
Computer-vision
Shape-detection
Sign-recognition
Visual aid system

ASJC Scopus subject areas

Software
Human-Computer Interaction
Computer Vision and Pattern Recognition
Computer Networks and Communications

Access to Document

10.1145/2987491.2987530

Cite this

Kunene, D., Vadapalli, H., & Cronje, J. (2016). Indoor sign recognition for the blind. In D. T. van der Haar, D. A. Coulter, M. Coetzee, E. M. Ehlers, C. Marnewick, F. F. Blauw, & W. S. Leung (Eds.), SAICSIT 2016 - Proceedings of the 2016 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists (ACM International Conference Proceeding Series; Vol. 26-28-September-2016). Association for Computing Machinery. https://doi.org/10.1145/2987491.2987530

Kunene, Dumisani ; Vadapalli, Hima ; Cronje, Jaco. / Indoor sign recognition for the blind. SAICSIT 2016 - Proceedings of the 2016 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists. editor / Dustin T van der Haar ; Duncan A. Coulter ; Marijke Coetzee ; Elize M. Ehlers ; Carl Marnewick ; Frans F. Blauw ; Wai Sze Leung. Association for Computing Machinery, 2016. (ACM International Conference Proceeding Series).

@inproceedings{e2ffa0cac03b461d879b49e8dc490c87,

title = "Indoor sign recognition for the blind",

abstract = "Blind people face diculties when navigating unfamiliar environments. The information displayed on indoor signs and notice boards is of no use to them. In order to assist them with this challenge, we propose a real time system that can recognise a selection of indoor navigational signs placed over clear backgrounds. The selection of signs will consist of common samples from several di erent types of indoor signs. Given a captured image, the approach is to use image processing techniques to find the region of interest(ROI) that contains the sign and then extract this region for classification. Using sliding windows for searching the ROI can be time consuming and can lead to many false classifications, hence we used a more explicit approach that is faster and more reliable. We first segment the signs by colour, and then by shape recognition. The sign-type classification is done using a tree search structure that enables the use of iterative contour descriptors like the speeded-up-robustfeatures( SURF). Once a sign has been detected, this information is communicated to the user via stereo headsets. To evaluate the system's performance, several random pictures with and without signs were used to determine the system's detection rate. The user-feedback performance was evaluated by testing the system's usability score with volunteers.",

keywords = "Colour-segmentation, Computer-vision, Shape-detection, Sign-recognition, Visual aid system",

author = "Dumisani Kunene and Hima Vadapalli and Jaco Cronje",

note = "Publisher Copyright: {\textcopyright}2016 ACM.; 2016 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists, SAICSIT 2016 ; Conference date: 26-09-2016 Through 28-09-2016",

year = "2016",

month = sep,

day = "26",

doi = "10.1145/2987491.2987530",

language = "English",

series = "ACM International Conference Proceeding Series",

publisher = "Association for Computing Machinery",

editor = "{van der Haar}, {Dustin T} and Coulter, {Duncan A.} and Marijke Coetzee and Ehlers, {Elize M.} and Carl Marnewick and Blauw, {Frans F.} and Leung, {Wai Sze}",

booktitle = "SAICSIT 2016 - Proceedings of the 2016 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists",

}

Kunene, D, Vadapalli, H & Cronje, J 2016, Indoor sign recognition for the blind. in DT van der Haar, DA Coulter, M Coetzee, EM Ehlers, C Marnewick, FF Blauw & WS Leung (eds), SAICSIT 2016 - Proceedings of the 2016 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists. ACM International Conference Proceeding Series, vol. 26-28-September-2016, Association for Computing Machinery, 2016 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists, SAICSIT 2016, Johannesburg, South Africa, 26/09/16. https://doi.org/10.1145/2987491.2987530

Indoor sign recognition for the blind. / Kunene, Dumisani; Vadapalli, Hima; Cronje, Jaco.
SAICSIT 2016 - Proceedings of the 2016 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists. ed. / Dustin T van der Haar; Duncan A. Coulter; Marijke Coetzee; Elize M. Ehlers; Carl Marnewick; Frans F. Blauw; Wai Sze Leung. Association for Computing Machinery, 2016. (ACM International Conference Proceeding Series; Vol. 26-28-September-2016).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Indoor sign recognition for the blind

AU - Kunene, Dumisani

AU - Vadapalli, Hima

AU - Cronje, Jaco

PY - 2016/9/26

Y1 - 2016/9/26

N2 - Blind people face diculties when navigating unfamiliar environments. The information displayed on indoor signs and notice boards is of no use to them. In order to assist them with this challenge, we propose a real time system that can recognise a selection of indoor navigational signs placed over clear backgrounds. The selection of signs will consist of common samples from several di erent types of indoor signs. Given a captured image, the approach is to use image processing techniques to find the region of interest(ROI) that contains the sign and then extract this region for classification. Using sliding windows for searching the ROI can be time consuming and can lead to many false classifications, hence we used a more explicit approach that is faster and more reliable. We first segment the signs by colour, and then by shape recognition. The sign-type classification is done using a tree search structure that enables the use of iterative contour descriptors like the speeded-up-robustfeatures( SURF). Once a sign has been detected, this information is communicated to the user via stereo headsets. To evaluate the system's performance, several random pictures with and without signs were used to determine the system's detection rate. The user-feedback performance was evaluated by testing the system's usability score with volunteers.

AB - Blind people face diculties when navigating unfamiliar environments. The information displayed on indoor signs and notice boards is of no use to them. In order to assist them with this challenge, we propose a real time system that can recognise a selection of indoor navigational signs placed over clear backgrounds. The selection of signs will consist of common samples from several di erent types of indoor signs. Given a captured image, the approach is to use image processing techniques to find the region of interest(ROI) that contains the sign and then extract this region for classification. Using sliding windows for searching the ROI can be time consuming and can lead to many false classifications, hence we used a more explicit approach that is faster and more reliable. We first segment the signs by colour, and then by shape recognition. The sign-type classification is done using a tree search structure that enables the use of iterative contour descriptors like the speeded-up-robustfeatures( SURF). Once a sign has been detected, this information is communicated to the user via stereo headsets. To evaluate the system's performance, several random pictures with and without signs were used to determine the system's detection rate. The user-feedback performance was evaluated by testing the system's usability score with volunteers.

KW - Colour-segmentation

KW - Computer-vision

KW - Shape-detection

KW - Sign-recognition

KW - Visual aid system

UR - http://www.scopus.com/inward/record.url?scp=84994351211&partnerID=8YFLogxK

U2 - 10.1145/2987491.2987530

DO - 10.1145/2987491.2987530

M3 - Conference contribution

AN - SCOPUS:84994351211

T3 - ACM International Conference Proceeding Series

BT - SAICSIT 2016 - Proceedings of the 2016 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists

A2 - van der Haar, Dustin T

A2 - Coulter, Duncan A.

A2 - Coetzee, Marijke

A2 - Ehlers, Elize M.

A2 - Marnewick, Carl

A2 - Blauw, Frans F.

A2 - Leung, Wai Sze

PB - Association for Computing Machinery

T2 - 2016 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists, SAICSIT 2016

Y2 - 26 September 2016 through 28 September 2016

ER -

Kunene D, Vadapalli H, Cronje J. Indoor sign recognition for the blind. In van der Haar DT, Coulter DA, Coetzee M, Ehlers EM, Marnewick C, Blauw FF, Leung WS, editors, SAICSIT 2016 - Proceedings of the 2016 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists. Association for Computing Machinery. 2016. (ACM International Conference Proceeding Series). doi: 10.1145/2987491.2987530

Indoor sign recognition for the blind

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this