Signboard Optical Character Recognition Isaac Wu, Hsiao-Chen Chang Department of Electrical Engineering, Stanford University
Motivation Having the ability to recognize any store just by taking a picture of its signboard is a powerful asset for business reviews and ratings companies such as Yelp to incorporate into their mobile app.
System Pipeline A
Phase 1: Training C Manually Segmented Images
Extract SIFT Descriptors
E
B
… D
Construct K-Means Codebook
Match Descriptors to Codes
Phase 2: Segmentation
Algorithm trainDatabase() if (SIFT with MSER has many matches) return result elseif (SIFT with Morphology has many matches) return result elseif (OCR with MSER seems valid) return result elseif (OCR with Morphology seems valid) return result else return null
Results
MSER
Grayscale
Morphology
Grayscale
* Multi-scale approach
Success Rate: 86% # Testing Images: 113 # Correctly Determined: 97
Techniques Used 7.2%
3.1%
Detect MSER Regions
Increase Contrast
Small Region Removal
Phase 3: Recognition
A
MSER x SIFT
C
E
MORPH x SIFT SIFT
MSER x OCR 89.7%
Extract SIFT Descriptors
Remove NonText Regions
Adaptive Thresholding
Region Labeling
B
Create Bounding Boxes of Each Region
Merge Boxes and Keep the Longest
Morphological Opening
Remove NonText Regions
Create Bounding Box
… D
Match Descriptors to Codes
Top 5 Database Matches
Perform SIFT Match with RANSAC
Return the Most Matches
MCDONALDS MORPH xOCR
OCR
Restrict OCR Matching to English Letters
Perform OCR
Remove short length words
Remove Spaces
