Skip to main navigation menu Skip to main content Skip to site footer

OLIV: An Artificial Intelligence-Powered Assistant for Object Localization for Impaired Vision


This paper introduces OLIV, a novel end-to-end artificial intelligence-
powered assistant system designed to aid individuals with impaired
vision in their day-to-day tasks in locating displaced objects. To
achieve this goal, OLIV leverages the current advances in AI-based
speech recognition, speech generation, and object detection to un-
derstand the user’s request and give directions to the relative loca-
tion of the displaced object. OLIV consists of three main modules:
i) a speech module, ii) an object detection module, and iii) a logic
unit module. The speech module interfaces with the user to inter-
pret the verbal query of the user and verbally responds to the user.
The object detection module identifies the objects of interest and
their associated locations in a scene. Finally, the logic unit module
makes sense of the user’s intent along with the localized objects of
interest, and builds a semantic description that the user can under-
stand for the speech module to convey verbally back to the user.
Initial results from a proof-of-concept system trained to localize four
different types of objects show promise to the feasibility of OLIV as
a useful aid for individuals with impaired vision.