Zhu, HY, Hossain, SN, Jin, C, Singh, AK, Nguyen, MTD, Deverell, L, Nguyen, V, Gates, FS, Fernandez, IG, Melencio, MV, Bell, J-AR & Lin, C-T 2023, 'An investigation into the effectiveness of using acoustic touch to assist people who are blind', PLOS ONE, vol. 18, no. 10, pp. e0290431-e0290431.
View/Download from: Publisher's site
View description>>
Wearable smart glasses are an emerging technology gaining popularity in the assistive technologies industry. Smart glasses aids typically leverage computer vision and other sensory information to translate the wearer’s surrounding into computer-synthesized speech. In this work, we explored the potential of a new technique known as “acoustic touch” to provide a wearable spatial audio solution for assisting people who are blind in finding objects. In contrast to traditional systems, this technique uses smart glasses to sonify objects into distinct sound auditory icons when the object enters the device’s field of view. We developed a wearable Foveated Audio Device to study the efficacy and usability of using acoustic touch to search, memorize, and reach items. Our evaluation study involved 14 participants, 7 blind or low-visioned and 7 blindfolded sighted (as a control group) participants. We compared the wearable device to two idealized conditions, a verbal clock face description and a sequential audio presentation through external speakers. We found that the wearable device can effectively aid the recognition and reaching of an object. We also observed that the device does not significantly increase the user’s cognitive workload. These promising results suggest that acoustic touch can provide a wearable and effective method of sensory augmentation.
Khan, MW, Sheng, H, Zhang, H, Du, H, Wang, S, Coroneo, MT, Hajati, F, Shariflou, S, Kalloniatis, M, Phu, J, Agar, A, Huang, Z, Golzan, M & Yu, X 1970, 'RVD: A Handheld Device-Based Fundus Video Dataset for Retinal Vessel Segmentation', Advances in Neural Information Processing Systems.
View description>>
Retinal vessel segmentation is generally grounded in image-based datasets collected with bench-top devices. The static images naturally lose the dynamic characteristics of retina fluctuation, resulting in diminished dataset richness, and the usage of bench-top devices further restricts dataset scalability due to its limited accessibility. Considering these limitations, we introduce the first video-based retinal dataset by employing handheld devices for data acquisition. The dataset comprises 635 smartphone-based fundus videos collected from four different clinics, involving 415 patients from 50 to 75 years old. It delivers comprehensive and precise annotations of retinal structures in both spatial and temporal dimensions, aiming to advance the landscape of vasculature segmentation. Specifically, the dataset provides three levels of spatial annotations: binary vessel masks for overall retinal structure delineation, general vein-artery masks for distinguishing the vein and artery, and fine-grained vein-artery masks for further characterizing the granularities of each artery and vein. In addition, the dataset offers temporal annotations that capture the vessel pulsation characteristics, assisting in detecting ocular diseases that require fine-grained recognition of hemodynamic fluctuation. In application, our dataset exhibits a significant domain shift with respect to data captured by bench-top devices, thus posing great challenges to existing methods. Thanks to rich annotations and data scales, our dataset potentially paves the path for more advanced retinal analysis and accurate disease diagnosis. In the experiments, we provide evaluation metrics and benchmark results on our dataset, reflecting both the potential and challenges it offers for vessel segmentation tasks. We hope this challenging dataset would significantly contribute to the development of eye disease diagnosis and early prevention. The dataset is available at RVD.
Nguyen, M, Zhu, H, Sun, H, Nguyen, V, Deverell, L, Singh, A, Jin, C & Lin, C-T 1970, 'An Evaluation of the Presentation of Acoustic Cues for Shorelining Techniques', 2023 Immersive and 3D Audio: from Architecture to Automotive (I3DA), 2023 Immersive and 3D Audio: from Architecture to Automotive (I3DA), IEEE, pp. 1-9.
View/Download from: Publisher's site
Shariflou, S, Agar, A, Phu, J, Coroneo, M, Kalloniatis, M & Golzan, M 1970, 'Association between optic nerve head perfusion measures and retinal ganglion cell count estimates in glaucoma', World Glaucoma Congress, Rome, Italy.
Nguyen, M, Zhu, H, Sun, H, Nguyen, V, Jin, C & Lin, C-T 2023, 'An evaluation of various spatial audio rendering and presentation techniques to enhance active navigation with sensory augmentation', Acoustical Society of America (ASA), pp. A196-A196.
View/Download from: Publisher's site
View description>>
Active navigation is essential in everyday life and refers to the combination of cognition (spatial mapping, path planning, and decision making) and motor-sensory execution (moving and sensing environment). For people who are blind or have low-vision, auditory and tactile sensory augmentation is critical to active navigation. In assistive technologies, binaural spatial audio rendering is widely adopted. However, the most effective methods to support fluent spatial navigation are still being studied. For example, in a previous study, we demonstrated the feasibility of using spatialized earcons to support a shorelining task. In this work, we use the same shorelining task to explore various forms of spatial earcon presentation with a focus on standardization and effectiveness. We also explore the development of an intuitive auditory grammar for spatial and contextual cues. We conduct psychophysical experiments and present experimental measures such as performance time and accuracy, heart-rate variability, and the NASA task load index.
Sun, H, Nguyen, M, Zhu, H, Nguyen, V, Lin, C-T & Jin, C 2023, 'A binaural room impulse response dataset and Shorelining psychophysical task for the evaluation of auditory sensory augmentation', Acoustical Society of America (ASA), pp. A195-A195.
View/Download from: Publisher's site
View description>>
Sensory augmentation using spatial sound presented in augmented-reality (AR) can assist people with low vision or blindness in navigating their environment (Katz et al., 2012). Nonetheless, in many situations, the poor quality of the binaural sound rendered using current tool sets limits the potential capability of the assistive technology. In particular, acoustic environments with near-field sources and reflections pose significant challenges. In this work, we provide a reference binaural room impulse response (BRIR) dataset with near-field sources and reflections and an associated shorelining psychophysical task that is useful for the evaluation of AR spatial audio. The dataset consists of 12∼small loudspeakers arranged on a 3-by-4 grid in a complex acoustic environment. BRIR measurements are recorded using the Head and Torso Simulator (HATS) for 17∼different receiver positions with a 5o angular resolution. Room impulse response measurements are also recorded using the Eigenmike for each of the 17 receiver positions. Using the Razer Anzu smart glasses to render the binaural AR spatial audio, we compare psychophysical performance on the shorelining navigation task using the recorded dataset and various existing binaural AR tool sets.
