Real-Time Speech and Audio Processing for Mobile Applications
DOI:
https://doi.org/10.63282/3050-9246.IJETCSIT-V7I1P117Keywords:
Real-Time Speech Processing, Mobile Applications, Low-Latency Recognition, Command Detection, Noise-Robust Modeling, Deep Learning Architectures, Feature Extraction, Model Compression, On-Device Inference, Real-Time OptimizationAbstract
Real-time audio and speech processing are now a necessary component of modern mobile applications, enabling users to engage comfortably with voice-based inputs and speech recognition capabilities. Mobile environments do come with inherent challenges such as limited computation, battery constraints, and the need for low-latency noise-immune solutions. The present paper addresses the design and implementation of real-time speech recognition and command-detection systems optimized for mobile platforms. We focus on low-latency methods and algorithms for wake word spotting and command understanding in order to provide quick reaction to user feedback. We also address noise-robust modeling techniques to battle the various acoustic scenes of mobile scenarios, making the most out of deep neural networks architectures, noise-irrelevant feature learning, and data augmentation. For optimizing model performance on mobile, we discuss model compression, pruning, and the benefits of on-device inference for real-time processing. Next, we discuss case studies and identify future directions and challenges in deploying real-time speech systems on mobile, in terms of accuracy vs. latency vs. power consumption. The results discussed herein present a comprehensive framework for building mobile-based speech and audio processing technology.
Downloads
References
[1] Sehgal, A., & Kehtarnavaz, N. (2018). A convolutional neural network smartphone app for real-time voice activity detection. IEEE access, 6, 9017-9026.
[2] Omyonga, K., & Shibwabo, B. K. (2015). The application of real-time voice recognition to control critical mobile device operations.
[3] Bhat, G. S., Shankar, N., Reddy, C. K., & Panahi, I. M. (2019). A real-time convolutional neural network based speech enhancement for hearing impaired listeners using smartphone. IEEE Access, 7, 78421-78433.
[4] Gokul, G., Yan, Y., Dantu, K., Ko, S. Y., & Ziarek, L. (2016, August). Real time sound processing on android. In Proceedings of the 14th International Workshop on Java Technologies for Real-Time and Embedded Systems (pp. 1-10).
[5] Sehgal, A., & Kehtarnavaz, N. (2018, January). Utilization of two microphones for real-time low-latency audio smartphone apps. In 2018 IEEE International Conference on Consumer Electronics (ICCE) (pp. 1-6). IEEE.
[6] Deligne, S., Dharanipragada, S., Gopinath, R., Maison, B., Olsen, P., & Printz, H. (2002). A robust high accuracy speech recognition system for mobile applications. IEEE Transactions on Speech and Audio Processing, 10(8), 551-561.
[7] Drakopoulos, F., Baby, D., & Verhulst, S. (2019). Real-time audio processing on a Raspberry Pi using deep neural networks (pp. 2827-2834). Universitätsbibliothek der RWTH Aachen.
[8] Ghosh, R., Ali, H., & Hansen, J. H. (2021). CCi-MOBILE: A portable real time speech processing platform for cochlear implant and hearing research. IEEE Transactions on Biomedical Engineering, 69(3), 1251-1263.
[9] Iwaya, Y., & Katz, B. F. (2018). Distributed signal processing architecture for real-time convolution of 3d audio rendering for mobile applications. In Virtual Reality and Augmented Reality: 15th EuroVR International Conference, EuroVR 2018, London, UK, October 22–23, 2018, Proceedings 15 (pp. 148-157). Springer International Publishing.
[10] SM, U. S., & Katiravan, J. (2022). Mobile application based speech and voice analysis for COVID-19 detection using computational audit techniques. International Journal of Pervasive Computing and Communications, 18(5), 508-517.
