Multimodal sensing-based camera applications

Miguel Bordallo López, Jari Hannuksela, Olli Silvén, Markku Vehviläinen

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

1 Citation (Scopus)


The increased sensing and computing capabilities of mobile devices can provide for enhanced mobile user experience. Integrating the data from different sensors offers a way to improve application performance in camera-based applications. A key advantage of using cameras as an input modality is that it enables recognizing the context. Therefore, computer vision has been traditionally utilized in user interfaces to observe and automatically detect the user actions. The imaging applications can also make use of various sensors for improving the interactivity and the robustness of the system. In this context, two applications fusing the sensor data with the results obtained from video analysis have been implemented on a Nokia Nseries mobile device. The first solution is a real-time user interface that can be used for browsing large images. The solution enables the display to be controlled by the motion of the user's hand using the built-in sensors as complementary information. The second application is a real-time panorama builder that uses the device's accelerometers to improve the overall quality, providing also instructions during the capture. The experiments show that fusing the sensor data improves camera-based applications especially when the conditions are not optimal for approaches using camera data alone.

Original languageEnglish
Title of host publicationProceedings of SPIE-IS and T Electronic Imaging - Multimedia on Mobile Devices 2011; and Multimedia Content Access
Subtitle of host publicationAlgorithms and Systems V
PublisherInternational Society for Optics and Photonics SPIE
ISBN (Print)978-081948418-5
Publication statusPublished - 2011
MoE publication typeNot Eligible
EventMultimedia on Mobile Devices 2011; and Multimedia Content Access: Algorithms and Systems V - San Francisco, CA, United States
Duration: 25 Jan 201126 Jan 2011

Publication series

SeriesProceedings of SPIE - The International Society for Optical Engineering


ConferenceMultimedia on Mobile Devices 2011; and Multimedia Content Access: Algorithms and Systems V
Country/TerritoryUnited States
CitySan Francisco, CA


  • Computer vision
  • Data fusion
  • Mobile device
  • Motion sensor
  • User interface


Dive into the research topics of 'Multimodal sensing-based camera applications'. Together they form a unique fingerprint.

Cite this