2024-singing-assistants.html

---
layout: publication
year: 2024
month: 10
selected: false
coming-soon: false
hidden: false
external : false
# link: https://dl.acm.org/doi/10.1145/3472749.3474750
pdf: https://doi.org/10.1145/3654777.3676416
title: "Towards Music-Aware Virtual Assistants"
authors:
  - Alexander Wang
  - David Lindlbauer
  - Chris Donahue

# blog: https://interactive-structures.org/publications/2023-10-parametric-haptics/
doi: 10.1145/3654777.3676416
venue_location: Pittsburgh, PA, USA
venue_url: https://uist.acm.org/2024/
venue_tags:
  - ACM UIST
type:
  - Conference
tags:
  - Computational Interaction
  - Voice assitants
  - Machine Learning
venue: ACM UIST

video-thumb: CRGXo4Rg5uM
video-30sec: CRGXo4Rg5uM
video-suppl: 1vlq7c_avAY
#video-talk-5min: l9ycUrf50TE
#video-talk-15min: l9ycUrf50TE

bibtex: "@inproceedings {Wang2024SingingAssistants, \n
author = {Wang, Alexander and Lindlbauer, David and Donahue, Chris}, \n
title = {Towards Music-Aware Virtual Assistants}, \n
year = {2024}, \n
publisher = {Association for Computing Machinery}, \n
address = {New York, NY, USA}, \n
doi = {10.1145/3654777.3676416}, \n
keywords = {Audio, Music, Virtual Assistants, Notification, Interruptions, Speech, Machine Learning}, \n
location = {Pittsburgh, PA, USA}, \n
series = {UIST '24} \n
}"

---

We propose a system for broadcasting speech notifications to music listeners in a manner that listeners find less intrusive and more delightful than existing conventions. Speech notifications provide convenient access to rich information without the need for a screen, playing a key role in virtual assistants that support users by e.g.,~dictating text notifications, or providing directions for navigation. Virtual assistants see prevalent use in hands-free settings such as driving or exercising, activities where users typically also enjoy listening to music. When broadcasting speech notifications to users, virtual assistants will typically mute a user's music to improve intelligibility. However, users may perceive these interruptions as intrusive, negatively impacting their music-listening experience. To address this challenge, we propose the concept of music-aware virtual assistants, where speech notifications are conveyed via singing them in harmony with a song that a user is listening to. We contribute a system that processes user music and notification text to produce a blended mix, replacing original song lyrics with the notification content. In a user study comparing musical assistants to standard virtual assistants, participants expressed a preference for our musical approach due to its better alignment with music, reduced intrusiveness, and more delightful user experience.

<h3>More information</h3>
<p>
<strong>Audio samples</strong> are available at <a href="https://drive.google.com/drive/folders/1xIwKG3nBfc5bRkfJ7YgcWn4r_NkgqB4f?usp=sharing">https://drive.google.com/drive/folders/1xIwKG3nBfc5bRkfJ7YgcWn4r_NkgqB4f?usp=sharing</a>
</p>