spotify-voice-assistant/README.md

Spotify Voice Assistant for Home Assistant

Voice-controlled Spotify playback using natural language. Just say "Play Coldplay on the kitchen speaker" and your music starts playing - no complex YAML patterns and no cookie authentication.

Requirements

Before using this integration, you need:

1. Home Assistant - Obviously!
2. Spotify Premium account - Required for Spotify Connect API
3. Spotify integration - Home Assistant's official Spotify integration (provides OAuth authentication)
4. A conversation agent with function calling support - Examples: Extended OpenAI Conversation (works with Ollama, OpenAI, LocalAI), OpenAI Conversation, Google Generative AI Conversation
5. At least one Spotify Connect device - Any speaker/device that appears in Spotify (see Compatible Speakers for examples)

Why Use This?

Perfect for you if:

• You want voice-controlled Spotify without running a full music server
• You use conversation agents with function calling (Extended OpenAI Conversation, OpenAI, Gemini, etc.)
• You want direct Spotify Connect integration with minimal dependencies
• You prefer simple, maintainable code (<120 lines)

Consider alternatives if:

• You need multi-provider music aggregation (Spotify + local files + streaming services) → Use Music Assistant
• You want rich UI-based queue management and multi-room features → Use Music Assistant
• You use Home Assistant Assist voice pipeline → Music Assistant has native voice support
• You need advanced Spotify API features → Use SpotifyPlus

Use both together: Many users run Music Assistant for UI/management and this integration for voice control via custom conversation agents.

⚠️ Important: This integration provides the search functionality - your conversation agent's LLM model and hardware determine the actual voice command accuracy and response time. For local voice pipelines, qwen3:4b is a pretty good starting point. See Performance for optimization guidance.

The Problem

Want to control Spotify with your voice in Home Assistant? You'll quickly hit these issues:

• Spotcast requires cookie authentication (sp_dc, sp_key) which breaks frequently with "serverTime" errors
• Music Assistant is a full-featured music server - excellent for complete music management, but requires running a separate server and aggregates multiple providers
• SpotifyPlus is feature-rich but complex to set up for simple voice commands
• Custom Sentences require rigid YAML patterns - "Play {artist} on {speaker}" instead of natural language
• Built-in Spotify integration has no search service for voice assistants to use
• Standard Spotify search returns wrong artists - Personalized recommendations instead of exact matches (ask for Coldplay, get Taylor Swift)

The Solution

A lightweight integration designed specifically for voice assistants with function calling:

✅ Direct Spotify Connect integration - No music server required, uses Home Assistant's official Spotify integration ✅ Natural language from day one - "Play Coldplay" not "play artist equals Coldplay" ✅ Zero additional authentication - Reuses your existing Home Assistant Spotify OAuth ✅ Exact match artist search - Finds Coldplay when you ask for Coldplay, not recommendations ✅ Works with any Spotify Connect device - WiiM, Sonos, Google Cast, Echo, whatever you have ✅ Under 120 lines of code - Simple, maintainable, easy to understand ✅ Voice assistant ready - Complete conversation agent examples included

Comparison with Alternatives

Feature	Spotify Voice Assistant	Spotcast	Music Assistant	SpotifyPlus
Primary Focus	Voice search for conversation agents	Casting	Complete music server	Spotify API wrapper
Architecture	Direct Spotify Connect	Spotify API	Music server (aggregates providers)	Spotify API
Lines of Code	<120	1000+	10,000+	5000+
Authentication	Reuses HA OAuth	Cookies (breaks often)	Own server auth	Own OAuth flow
Setup Complexity	1 YAML line	Cookies + config	Full server install	Complex config
Exact Artist Match	✅ Yes	❌ No	✅ Yes	✅ Yes
Voice Support	✅ Function calling	⚠️ Manual scripts	✅ HA Assist integration	⚠️ Manual scripts
Natural Language	✅ Via conversation agent	❌ Rigid patterns	✅ Via HA Assist	❌ Manual calls
External Dependencies	None	None	Separate server	None
Hardware Support	Any Spotify Connect	Chromecast focused	Universal	Any Spotify Connect

Features

• ✅ Voice-first design - Built for natural language from day one
• ✅ Hardware agnostic - Works with any Spotify Connect device
• ✅ Search by type - Artists, albums, tracks, and playlists
• ✅ Exact match preference - Finds what you ask for across all content types, not recommendations
• ✅ Smart playlist search - Checks your personal playlists first, then falls back to public Spotify playlists
• ✅ Complete examples - Extended OpenAI Conversation config included
• ✅ Playback control - Pause, play, skip, volume, shuffle - all via voice
• ✅ Artist radio mode - Automatically shuffles when playing artists for dynamic playlists
• ✅ Zero config authentication - Leverages existing Spotify integration
• ✅ Detailed logging - Debug mode shows exactly what's happening
• ✅ Service response support - Returns data to automations/scripts

Quick Start

Say goodbye to complex configurations. Get voice-controlled music in 3 steps:

1. Install Prerequisites

• Spotify Premium account (required for Spotify Connect)
• Official Spotify integration configured in Home Assistant
• Extended OpenAI Conversation (or another LLM conversation agent)
• At least one Spotify Connect device (see Compatible Speakers)

2. Install This Integration

Via HACS (Recommended):

1. HACS > Integrations > ⋮ (menu) > Custom repositories
2. Add: https://github.com/cauld/spotify-voice-assistant
3. Category: Integration
4. Install "Spotify Voice Assistant"
5. Restart Home Assistant

Manual Installation:

1. Download this repository
2. Copy custom_components/spotify_search to your HA custom_components folder
3. Restart Home Assistant

Add to configuration.yaml:

spotify_search:

3. Configure Extended OpenAI Conversation

Copy these functions to Extended OpenAI Conversation settings:

Click to expand function configuration

- spec:
    name: search_spotify
    description: Search Spotify for an artist, album, track, or playlist and return the Spotify URI. Use this before playing music to get the URI.
    parameters:
      type: object
      properties:
        query:
          type: string
          description: Artist, album, track, or playlist name to search for (e.g., "Coldplay", "Parachutes", "Yellow", "Today's Top Hits")
        type:
          type: string
          enum: [artist, album, track, playlist]
          description: Type of content to search for. Playlist searches check your personal playlists first, then fall back to public Spotify playlists.
      required:
        - query
        - type
  function:
    type: script
    sequence:
      - service: spotify_search.search
        response_variable: _function_result
        data:
          query: "{{ query }}"
          type: "{{ type }}"

- spec:
    name: play_music
    description: Play music on a Spotify Connect device using a Spotify URI. You must first call search_spotify to get the URI.
    parameters:
      type: object
      properties:
        spotify_uri:
          type: string
          description: Spotify URI from search_spotify (e.g., "spotify:artist:4gzpq5DPGxSnKTe4SA8HAU")
        media_player:
          type: string
          description: Entity ID of Spotify Connect media player (e.g., "media_player.kitchen_speaker")
      required:
        - spotify_uri
        - media_player
  function:
    type: script
    sequence:
      - service: media_player.play_media
        target:
          entity_id: "{{ media_player }}"
        data:
          media_content_id: "{{ spotify_uri }}"
          media_content_type: "music"

- spec:
    name: control_playback
    description: Control music playback (pause, resume, stop, next track, previous track, set volume, shuffle)
    parameters:
      type: object
      properties:
        action:
          type: string
          enum: [pause, play, stop, next_track, previous_track, volume_set, shuffle_on, shuffle_off]
          description: Playback control action
        media_player:
          type: string
          description: Entity ID of media player
        volume_level:
          type: number
          description: Volume level (0-100) for volume_set action only
      required:
        - action
        - media_player
  function:
    type: script
    sequence:
      - if: "{{ action == 'pause' }}"
        then:
          - service: media_player.media_pause
            target:
              entity_id: "{{ media_player }}"
      - if: "{{ action == 'play' }}"
        then:
          - service: media_player.media_play
            target:
              entity_id: "{{ media_player }}"
      - if: "{{ action == 'stop' }}"
        then:
          - service: media_player.media_stop
            target:
              entity_id: "{{ media_player }}"
      - if: "{{ action == 'next_track' }}"
        then:
          - service: media_player.media_next_track
            target:
              entity_id: "{{ media_player }}"
      - if: "{{ action == 'previous_track' }}"
        then:
          - service: media_player.media_previous_track
            target:
              entity_id: "{{ media_player }}"
      - if: "{{ action == 'volume_set' }}"
        then:
          - service: media_player.volume_set
            target:
              entity_id: "{{ media_player }}"
            data:
              volume_level: "{{ volume_level / 100 }}"
      - if: "{{ action == 'shuffle_on' }}"
        then:
          - service: media_player.shuffle_set
            target:
              entity_id: "{{ media_player }}"
            data:
              shuffle: true
      - if: "{{ action == 'shuffle_off' }}"
        then:
          - service: media_player.shuffle_set
            target:
              entity_id: "{{ media_player }}"
            data:
              shuffle: false

Add this to your Extended OpenAI Conversation system prompt:

Music Playback:
- When asked to play music, follow this two-step process: 1) Call search_spotify to get the Spotify URI, 2) Call play_music with the URI and media player entity
- Available media players: media_player.kitchen_speaker, media_player.gaming_room_speaker, media_player.office_speaker
- Default speaker: If no speaker is specified, use media_player.kitchen_speaker
- Parse commands like "Play {Artist/Album/Track/Playlist}" and determine the media type automatically (artist, album, track, playlist, or user_playlist)
- IMPORTANT: Use type="user_playlist" when user says "my playlist", "my [playlist name]", or refers to their personal saved playlists. Use type="playlist" for public Spotify playlists like "Today's Top Hits"
- IMPORTANT: When playing an artist, always enable shuffle after starting playback by calling control_playback with action: shuffle_on to create a dynamic playlist experience
- For playback control (pause, skip, volume, shuffle), use the control_playback function

Customization tips:

• Entity IDs are descriptive (like media_player.kitchen_speaker): Just list the entity IDs as shown above
• Entity IDs are cryptic (like media_player.wiim_12345): Include friendly names for better LLM understanding:

  - Available speakers: Kitchen Speaker (media_player.wiim_12345), Gaming Room (media_player.sonos_abc), Office (media_player.cast_xyz)
  

• Default speaker: Change media_player.kitchen_speaker to your preferred default location

Done! Try It

Standard Patterns (like intent-based systems):

• "Play Coldplay on the kitchen speaker"
• "Play the album Parachutes"
• "Play Yellow by Coldplay"
• "Play Today's Top Hits"
• "Play my workout playlist"
• "Pause the music"
• "Skip to the next track"
• "Set volume to 50%"
• "Shuffle on"

Natural Language (LLM advantage):

• "I'm in the mood for some Coldplay"
• "Put on that song Yellow"
• "Start playing Coldplay" (uses default speaker from prompt, automatically shuffles artist)
• "Can you play the Parachutes album?"
• "Play me something by Coldplay"
• "I want to hear some chill music from Coldplay"
• "Play Coldplay but shuffle it"
• "Put on some music from that British band with Chris Martin"
• "Play my chill vibes playlist"
• "Put on my running music"

Conversational Playback Control:

• "Make it louder" (instead of "Set volume to 70")
• "Turn it down a bit" (instead of "Set volume to 30")
• "Next song please" (instead of "Skip to next track")
• "Stop the music" (instead of "Pause playback")
• "Shuffle this" (instead of "Set shuffle to on")

Note: When no speaker is specified, the LLM will use the default media player defined in your Extended OpenAI Conversation prompt. Add instructions like "If no speaker specified, use media_player.kitchen_speaker" to your system prompt.

Why Natural Language Matters

Traditional voice assistants require specific sentence patterns:

Intent Pattern: "Play [artist] on [speaker]"
✅ Works: "Play Coldplay on kitchen speaker"
❌ Fails: "I want to listen to Coldplay"
❌ Fails: "Put on some Coldplay"
❌ Fails: "Start playing Coldplay in the kitchen"

This integration uses LLM-based function calling to understand conversational requests:

LLM Understanding: Extracts intent from natural speech
✅ Works: "Play Coldplay on kitchen speaker"
✅ Works: "I want to listen to Coldplay"
✅ Works: "Put on some Coldplay"
✅ Works: "Start playing Coldplay in the kitchen"
✅ Works: "Play me that British band with Chris Martin"

The LLM understands what you mean, not just what you say.

Compatible Speakers

This integration works with any Spotify Connect-compatible device:

Tested & Verified

• WiiM Audio Pro (and all WiiM speakers)
• Sonos speakers (all models with Spotify)
• Google Nest/Home speakers
• Amazon Echo with Spotify

Should Work (Spotify Connect Compatible)

• Smart Speakers:

  • Apple HomePod (with Spotify app)
  • Bose Home Speaker series
  • JBL Link series
  • Harman Kardon Citation series
  • Bang & Olufsen Beosound series

• Streaming Devices:

  • Chromecast Audio
  • Roku with Spotify channel
  • Fire TV with Spotify
  • Apple TV with Spotify app

• AV Receivers:

  • Denon HEOS-enabled receivers
  • Yamaha MusicCast receivers
  • Marantz NR/SR series
  • Pioneer VSX series with Spotify

• DIY Solutions:

  • Raspberry Pi with Raspotify
  • Raspberry Pi with Spotifyd
  • Any Linux/Mac/Windows computer running Spotify

• Mobile Devices:

  • Phones/tablets running Spotify app (for testing)

Requirements

• Spotify Premium account (Spotify Connect requires Premium)
• Device must show up in Home Assistant as a media_player entity
• Device must support Spotify Connect (check manufacturer specs)

Note: If your device runs Spotify and shows up in Home Assistant's integrations, it will work with this integration.

Works Alongside Music Assistant

Music Assistant and this integration serve different purposes and work well together:

Music Assistant

• Purpose: Full-featured music server with rich UI, multi-room audio, queue management
• Architecture: Music server that aggregates multiple providers (Spotify, local files, streaming services)
• Voice support: Built-in voice integration works with Home Assistant Assist (with or without LLM enhancement)
• Best for: Complete music management solution with visual control

This Integration (Spotify Voice Assistant)

• Purpose: Lightweight Spotify search service for conversation agents with function calling support
• Architecture: Direct Spotify Connect integration via Home Assistant's official Spotify integration (no music server required)
• Works with: Any conversation agent that supports function calling (Extended OpenAI Conversation, OpenAI Conversation, Google Generative AI Conversation, etc.)
• Best for: Adding Spotify voice control without running a full music server

Using Both Together

Many users run both simultaneously:

• Music Assistant: Provides UI for manual control, queue management, and automations (skip voice setup)
• This integration: Handles voice control via custom conversation agents
• They work together: Music started via voice appears in Music Assistant UI and remains fully controllable there

Example workflow: Say "Play Coldplay on office speaker" → music starts → open Music Assistant UI → see what's playing, adjust volume, skip tracks, manage queue.

They don't directly integrate but work together because both control the same Spotify Connect devices independently.

Choose Your Setup

• Voice only: Use this integration alone
• Voice + rich UI: Use both (Music Assistant without voice + this integration)
• Home Assistant Assist voice: Use Music Assistant's built-in voice support
• Custom conversation agent voice: Use this integration

How It Works

The Two-Step Process

1. Search Spotify

You: "Play Coldplay"
LLM: Calls search_spotify(query="Coldplay", type="artist")
Response: {"uri": "spotify:artist:4gzpq5DPGxSnKTe4SA8HAU", "name": "Coldplay"}

2. Play on Device

LLM: Calls play_music(uri="spotify:artist:...", media_player="media_player.kitchen_speaker")
Result: Music starts playing

Exact Match Search

This integration uses smart matching across all content types to avoid Spotify's personalization issues:

1. Queries Spotify for top 10 results (instead of just 1)
2. Checks each result for exact name match (case-insensitive)
3. Returns exact match if found, otherwise returns first result
4. Logs match type (exact/first/partial) for debugging

Applies to: Artists, albums, tracks, and playlists

Why this matters: Standard Spotify search prioritizes personalized recommendations. If you search "Coldplay," you might get Taylor Swift if you listen to her frequently. Our exact matching ensures you get Coldplay when you ask for Coldplay.

Smart Playlist Search

Playlist searches automatically check your personal playlists first, then fall back to public Spotify playlists:

1. Cleans the query (removes "playlist" and "playlists" from search terms)
2. Searches your saved Spotify playlists for exact match
3. If no exact match, searches your playlists for partial match
4. If still not found, searches public Spotify playlists
5. Returns exact match from public results if found, otherwise first result

Examples:

• "Play my workout playlist" → Checks your saved playlists first
• "Play vibes" → Finds your "Vibes" playlist if you have one, otherwise searches public playlists
• "Play Today's Top Hits" → Checks your playlists first, then finds public playlist

No Additional Authentication

The integration leverages Home Assistant's official Spotify integration:

• Uses existing OAuth tokens
• No cookies to manage (sp_dc, sp_key)
• No tokens to refresh
• Works as long as your Spotify integration works

Performance Optimization

The integration includes smart caching for optimal performance:

Spotify Client Cache:

• First search: ~1.6-5.6ms (finds and caches Spotify client)
• Subsequent searches: ~0.1ms (uses cached client)
• Performance gain: 15-50x faster on repeated searches

User Playlist Cache:

• First user_playlist search: Fetches all user playlists from Spotify API
• Subsequent user_playlist searches: Uses cached playlist data
• Performance gain: Significantly faster for users with large playlist libraries

Cache Features:

• Automatically validates cached client on every call
• Self-invalidates when Spotify integration is reloaded or removed
• Caches user playlists to avoid repeated API calls
• Manual cache clearing available via spotify_search.clear_cache service

The cache is module-level and process-scoped, so it clears automatically on Home Assistant restart.

Advanced Usage

Search Types

The integration supports five search types:

Type	Description	Example Query	Use Case
artist	Search all Spotify artists	"Coldplay"	Play an artist's music
album	Search all Spotify albums	"Parachutes"	Play a specific album
track	Search all Spotify tracks	"Yellow"	Play a specific song
playlist	Search all public Spotify playlists	"Today's Top Hits"	Play curated or public playlists
user_playlist	Search only your saved playlists	"workout"	Play your personal playlists

Key difference:

• playlist - Searches all of Spotify's public playlists
• user_playlist - Searches only playlists you've saved/created

Developer Tools Testing

Test the search service directly:

service: spotify_search.search
data:
  query: "Coldplay"
  type: "artist"

Response:

{
  "uri": "spotify:artist:4gzpq5DPGxSnKTe4SA8HAU",
  "name": "Coldplay",
  "type": "artist"
}

Test user playlist search:

service: spotify_search.search
data:
  query: "workout"
  type: "user_playlist"

Response:

{
  "uri": "spotify:playlist:37i9dQZF1DX76Wlfdnj7AP",
  "name": "Workout Beats",
  "type": "playlist"
}

Use in Automations

automation:
  - alias: "Morning Music"
    trigger:
      - platform: time
        at: "07:00:00"
    action:
      - service: spotify_search.search
        response_variable: artist_result
        data:
          query: "Chillhop Music"
          type: "artist"
      - service: media_player.play_media
        target:
          entity_id: media_player.kitchen_speaker
        data:
          media_content_id: "{{ artist_result.uri }}"
          media_content_type: "music"

Prompt Optimization for Multi-part Queries

For better accuracy when users provide both song/album AND artist names, enhance your Extended OpenAI Conversation system prompt to combine search terms:

Music Query Optimization:
- When user provides both song/album AND artist, combine them into the search query
- Example: "Play Yellow by Coldplay" → search_spotify(query="Yellow Coldplay", type="track")
- Example: "Play Parachutes album by Coldplay" → search_spotify(query="Parachutes Coldplay", type="album")
- The combined query improves search accuracy by providing more context to Spotify

This approach leverages Spotify's search algorithm to naturally weight both terms, improving accuracy when multiple artists have songs/albums with the same name.

Clear Cache Service

If you experience issues or want to refresh cached data (e.g., after adding new playlists to your Spotify library), you can manually clear the cache:

service: spotify_search.clear_cache

This clears:

• Cached Spotify client reference
• Cached user playlist data

When to use:

• After adding/removing playlists in Spotify (to refresh user_playlist searches)
• After removing or re-adding the Spotify integration
• If experiencing unexpected search results

The cache automatically invalidates when the Spotify integration is reloaded, so manual clearing is rarely needed.

Debug Logging

Enable detailed logging in configuration.yaml:

logger:
  default: info
  logs:
    custom_components.spotify_search: debug

This shows search queries, match types (exact vs. first result), and any errors.

Troubleshooting

"No Spotify media player entity found"

Cause: Official Spotify integration not configured.

Solution:

1. Settings > Integrations > Add Integration
2. Search for "Spotify"
3. Complete OAuth authentication
4. Verify media_player.spotify_* entity appears
5. Restart Home Assistant

"Spotify entity does not have a coordinator"

Cause: Spotify integration hasn't fully initialized.

Solution: Wait 30 seconds after HA starts, or restart Home Assistant.

Voice commands not working

Cause: Extended OpenAI Conversation functions not configured.

Solution:

1. Verify functions are added to Extended OpenAI settings
2. Check system prompt includes music playback instructions
3. Test search service manually in Developer Tools
4. Check Home Assistant logs for errors

Wrong artist/song playing

Cause: Exact match not found in top 10 results.

Solution:

• Use more specific query: "Coldplay band" instead of just "Coldplay"
• Check search result in Developer Tools first
• Try searching by track or album if artist search fails

Music won't play on speaker

Cause: Device not Spotify Connect compatible or not in HA.

Solution:

1. Verify device supports Spotify Connect (check manufacturer site)
2. Check device shows in Settings > Integrations
3. Test playing Spotify directly to device from Spotify app
4. Verify Spotify Premium account is active

Performance

The integration itself is highly optimized with client caching (15-50x faster after first search). However, overall voice command response time is dominated by your conversation agent's LLM processing.

Response Time Breakdown

Typical "Play Coldplay" command:

• Spotify search: ~500ms first time, ~10ms cached ✅ Fast
• LLM processing: 2-10+ seconds per function call ⚠️ Bottleneck
• Total: Depends almost entirely on your LLM

Optimization Tips

1. Choose a faster LLM

• Slow (2-5 min): Large models on CPU (qwen3:8b, llama3:8b)
• Medium (5-15 sec): Small local models (qwen3:1.8b, phi3:mini)
• Fast (2-5 sec): Cloud APIs (OpenAI GPT-4o-mini, Anthropic Claude Haiku)

2. Reduce exposed devices

Each device in your conversation agent adds tokens to every LLM call, slowing processing:

• 38 devices: ~1,600 tokens = slower responses
• 25 devices: ~1,000 tokens = 20-30% faster

Safe to remove from Extended OpenAI Conversation:

• Voice satellite media players (if not directly controlled)
• Unavailable/unused devices
• Status sensors not needed for voice commands

3. Enable GPU acceleration

If using Ollama locally, GPU acceleration provides 5-10x speedup over CPU.

Expected Performance

Configuration	Response Time
Cloud API (GPT-4o-mini) + 25 devices	2-5 seconds ✅
Local GPU (qwen3:8b) + 25 devices	10-30 seconds
Local CPU (qwen3:8b) + 38 devices	2-5 minutes ❌

Bottom line: This integration adds <1 second overhead. LLM choice determines user experience.

Roadmap

See TODO.md for detailed enhancement proposals and implementation plans.

Contributing

Contributions welcome! Please submit a Pull Request.

Support

• Issues: GitHub Issues
• Discussions: Home Assistant Community

License

MIT License - See LICENSE file for details

Acknowledgments

• Built on Home Assistant Spotify integration
• Designed for Extended OpenAI Conversation
• Inspired by the HA community's need for simple voice-controlled music