From ec0bcd174d304462b64ccf17a3287057abf604b9 Mon Sep 17 00:00:00 2001 From: Chad Auld Date: Sun, 16 Nov 2025 12:28:20 -0700 Subject: [PATCH] Add shuffle support for artist radio/dynamic playlists - Add shuffle_on and shuffle_off actions to control_playback function - Update prompt to automatically enable shuffle when playing artists - Add artist radio mode feature to README - Update all example configurations with shuffle support - Simplify control_playback YAML syntax (if/then vs choose/conditions) - Add shuffle examples to voice command demonstrations --- README.md | 243 +++++++++++++++--------- examples/extended_openai_functions.yaml | 22 ++- examples/system_prompt.txt | 3 +- 3 files changed, 179 insertions(+), 89 deletions(-) diff --git a/README.md b/README.md index 23ee4fc..f6680c4 100644 --- a/README.md +++ b/README.md @@ -9,17 +9,31 @@ Before using this integration, you need: 1. **Home Assistant** - Obviously! 2. **Spotify Premium account** - Required for Spotify Connect API 3. **[Spotify integration](https://www.home-assistant.io/integrations/spotify/)** - Home Assistant's official Spotify integration (provides OAuth authentication) -4. **[Extended OpenAI Conversation](https://github.com/jekalmin/extended_openai_conversation)** - Or another LLM-based conversation agent (Ollama, Google Generative AI, etc.) +4. **A conversation agent with function calling support** - Examples: [Extended OpenAI Conversation](https://github.com/jekalmin/extended_openai_conversation) (works with Ollama, OpenAI, LocalAI), OpenAI Conversation, Google Generative AI Conversation 5. **At least one Spotify Connect device** - Any speaker/device that appears in Spotify (see [Compatible Speakers](#compatible-speakers) for examples) -**Note:** This integration is specifically designed for custom voice assistants (Extended OpenAI Conversation, etc.). If you're using Home Assistant's built-in voice pipeline without a custom LLM, Music Assistant's native voice support may be sufficient. +## Why Use This? + +**Perfect for you if:** +- You want voice-controlled Spotify without running a full music server +- You use conversation agents with function calling (Extended OpenAI Conversation, OpenAI, Gemini, etc.) +- You want direct Spotify Connect integration with minimal dependencies +- You prefer simple, maintainable code (<120 lines) + +**Consider alternatives if:** +- You need multi-provider music aggregation (Spotify + local files + streaming services) → Use Music Assistant +- You want rich UI-based queue management and multi-room features → Use Music Assistant +- You use Home Assistant Assist voice pipeline → Music Assistant has [native voice support](https://github.com/music-assistant/voice-support) +- You need advanced Spotify API features → Use SpotifyPlus + +**Use both together:** Many users run Music Assistant for UI/management and this integration for voice control via custom conversation agents. ## The Problem Want to control Spotify with your voice in Home Assistant? You'll quickly hit these issues: - **Spotcast** requires cookie authentication (`sp_dc`, `sp_key`) which breaks frequently with "serverTime" errors -- **Music Assistant** is excellent for UI-based music control, but voice commands require specific sentence patterns that break when using custom voice assistants (Extended OpenAI Conversation, etc.) +- **Music Assistant** is a full-featured music server - excellent for complete music management, but requires running a separate server and aggregates multiple providers - **SpotifyPlus** is feature-rich but complex to set up for simple voice commands - **Custom Sentences** require rigid YAML patterns - "Play {artist} on {speaker}" instead of natural language - **Built-in Spotify integration** has no search service for voice assistants to use @@ -27,26 +41,28 @@ Want to control Spotify with your voice in Home Assistant? You'll quickly hit th ## The Solution -A lightweight integration designed specifically for voice assistants that: +A lightweight integration designed specifically for voice assistants with function calling: +✅ **Direct Spotify Connect integration** - No music server required, uses Home Assistant's official Spotify integration ✅ **Natural language from day one** - "Play Coldplay" not "play artist equals Coldplay" ✅ **Zero additional authentication** - Reuses your existing Home Assistant Spotify OAuth ✅ **Exact match artist search** - Finds Coldplay when you ask for Coldplay, not recommendations ✅ **Works with any Spotify Connect device** - WiiM, Sonos, Google Cast, Echo, whatever you have ✅ **Under 120 lines of code** - Simple, maintainable, easy to understand -✅ **Voice assistant ready** - Complete Extended OpenAI Conversation examples included +✅ **Voice assistant ready** - Complete conversation agent examples included ## Comparison with Alternatives | Feature | Spotify Voice Assistant | Spotcast | Music Assistant | SpotifyPlus | |---------|------------------------|----------|-----------------|-------------| -| **Primary Focus** | Voice control | Casting | Full music server | Spotify API wrapper | +| **Primary Focus** | Voice search for conversation agents | Casting | Complete music server | Spotify API wrapper | +| **Architecture** | Direct Spotify Connect | Spotify API | Music server (aggregates providers) | Spotify API | | **Lines of Code** | <120 | 1000+ | 10,000+ | 5000+ | | **Authentication** | Reuses HA OAuth | Cookies (breaks often) | Own server auth | Own OAuth flow | | **Setup Complexity** | 1 YAML line | Cookies + config | Full server install | Complex config | | **Exact Artist Match** | ✅ Yes | ❌ No | ✅ Yes | ✅ Yes | -| **Voice Assistant Ready** | ✅ Yes (built-in) | ⚠️ Partial | ✅ Yes | ⚠️ Partial | -| **Natural Language** | ✅ LLM-based | ❌ Rigid patterns | ✅ LLM-based | ❌ Manual calls | +| **Voice Support** | ✅ Function calling | ⚠️ Manual scripts | ✅ HA Assist integration | ⚠️ Manual scripts | +| **Natural Language** | ✅ Via conversation agent | ❌ Rigid patterns | ✅ Via HA Assist | ❌ Manual calls | | **External Dependencies** | None | None | Separate server | None | | **Hardware Support** | Any Spotify Connect | Chromecast focused | Universal | Any Spotify Connect | @@ -57,7 +73,8 @@ A lightweight integration designed specifically for voice assistants that: - ✅ **Search by type** - Artists, albums, tracks, playlists - ✅ **Exact match preference** - Finds what you ask for, not recommendations - ✅ **Complete examples** - Extended OpenAI Conversation config included -- ✅ **Playback control** - Pause, play, skip, volume - all via voice +- ✅ **Playback control** - Pause, play, skip, volume, shuffle - all via voice +- ✅ **Artist radio mode** - Automatically shuffles when playing artists for dynamic playlists - ✅ **Zero config authentication** - Leverages existing Spotify integration - ✅ **Detailed logging** - Debug mode shows exactly what's happening - ✅ **Service response support** - Returns data to automations/scripts @@ -152,13 +169,13 @@ Copy these functions to Extended OpenAI Conversation settings: - spec: name: control_playback - description: Control music playback (pause, resume, stop, next track, previous track, set volume) + description: Control music playback (pause, resume, stop, next track, previous track, set volume, shuffle) parameters: type: object properties: action: type: string - enum: [pause, play, stop, next_track, previous_track, volume_set] + enum: [pause, play, stop, next_track, previous_track, volume_set, shuffle_on, shuffle_off] description: Playback control action media_player: type: string @@ -172,51 +189,52 @@ Copy these functions to Extended OpenAI Conversation settings: function: type: script sequence: - - choose: - - conditions: - - condition: template - value_template: "{{ action == 'pause' }}" - sequence: - - service: media_player.media_pause - target: - entity_id: "{{ media_player }}" - - conditions: - - condition: template - value_template: "{{ action == 'play' }}" - sequence: - - service: media_player.media_play - target: - entity_id: "{{ media_player }}" - - conditions: - - condition: template - value_template: "{{ action == 'stop' }}" - sequence: - - service: media_player.media_stop - target: - entity_id: "{{ media_player }}" - - conditions: - - condition: template - value_template: "{{ action == 'next_track' }}" - sequence: - - service: media_player.media_next_track - target: - entity_id: "{{ media_player }}" - - conditions: - - condition: template - value_template: "{{ action == 'previous_track' }}" - sequence: - - service: media_player.media_previous_track - target: - entity_id: "{{ media_player }}" - - conditions: - - condition: template - value_template: "{{ action == 'volume_set' }}" - sequence: - - service: media_player.volume_set - target: - entity_id: "{{ media_player }}" - data: - volume_level: "{{ volume_level / 100 }}" + - if: "{{ action == 'pause' }}" + then: + - service: media_player.media_pause + target: + entity_id: "{{ media_player }}" + - if: "{{ action == 'play' }}" + then: + - service: media_player.media_play + target: + entity_id: "{{ media_player }}" + - if: "{{ action == 'stop' }}" + then: + - service: media_player.media_stop + target: + entity_id: "{{ media_player }}" + - if: "{{ action == 'next_track' }}" + then: + - service: media_player.media_next_track + target: + entity_id: "{{ media_player }}" + - if: "{{ action == 'previous_track' }}" + then: + - service: media_player.media_previous_track + target: + entity_id: "{{ media_player }}" + - if: "{{ action == 'volume_set' }}" + then: + - service: media_player.volume_set + target: + entity_id: "{{ media_player }}" + data: + volume_level: "{{ volume_level / 100 }}" + - if: "{{ action == 'shuffle_on' }}" + then: + - service: media_player.shuffle_set + target: + entity_id: "{{ media_player }}" + data: + shuffle: true + - if: "{{ action == 'shuffle_off' }}" + then: + - service: media_player.shuffle_set + target: + entity_id: "{{ media_player }}" + data: + shuffle: false ``` @@ -225,13 +243,20 @@ Add this to your Extended OpenAI Conversation system prompt: ``` Music Playback: - When asked to play music, follow this two-step process: 1) Call search_spotify to get the Spotify URI, 2) Call play_music with the URI and media player entity -- Available media players: [list your Spotify Connect devices here, e.g., "media_player.kitchen_speaker, media_player.living_room_sonos"] -- Default speaker: If no speaker is specified, use media_player.kitchen_speaker (change this to your preferred default) +- Available media players: media_player.kitchen_speaker, media_player.gaming_room_speaker, media_player.office_speaker +- Default speaker: If no speaker is specified, use media_player.kitchen_speaker - Parse commands like "Play {Artist/Album/Track}" and determine the media type automatically -- For playback control (pause, skip, volume), use the control_playback function +- IMPORTANT: When playing an artist, always enable shuffle after starting playback by calling control_playback with action: shuffle_on to create a dynamic playlist experience +- For playback control (pause, skip, volume, shuffle), use the control_playback function ``` -**Customize the default speaker:** Replace `media_player.kitchen_speaker` with your preferred media player entity ID. The LLM will use this when you say "Play Coldplay" without specifying a location. +**Customization tips:** +- **Entity IDs are descriptive** (like `media_player.kitchen_speaker`): Just list the entity IDs as shown above +- **Entity IDs are cryptic** (like `media_player.wiim_12345`): Include friendly names for better LLM understanding: + ``` + - Available speakers: Kitchen Speaker (media_player.wiim_12345), Gaming Room (media_player.sonos_abc), Office (media_player.cast_xyz) + ``` +- **Default speaker:** Change `media_player.kitchen_speaker` to your preferred default location ### Done! Try It @@ -242,11 +267,12 @@ Music Playback: - "Pause the music" - "Skip to the next track" - "Set volume to 50%" +- "Shuffle on" **Natural Language (LLM advantage):** - "I'm in the mood for some Coldplay" - "Put on that song Yellow" -- "Start playing Coldplay" (uses default speaker from prompt) +- "Start playing Coldplay" (uses default speaker from prompt, automatically shuffles artist) - "Can you play the Parachutes album?" - "Play me something by Coldplay" - "I want to hear some chill music from Coldplay" @@ -258,6 +284,7 @@ Music Playback: - "Turn it down a bit" (instead of "Set volume to 30") - "Next song please" (instead of "Skip to next track") - "Stop the music" (instead of "Pause playback") +- "Shuffle this" (instead of "Set shuffle to on") > **Note:** When no speaker is specified, the LLM will use the default media player defined in your Extended OpenAI Conversation prompt. Add instructions like "If no speaker specified, use media_player.kitchen_speaker" to your system prompt. @@ -329,31 +356,39 @@ This integration works with **any Spotify Connect-compatible device**: **Note:** If your device runs Spotify and shows up in Home Assistant's integrations, it will work with this integration. -## Use With Music Assistant +## Works Alongside Music Assistant -**You can (and should!) use both Music Assistant and this integration together.** They complement each other perfectly: +Music Assistant and this integration serve different purposes and work well together: ### Music Assistant -- **Best for:** Rich UI-based music control in dashboards and automations -- **Strengths:** Multi-room audio, queue management, playlist management, beautiful UI -- **Voice support:** Built-in voice commands work with Home Assistant's default voice pipeline -- **Limitation:** Voice commands require specific sentence patterns and don't work with custom voice assistants (Extended OpenAI Conversation, custom LLMs, etc.) +- **Purpose:** Full-featured music server with rich UI, multi-room audio, queue management +- **Architecture:** Music server that aggregates multiple providers (Spotify, local files, streaming services) +- **Voice support:** [Built-in voice integration](https://github.com/music-assistant/voice-support) works with Home Assistant Assist (with or without LLM enhancement) +- **Best for:** Complete music management solution with visual control ### This Integration (Spotify Voice Assistant) -- **Best for:** Natural language voice control with custom voice assistants -- **Strengths:** Works with Extended OpenAI Conversation, fully natural language, exact artist matching -- **Use case:** "Play some Coldplay in the kitchen" → LLM interprets naturally and plays music -- **Limitation:** Not a full music server, just search + playback control +- **Purpose:** Lightweight Spotify search service for conversation agents with function calling support +- **Architecture:** Direct Spotify Connect integration via Home Assistant's official Spotify integration (no music server required) +- **Works with:** Any conversation agent that supports function calling (Extended OpenAI Conversation, OpenAI Conversation, Google Generative AI Conversation, etc.) +- **Best for:** Adding Spotify voice control without running a full music server -### Why Use Both? -Many users (including the author) run both together: -- **Music Assistant** provides the UI for manual control, queue management, and automations -- **This integration** handles natural language voice commands via custom LLM assistants -- **They work together seamlessly:** Music started via voice appears in Music Assistant's UI and remains fully controllable there +### Using Both Together -Example workflow: Say "Play Coldplay on office speaker" → music starts → open Music Assistant UI → see what's playing, adjust volume, skip tracks, manage queue - all through MA's interface. +Many users run both simultaneously: +- **Music Assistant:** Provides UI for manual control, queue management, and automations (skip voice setup) +- **This integration:** Handles voice control via custom conversation agents +- **They work together:** Music started via voice appears in Music Assistant UI and remains fully controllable there -They don't conflict because they access the same underlying Spotify Connect devices. This integration just provides the search service that custom voice assistants need to find music. +**Example workflow:** Say "Play Coldplay on office speaker" → music starts → open Music Assistant UI → see what's playing, adjust volume, skip tracks, manage queue. + +They don't directly integrate but work together because both control the same Spotify Connect devices independently. + +### Choose Your Setup + +- **Voice only:** Use this integration alone +- **Voice + rich UI:** Use both (Music Assistant without voice + this integration) +- **Home Assistant Assist voice:** Use Music Assistant's [built-in voice support](https://github.com/music-assistant/voice-support) +- **Custom conversation agent voice:** Use this integration ## How It Works @@ -535,16 +570,52 @@ This shows search queries, match types (exact vs. first result), and any errors. 3. Test playing Spotify directly to device from Spotify app 4. Verify Spotify Premium account is active +## Performance + +The integration itself is highly optimized with client caching (15-50x faster after first search). However, overall voice command response time is dominated by your conversation agent's LLM processing. + +### Response Time Breakdown + +Typical "Play Coldplay" command: +- **Spotify search:** ~500ms first time, ~10ms cached ✅ Fast +- **LLM processing:** 2-10+ seconds per function call ⚠️ Bottleneck +- **Total:** Depends almost entirely on your LLM + +### Optimization Tips + +**1. Choose a faster LLM** +- **Slow (2-5 min):** Large models on CPU (qwen3:8b, llama3:8b) +- **Medium (5-15 sec):** Small local models (qwen3:1.8b, phi3:mini) +- **Fast (2-5 sec):** Cloud APIs (OpenAI GPT-4o-mini, Anthropic Claude Haiku) + +**2. Reduce exposed devices** + +Each device in your conversation agent adds tokens to every LLM call, slowing processing: +- **38 devices:** ~1,600 tokens = slower responses +- **25 devices:** ~1,000 tokens = 20-30% faster + +**Safe to remove from Extended OpenAI Conversation:** +- Voice satellite media players (if not directly controlled) +- Unavailable/unused devices +- Status sensors not needed for voice commands + +**3. Enable GPU acceleration** + +If using Ollama locally, GPU acceleration provides 5-10x speedup over CPU. + +### Expected Performance + +| Configuration | Response Time | +|---------------|---------------| +| Cloud API (GPT-4o-mini) + 25 devices | 2-5 seconds ✅ | +| Local GPU (qwen3:8b) + 25 devices | 10-30 seconds | +| Local CPU (qwen3:8b) + 38 devices | 2-5 minutes ❌ | + +**Bottom line:** This integration adds <1 second overhead. LLM choice determines user experience. + ## Roadmap -- [ ] Playlist search support -- [ ] Configuration options (search limits, match behavior) -- [ ] Search result caching for faster responses -- [ ] Support for multiple search result types -- [ ] Album/track exact name matching -- [ ] Integration with HA Assist (native conversation) -- [ ] Favorite/saved content quick access -- [ ] Multi-room playback examples +See [TODO.md](TODO.md) for detailed enhancement proposals and implementation plans. ## Contributing diff --git a/examples/extended_openai_functions.yaml b/examples/extended_openai_functions.yaml index 2cffbf1..cbdf0a9 100644 --- a/examples/extended_openai_functions.yaml +++ b/examples/extended_openai_functions.yaml @@ -53,13 +53,13 @@ - spec: name: control_playback - description: Control music playback (pause, resume, stop, next track, previous track, set volume) + description: Control music playback (pause, resume, stop, next track, previous track, set volume, shuffle) parameters: type: object properties: action: type: string - enum: [pause, play, stop, next_track, previous_track, volume_set] + enum: [pause, play, stop, next_track, previous_track, volume_set, shuffle_on, shuffle_off] description: Playback control action media_player: type: string @@ -118,3 +118,21 @@ entity_id: "{{ media_player }}" data: volume_level: "{{ volume_level / 100 }}" + - conditions: + - condition: template + value_template: "{{ action == 'shuffle_on' }}" + sequence: + - service: media_player.shuffle_set + target: + entity_id: "{{ media_player }}" + data: + shuffle: true + - conditions: + - condition: template + value_template: "{{ action == 'shuffle_off' }}" + sequence: + - service: media_player.shuffle_set + target: + entity_id: "{{ media_player }}" + data: + shuffle: false diff --git a/examples/system_prompt.txt b/examples/system_prompt.txt index 5fdac42..a6ac1c8 100644 --- a/examples/system_prompt.txt +++ b/examples/system_prompt.txt @@ -2,5 +2,6 @@ Music Playback: - When asked to play music, follow this two-step process: 1) Call search_spotify to get the Spotify URI, 2) Call play_music with the URI and media player entity - Available media players: media_player.kitchen_speaker, media_player.living_room_speaker (customize with your actual entity IDs) - Parse commands like "Play {Artist/Album/Track}" and determine the media type automatically (artist, album, or track) -- For playback control (pause, skip, volume), use the control_playback function +- IMPORTANT: When playing an artist, always enable shuffle after starting playback by calling control_playback with action: shuffle_on to create a dynamic playlist experience +- For playback control (pause, skip, volume, shuffle), use the control_playback function - Always use natural language interpretation - users will say things like "play Coldplay" not "search for artist Coldplay"