GitHub
🦆🧑‍🦯 QuackHack-McBlindy blog
🦆

Navigation

🧑‍🦯 ⮟
🦆 says ⮞ qwack qwack, plx pick!

Defining Your Home

Part 5 - Writing a Client With Voice Commands

says ⮞ I use 20x magnification when I code and debug. I use emoji to simplify logs for myself. If you can't handle my code style you can disable most of it on this website by toggling the button in the navbar. Shall duck continue?

Hello Voice Control


If you have followed along these past pages you can already control your devices using commands:

zig - For controlling devices.
scene - For setting scenes.

They are both basic commands and might not be optimal, let's write a proper one.
This part is going to be voice commands! focused.
Of course you can do this in multiple other ways if you don't want to go this route, but I am going to assume that you are using yo for this time.

This voice controlled Zigbee light controller will be able to handle the device in every single possible way.


I will split it into three phases, I think the parameters section and the entity lists might get intense, so hold on.

says ⮞ let'z call da script DuckBee, DB - ok let'z qwaaaack.

Phase 1: Yo! Let's Definine DB!

We start by giving it a name, description, alias etc.
We will also define the parameters for the Zigbee Voice Controller.

⮞ View yo.scripts.DuckBee
 

  yo.scripts.DuckBee = {
    description = "DuckBee Control lights and other home automatioon devices";
    category = "🛖 Home Automation";
    aliases = [ "DB" ];
    autoStart = false;
    logLevel = "DEBUG";
    parameters = [   
      { name = "device"; description = "Device to control"; optional = true; }
      { name = "state"; type = "string"; description = "State of the device or group"; } 
      { name = "brightness"; description = "Brightness value of the device or group"; optional = true; type = "int"; }    
      { name = "color"; description = "Color to set on the device"; optional = true; }    
      { name = "temperature"; description = "Light color temperature to set on the device"; optional = true; }          
      { name = "scene"; description = "Activate a predefined scene"; optional = true; }     
      { name = "room"; description = "Room to target"; optional = true; }        
      { name = "user"; description = "Mosquitto username to use"; default = "mqtt"; }    
      { name = "passwordfile"; description = "File path containing password for Mosquitto user"; default = config.sops.secrets.mosquitto.path; }
      { name = "flake"; description = "Path containing flake.nix"; default = config.this.user.me.dotfilesDir; }
      { name = "pair"; type = "bool"; description = "Activate zigbee2mqtt pairring and start searching for new devices"; default = false; }
      { name = "cheapMode"; type = "bool"; description = "Energy saving mode. Turns off the lights again after X seconds."; default = false; }
    ];

Phase 2: Yo Bash! Take it from here!

Here you can do basically anything you want.
I hpwever, are gpomg to show how to control your devices!

⮞ View yo.scripts.DuckBee.code
 

      ${cmdHelpers}
 #     set -euo pipefail
      # 🦆 says ⮞ create case insensitive map of device friendly_name
      declare -A device_map=( ${lib.concatStringsSep "\n" (lib.mapAttrsToList (k: v: "['${lib.toLower k}']='${v}'") normalizedDeviceMap)} )
      available_devices=( ${toString deviceList} )      
      DOTFILES="$flake"
      STATE_DIR="${zigduckDir}"
      DEVICE="$device"
      STATE="$state"
      SCENE="$scene"
      BRIGHTNESS="$brightness"
      COLOR="$color"
      TEMP="$temperature"
      MQTT_BROKER="${mqttHostIp}"
      PWFILE="$passwordfile"
      MQTT_USER="$user"
      MQTT_PASSWORD=$(cat "$PWFILE")
      ROOM="$device"
      touch "$STATE_DIR/voice-debug.log"        
      if [[ "$cheapMode" == "true" ]]; then
        reset_room_timer "$ROOM"
        dt_info "Restarting room timer for $ROOM" 
        exit 0
      fi
      if [[ -n "$SCENE" && -z "$DEVICE" ]]; then
        scene "$SCENE"
        exit 0
      fi
      
      if [[ "$pair" == "true" ]]; then
        echo "🦆 Activating Zigbee2MQTT pairing mode for 120 seconds..."
        mqtt_pub -t "zigbee2mqtt/bridge/request/permit_join" -m '{"value": true, "time": 120}'    
        dt_info "📡 Searching for new Zigbee devices... Put your device in pairing mode now!"
        dt_info "⏰ Pairing mode! 120 sec... (Ctrl+C to stop early)"
        cleanup() {
          dt_debug "Disabling pairing mode..."
          mqtt_pub -t "zigbee2mqtt/bridge/request/permit_join" -m '{"value": false}'
          exit 0
        }
        trap cleanup INT TERM EXIT
        ${pkgs.mosquitto}/bin/mosquitto_sub -h "$MQTT_BROKER" -u "$MQTT_USER" -P "$MQTT_PASSWORD" \
        -t "zigbee2mqtt/bridge/event" -t "zigbee2mqtt/bridge/log" | while IFS= read -r line; do
        
        dt_debug "Received: $line"
                   
        if echo "$line" | jq -e '.type == "device_joined"' > /dev/null 2>&1; then
          device_data=$(echo "$line" | jq -r '.data')
          friendly_name=$(echo "$device_data" | jq -r '.friendly_name')
          ieee_address=$(echo "$device_data" | jq -r '.ieee_address')
          echo "✅ New device joined: $friendly_name ($ieee_address)"
        fi
                
        if echo "$line" | jq -e '.type == "device_interview"' > /dev/null; then
          interview_data=$(echo "$line" | jq -r '.data')
          status=$(echo "$interview_data" | jq -r '.status')
          ieee_address=$(echo "$interview_data" | jq -r '.ieee_address')
                    
          if [[ "$status" == "successful" ]]; then
            model=$(echo "$interview_data" | jq -r '.definition.model // "unknown"')
            vendor=$(echo "$interview_data" | jq -r '.definition.vendor // "unknown"')
            description=$(echo "$interview_data" | jq -r '.definition.description // "unknown"')      
            dt_info "🎉 Device interview successful!"
            dt_info "Model: $model"
            dt_info "Vendor: $vendor" 
            dt_info "Description: $description"
            dt_info "IEEE: $ieee_address"
                        
            cat << EOF
🦆 says ⮞ To add this device to your Nix configuration, add to `house.zigbee.devices`:

''${ieee_address} = {
  friendly_name = "$friendly_name";
  room = "unknown"; # ⮜ 🦆 says ⮞ Set the room name
  type = "unknown"; # ⮜ 🦆 says ⮞ Set device type (light, dimmer, sensor, motion, outlet, remote, pusher, blind)
  endpoint = 11;     # ⮜ 🦆 says ⮞ Set endpoint if needed
  icon = "mdi:toggle-switch"; # ⮜ 🦆 says ⮞ Set icon for frontend
  batteryType = "CR2450"; }; # ⮜ 🦆 says ⮞ Optional option if device has a battery. (AAA, CR1, CR2032, CR2450) 
};

EOF
            elif [[ "$status" == "failed" ]]; then
              dt_warning "❌ Device interview failed for $ieee_address"
            fi
          fi
        
          if echo "$line" | jq -e '.message != null' > /dev/null 2>&1; then
            message=$(echo "$line" | jq -r '.message')
            level=$(echo "$line" | jq -r '.level // "info"')
            
            if [[ "$level" == "info" ]]; then
              dt_info "Bridge: $message"
            elif [[ "$level" == "warning" ]]; then
              dt_warning "Bridge: $message"
            elif [[ "$level" == "error" ]]; then
              dt_error "Bridge: $message"
            else
              dt_debug "Bridge: $message"
            fi
          fi
        done
    
        cleanup
      fi
      if [[ "$DEVICE" == "ALL_LIGHTS" ]]; then
        if [[ "$STATE" == "ON" ]]; then
          scene max
          if_voice_say "Jag maxade alla lampor brorsan."
        elif [[ "$STATE" == "OFF" ]]; then
          scene dark
          if_voice_say "Nu blev det mörkt!"
        else
          echo "$(date) - ❌ Unknown state for all_lights: $STATE" >> "$STATE_DIR/voice-debug.log"
          say_duck "❌ Unknown state for all_lights: $STATE"
          exit 1
        fi
        exit 0
      fi
      control_device() {
        local dev="$1"
        local state="$2"
        local brightness="$3"
        local color_input="$4"      
        local hex_code=""
        if [[ "$dev" == "Smoke Alarm Kitchen" ]]; then
          dt_info "$dev is a sensor, exiting"
          return 0
        fi
        if [[ -n "$color_input" ]]; then
          if [[ "$color_input" =~ ^#[0-9a-fA-F]{6}$ ]]; then
            hex_code="$color_input"
          else
            hex_code=$(color2hex "$color_input") || {
              echo "$(date) - ❌ Unknown color: $color_input" >> "$STATE_DIR/voice-debug.log"
              say_duck "fuck ❌ Invalid color: $color_input"
              exit 1
            }
          fi
        fi
        
        if [[ -n "$SCENE" ]]; then
          scene $SCENE
          say_duck "Activated scene $SCENE"
        fi   
        
        if [[ "$state" == "off" ]]; then
          mqtt_pub -t "zigbee2mqtt/$dev/set" -m '{"state":"OFF"}'
          say_duck "Turned off $dev"
          if_voice_say "Stängde av $dev"
        else
          # 🦆 says ⮞ Validate brightness value
          if [[ -n "$brightness" ]]; then
            if ! [[ "$brightness" =~ ^[0-9]+$ ]] || [ "$brightness" -lt 1 ] || [ "$brightness" -gt 100 ]; then
              echo "Unknown brightness: $brightness" >> "$STATE_DIR/voice-debug.log"
              say_duck "Ogiltig ljusstyrka: $brightness%. Ange 1-100."
              exit 1
            fi
            brightness=$((brightness * 254 / 100))
          fi
          local payload='{"state":"ON"'
          [[ -n "$brightness" ]] && payload+=", \"brightness\":$brightness"
          [[ -n "$hex_code" ]] && payload+=", \"color\":{\"hex\":\"$hex_code\"}"
          payload+="}"
          mqtt_pub -t "zigbee2mqtt/$dev/set" -m "$payload"
          say_duck "Set $dev: $payload"
          if_voice_say "Klart kompis"
        fi
      }
      
      if [[ -n "$DEVICE" ]]; then
        input_lower=$(echo "$DEVICE" | tr '[:upper:]' '[:lower:]')
        exact_name="''${device_map["''$input_lower"]:-}"   
        if [[ -n "$exact_name" ]]; then
          control_device "$exact_name" "$STATE" "$BRIGHTNESS" "$COLOR"
          exit 0

        else
          for dev in "''${!device_map[@]}"; do
            if [[ "$dev" == *"$input_lower"* ]]; then
              exact_name="''${device_map[$dev]}"
              break
            fi
          done
          
          if [[ -n "$exact_name" ]]; then
            control_device "$exact_name" "$STATE" "$BRIGHTNESS" "$COLOR"
            exit 0
          fi
          
          group_topics=($(jq -r '.groups | keys[]' "$STATE_DIR/zigbee_devices.json"))
          for group in "''${group_topics[@]}"; do
            if [[ "$(echo "$group" | tr '[:upper:]' '[:lower:]')" == *"$input_lower"* ]]; then
              control_group "$group" "$STATE" "$BRIGHTNESS" "$COLOR"
              exit 0
            fi
          done
             
          AREA="$DEVICE"
          say_duck "⚠️ Device '$DEVICE' not found, trying as area '$AREA'"
          echo "$(date) - ⚠️ Device $DEVICE not found as area" >> "$STATE_DIR/voice-debug.log"
        fi
      fi
            
      control_room() {
        local room="$1"
        if [[ -z "$room" ]]; then
          echo "Usage: control_room "
          return 1
        fi
      
        readarray -t devices < <(nix eval $DOTFILES#nixosConfigurations.desktop.config.house --json \
          | jq -r --arg room "$room" '
              .zigbee.devices
              | to_entries
              | map(select(.value.room == $room and .value.type == "light"))
              | map(.value.friendly_name)
              | .[]')
      
        for light_id in "''${devices[@]}"; do
          local hex_code=""
               
          if [[ -n "$COLOR" ]]; then
            hex_code=$(color2hex "$COLOR") || {
              echo "$(date) - ❌ Unknown color: $COLOR" >> "$STATE_DIR/voice-debug.log"
              say_duck "❌ Invalid color: $COLOR"
              continue
            }
          fi
      
          local payload='{"state":"ON"'
          [[ -n "$BRIGHTNESS" ]] && payload+=", \"brightness\":$BRIGHTNESS"
          [[ -n "$hex_code" ]] && payload+=", \"color\":{\"hex\":\"$hex_code\"}"
          payload+="}"
      
          if [[ "$light_id" == *"Smoke"* || "$light_id" == *"Sensor"* || "$light_id" == *"Alarm"* ]]; then
            echo "Skipping invalid light device: $light_id" >> "$STATE_DIR/voice-debug.log"
            continue
          fi

      
          mqtt_pub -t "zigbee2mqtt/$light_id/set" -m "$payload"
          say_duck "$light_id $payload"
        done
      }
      
      if [[ -n "$AREA" ]]; then
        normalized_area=$(echo "$AREA" | tr '[:upper:]' '[:lower:]' | tr -d '[:space:]')
        control_room $AREA
      fi        
    '';

Phase 3: Yo! Define DB Sentences!

We must first define sentences.
Then we map the entity lists for the parameter resolution.


REMEMBER:

(alternative|words) - These words are alternative words, one must be said.
[optional|words] - Any of these words can be said. But not rquired.
{parameters}- These are parameters. If the parameter is wildcard, it can be anything.


⮞ View yo.scripts.DuckBee.voice
 

  voice = {
      priority = 1;
      sentences = [
        "(turn on|switch on|activate|start|power on) {device}"
        "(turn off|switch off|deactivate|stop|power off) {device}"
        "(turn|switch) {device} {state}"
        "(turn|switch) {state} {device}"
        "{device} {state} and brightness {brightness} percent"
        "{device} {state} with {color} color and {brightness} percent brightness"
        "{device} {state} in {room} and [change] color [to] {color} [and] brightness [to] {brightness} percent"

        # 🦆 says ⮞ simple room-light control
        "{state} (all|every) light[s] in {room}"
        "{state} {room} light[s]"
        "{state} (all|every) light[s]"
        "{state} (all|every) {device} light[s]"
        "control (all|every) light[s] in {room}"
        "manage {room} lighting"

        # 🦆 says ⮞ color control
        "(change|set|adjust) {device} color to {color}"
        "(change|set|adjust) color of {device} to {color}"
        "make {device} {color}"
        "set {device} to {color} color"
        "apply {color} color to {device}"

        # 🦆 says ⮞ brightness control
        "(adjust|set|change) {device} brightness to {brightness} percent"
        "(adjust|set|change) brightness of {device} to {brightness} percent"
        "make {device} {brightness} percent brightness"
        "set {device} to {brightness} percent brightness"
        "dim {device} to {brightness} percent"
        "brighten {device} to {brightness} percent"

        # 🦆 says ⮞ scene/ambiance control
        "create {scene} (scene|mode|atmosphere) in {room}"
        "set {room} to {scene} (scene|mode|atmosphere)"
        "activate {scene} mode"

        # 🦆 says ⮞ multi taskerz (complex, strict patterns LAST)
        "set {room} lights to {state} with {color} at {brightness} percent and activate {scene}"
        "{device} {state} in {room} with {color} color at {brightness} percent and temperature {temperature} using scene {scene}"
        "configure {device} in {room} with {color} color, {brightness} percent brightness, and {temperature} temperature"
        "create {scene} atmosphere in {room} using {color} lighting at {brightness} percent [in cheap mode]"
        "adjust all lights in {room} to {color} with {brightness} percent brightness and set {temperature} temperature"

        # 🦆 says ⮞ pairing mode
        "{pair} [new] [zigbee] (device|devices|sensor|sensors)"
        "start {pair} [new] device[s]"
        "enable {pair} mode for new devices"
        "begin device {pair}"
      ];        
      lists = {
        state.values = [
          { "in" = "[tänd|tända|tänk|start|starta|på|tönd|tömd]"; out = "ON"; }             
          { "in" = "[släck|släcka|slick|av|stäng|stäng av]"; out = "OFF"; } 
        ];
        brightness.values = builtins.genList (i: {
          "in" = toString (i + 1);
          out = toString (i + 1);
        }) 100 ++ [
          { "in" = "[full|maximum|max|hundred]"; out = "100"; }
          { "in" = "[half|fifty|medium]"; out = "50"; }
          { "in" = "[quarter|twenty five|low]"; out = "25"; }
          { "in" = "[ten percent|minimal|dim]"; out = "10"; }
          { "in" = "[five percent|minimum|min]"; out = "5"; }
          { "in" = "[zero|off|dark]"; out = "0"; }
        ];
# 🦆 says ⮞ automatically add all zigbee devices  
        device.values = let
          reservedNames = [ "hall" "kitchen" "bedroom" "bathroom" "wc" "livingroom" "kitchen" "switch" "all" "every" ];
          sanitize = str:
            lib.replaceStrings [ "/" " " ] [ "" "_" ] str;
        in [
          { "in" = "[living room|livingroom|livingroom|main room|front room]"; out = "livingroom"; }
          { "in" = "[kitchen|cooking area|kitchen area]"; out = "kitchen"; }
          { "in" = "[bedroom|sleeping room|master bedroom]"; out = "bedroom"; }
          { "in" = "[bathroom|restroom|washroom]"; out = "bathroom"; }
          { "in" = "[hallway|hall|corridor|passage]"; out = "hallway"; }
          { "in" = "[all|every|everything|all lights]"; out = "ALL_LIGHTS"; }    
        ] ++
        (lib.filter (x: x != null) (
          lib.mapAttrsToList (_: device:
           let
              baseRaw = lib.toLower device.friendly_name;
              base = sanitize baseRaw;
              baseWords = lib.splitString " " base;
              isAmbiguous = lib.any (word: lib.elem word reservedNames) baseWords;
              hasLampSuffix = lib.hasSuffix "lamp" base;
              lampanVariant = if hasLampSuffix then [ "${base}s" "${base} light" ] else [];  
              enVariant = [ "${base}s" "${base} light" ]; # English variants
              variations = lib.unique (
                [
                  base
                  (sanitize (lib.replaceStrings [ " " ] [ "" ] base))
                  (lib.replaceStrings [ "_" ] [ " " ] base)
                ] ++ lampanVariant ++ enVariant
              );
            in if isAmbiguous then null else {
              "in" = "[" + lib.concatStringsSep "|" variations + "]";
              out = device.friendly_name;
           }
          ) zigbeeDevices
        ));      
        # 🦆 says ⮞ color yo        
        color.values = [
          { "in" = "[red|red color|crimson]"; out = "red"; }
          { "in" = "[green|green color|emerald]"; out = "green"; }
          { "in" = "[blue|blue color|azure]"; out = "blue"; }
          { "in" = "[yellow|yellow color|golden]"; out = "yellow"; }
          { "in" = "[orange|orange color|amber]"; out = "orange"; }
          { "in" = "[purple|purple color|violet]"; out = "purple"; }
          { "in" = "[pink|pink color|rose]"; out = "pink"; }
          { "in" = "[white|white color|pure white]"; out = "white"; }
        ];      
        pair.values = [
          { "in" = "[pair|pairing|discover|scan]"; out = "true"; }
        ];

Something like that maybe? Sorry English is not my native language.
But you get the point.

Full Source Code

View source code on GitHub

Keep Reading

Part 1. The module, the options and defining devices
Part 2. Configure your Mosquitto/Z2MQTT
Part 3. Nix Configured Automations
Part 4. Writing a Server Service - in Rust
Part 5. Writing a Client - With Voice Commands ⮜🦆here u are
Part 6. The Auto-Generated Dashboard


Comments on this blog post