SEPIA-Home v2.7.0 with custom theme editor and Home Assistant support

v2.7.0 - 2022.11.03 | [EDIT 2022.11.29: Android APK update]

Important: If you're updating from a clean v2.6.0 using the integrated script please read this issue: #155.

SEPIA-Home v2.7.0 extends customization options, improves smart-home interfaces, adds Home Assistant support, enables task-based ASR models and implements a lot of smaller improvements and fixes. Here are some highlights in detail:

The custom theme editor 📱🎨🖌️ allows you to create your own look for SEPIA. Based on (currently) 2 templates (classic light, neo dark ⚪⚫) you can change the color of almost every component (background + image, buttons, menus, etc.). When you're ready you can export your theme file to store and share it 🧑‍🎨.

You can now control your Home Assistant smart-home devices via SEPIA! 🗣️🤖💡:house:. Check out the new Home Assistant setup page to get started.

To improve the user-experience for example when using the cloud-free SEPIA STT-Server ☁️🚫, there is a new task-property integrated into the client and the SEPIA smart-services (check STT settings page ⚙️). It can be used (among other things) to hot-swap the active ASR model and load one that is optimized for a specific 'task' like 'navigation' 🚗 , 'web-search' 🔍, 'music' 🎵 etc. (optimized, official models will follow soon).

The updated Mary-TTS integration should make it easier to use Text-To-Speech systems with a compatible API like Mimic3 by Mycroft 🤖💬 (simply replace your Mary-TTS server with Mimic3 or use it as "external" server via app settings).

As usual the release has a lot more to discover, so check-out the extended release notes or the "shorter" version below! Hope you enjoy the update ✌️😃.

Download SEPIA-Home - Download Android APK - See Twitter news feed

[EDIT] Android APK has been updated at 2022.11.29 to fix Android intent-based media controls

Release notes (short) - Extended version: here

General updates and changes to all SEPIA-Home servers:

• Breaking change: Java 11 is required now and Java 8 support has been dropped. Please UPDATE TO JAVA 11 if you're still using 8!
• Breaking change: Check out the new server settings options 'languages_preload' (default: en, de) and 'allow_non_commercial_features' (default: false) below
• New SEPIA-Home server start-page to quickly access apps, check status info and get help with issues. The page can be found at the proxy root URL http(s)://[my-sepia-home-ip]:20726 (default) after a clean Nginx setup (or manual update)
• Optimized Elasticsearch v5.3.3 database to run on Windows 11
  ...

Updated client to v0.25.0:

• New custom skin feature and theme editor, including one light and dark template
• Added better support for safe-areas of display (e.g. iPhone notch) and ability to add new custom borders
• New 'dialog_task' variable to be able to optimize client and services depending on defined 'tasks'
• Added support for STT 'task'-based custom models and improved STT settings view
• Improved widgets (custom views/frames) interface and added support for per-widget 'activitySettings' like language and task
• New time-event card icons to indicate "next", "expired" and "faulty"
• Android: Completely refactored and optimized build setup + set official target to Android 12L (NOTE: min. Android is 5.1 now!)
• Android: Added support for TTS native-engine voice selection
• iOS PWA: Greatly improved PWA experience on iOS with fixes for splash-screen, safe-areas, overscrolling, sound initialization, error handling and more. Safari is still buggy though.
• Optimized landscape-mode for Teach-UI
• Allow abort of 'setup-mode' to enable on-screen login (e.g. via virtual keyboard) in DIY clients (usually in 'pseudo-headless' mode)
  ...

Updated Control-HUB (admin-tools) to v1.5.0:

• Smart-home: Improved UI, animations and device cards + new reload time indicator
• Smart-home: Added 'Home Assistant' support (BETA) and new device types ('fan', 'garage_door', 'air_conditioner' and 'temperature_control')
  ...

Updated Assist-server to v2.7.0:

• Added 'Home Assistant' smart-home HUB interface (BETA) and improved existing interfaces (e.g. auto-detection of openHAB devices)
• Support for new smart-home devices: 'fan', 'garage_door', 'air_conditioner' and 'temperature_control'
• Updated 'SmartHomeHubConnector' to handle device type groups (first group: temperature). Allows to better handle similar devices like 'air_conditioner' + 'heater'
• Introduced new smart-home interface device configuration to handle complex HUB APIs and complement 'set commands' to control devices
• Improved code of several smart-home HUB integrations (openHAB, FHEM, ioBroker, test-HUB etc.)
• Improved parsing of Mary-TTS API compatible voices to support engines like Mimic3 + updated Mary-TTS INSTALL info
• Implemented new 'InterviewMetaData' feature with 'dialog_task' support enabling services to optimize client settings (e.g. switching task-based ASR models)
• Several TTS pronunciation tweaks for example to better handle units like kWh, Wh, kW and W and fixes for 'txt2pho' engine
• Several NLU tweaks and fixes, e.g for 'smart_device' parameters or 'Alarms' service and language switching
• Improved 'action' NLU parameter by splitting 'open/close' from 'on/off' and adding 'cancel'
• New server settings option 'languages_preload' (default: en, de), controlling for example what language files are pre-loaded at start-up
• New server settings option 'allow_non_commercial_features' (default: false), to globally handle specific license restrictions. NOTE: Please set it to true to use 'MBROLA' and 'txt2pho' TTS engines
• Disable sentences import into DB by default during setup (can be run manually instead)
• Added new calculator library and tool to parse mathematical expressions from strings (used e.g. for smart-home state calculations)
• Updated Elasticsearch mappings for 'smart-devices' and 'smart-interfaces' to better handle meta data (NOTE: this will not automatically apply to old installations)
• Added Italian translations of 'Assistant' files (answers, chats, teachIt) thanks to @giostark
  ...

Updated Core-tools to v2.3.0:

• Improved private network detection
• Updated spark-core to v2.9.4
  ...

Other servers, tools and common changes:

• Updated SEPIA STT-Server to v1.0.0 to support Coqui-STT and 'tasks'
• Tweaks and smaller fixes for SEPIA-Home build and run scripts
• Updated Teach-Server to v2.3.0 and slightly improved common.json for Teach-UI
• Updated WebSocket Chat-Server to v1.4.0
• Updated SDK to support new SEPIA-Home v2.7.0 (NOTE: please run mvn clean once after update!)
• Updated Mesh-Node to v0.10.0 (no other changes except updated dependencies)

SEPIA-Home v2.6.2 with a lot of stability + usability improvements and a new automatic setup mode

v2.6.2 - 2022.05.08

Important: If you're updating from a clean v2.6.0 using the integrated script please read this issue: #155.

SEPIA-Home v2.6.2 improves many of the existing and new features of v2.6.1, fixes a lot of smaller issues and introduces a new automatic setup (with YAML config) and external data folder to better support Docker containers. Here are some other highlights:

The remote media control feature has been enhanced to support volume adjustment and the client will remember this setting after a restart. In addition the volume control will now work for the Youtube widget as well 📻 🎶.

Teach-UI commands and app settings can now be exported as JSON files and imported via drag & drop (or file select) 🗄️ 🤓 .

The news region can now be defined independently from the app language and news outlets can be configured per language + region (news-outlets.json). NLU supports "show me news in German" etc. as well 📰 👀.

There is a very nice new avatar + new skins 🤖 🎨 🙂 and Porcupine wake-words have been updated to v2.1 (if you have a custom 2.0 ww use the included download scripts).

As usual the release has a lot more to discover, so check-out the extended release notes or the "shorter" version below! Hope you enjoy the update ✌️ 😃.

Download SEPIA-Home - Download Android APK - See Twitter news feed

Release notes (short) - Extended version: here

Updated client to v0.24.2:

• Added virtual keyboard for DIY clients with touchscreen + 'virtualKeyboard' URL parameter
• Remember media player volume and added option 'playerVolume' to settings file
• Added Teach-UI basic command import/export + support for file drag & drop
• Added new settings option for 'defaultNewsRegion' + append 'defaultNewsRegion' and 'appRegionCode' to assistant requests
• New avatar 'Proto' + improved avatar code
• New skins 'Blue Canary', 'Odyssey AI' and 'Vectors Reloaded' + style tweaks
• Added CLEXI GPIO events for audio 'eventEffectsOn' and 'eventEffectsOff' + fixed a bug in LED control
• Updated Porcupine to v2.1 (replaced v2.0)
  ...

Updated Control-HUB (admin-tools) to v1.4.3:

• Improved smart-home page with better UX for authentication data and tweaked some styles
  ...

Updated Assist-server to v2.6.2:

• New automatic-setup feature using YAML config file to define setup tasks, users and DNS + improved setup code (create random passwords, write log etc.)
• Added a new TTS endpoint to support byte-range streaming audio-files (required by Apple devices and potantially faster for large files)
• Added installer for 'txt2pho' TTS engine (German MBROLA only) + tweaked some voice speeds
• Added 'appRegionCode' and new 'defaultNewsRegion' to NluInput class (to be used in services)
• Improved news service to support 'language' parameter (e.g.: "show me news in German") + 'defaultNewsRegion' and 'appRegionCode' to make news independent of user language
  ...

Updated Core-tools to v2.2.11:

• Added new 'readYamlFile' to 'FilesAndStreams' class
• Added new property-file reader with caching
• Added new 'Spark.streamFile' feature for byte-range requests (see new TTS features above)
  ...

Updated WebSocket Chat-Server to v1.3.4:

• Updated eclipse.paho MQTT client to v1.2.5
  ...

Other servers, tools and common changes:

• Improved SEPIA-Home install and run scripts and added an option to create one single, external folder for user data (using symlinks)
• Improved DIY client package with more Pulseaudio scripts e.g. for AEC (beta), more hardware scripts, better installation and run scripts etc.
• Improved CLEXI to support WS281x and APA102 RGB LEDs via SPI bus (new GPIO item)
  ...

SEPIA-Home v2.6.1 with new shared-access feature, custom wake-words, LED hardware interface and many more improvements

v2.6.1 - 2022.01.30

Important: If you're updating from a clean v2.6.0 using the integrated script please read this issue: #155.

After the humongous update from last October that introduced the new web-audio, speech-recognition and media-player interfaces SEPIA-Home v2.6.1 is a smaller release with a lot of incremental improvements. Yet it includes a set of very exciting new features as well so check out the notes below ✨ 😃.

The first important change is the introduction of the new shared-access feature. It allows you to create a list of users (see app setings 'account' page) that are allowed to send remote-actions to one or more of your devices. A very handy feature if you for example operate one SEPIA client in the living-room and want to allow different users to control music on this specific device 👦 📲 〰️ 🤖 🎶. New remote-action shortcuts in the app will simplify the process even further 🙂.

The next noticeable change is the support for Porcupine 2.0 the semi-open wake-word engine by Picovoice. In v2.0 users can create their very own wake-word 🗣️ 🤖 💭 via the Picovoice web console. To use v2.0 though you have to register for their free tier (3 devices allowed) and copy your access key to SEPIA's wake-word settings. Read more about it here.

The 3rd exciting new feature is support for LED arrays and hardware buttons using the new CLEXI GPIO interface! 🚨 💫 . This works for example with USB devices like the ReSpeaker Mic Array v2.0 or Raspberry Pi HATs with integrated LEDs and buttons. Example configurations can be found here.

Some honorable mentions are support for Graphhopper 🌍 as alternative to Google Maps (geo-coding, directions, etc.), extended smart-home commands with new 'general value' input (see updated Teach-UI) and integration of MBROLA voices and customizable settings for TTS.

As usual the release has a lot more to discover, so check-out the extended release notes or the "shorter" version below! Hope you enjoy the update ✌️ 😃.

Download SEPIA-Home - Download Android APK - See Twitter news feed

Release notes (short) - Extended version: here

Updated client to v0.24.1:

• Added support for new 'shared-access' permissions and new settings page to allow other users limited access to specific devices and remote-actions
• Added new 'remote media player' buttons to quickly control media on other clients (start, stop, skip)
• Added support for Porcupine 2.0 and custom wake-words. Create wake-words via Porcupine console and use with free-tier access key + updated wake-word settings page
• Updated web-audio lib and speech module to respond a bit more snappy and better handle timeouts, connection issues and no-speech events
• Updated CLEXI client to v0.9.0 and introduced GPIO interface to control for example LED arrays or read hardware buttons + added "clexiGpioInterface" section to settings.js
• Added remote terminal functions (CLEXI) to set/get/test microphone (gain, mictest) + added new debug page to STT settings
• Implemented 'Config.appRegionCode' to enable region specific service actions for same languages
• Improved avatar and app styles for example to better indicate wake-word activate state
• Updated input-controls view and improved Bluetooth BLE beacon security with "known addresses" check
• New skins: 'Funky Sunrise' and 'The Vectors'
• ...

Updated Control-HUB (admin-tools) to v1.4.2:

• Improved look, added skin support + new "light" skin and choose dark/light skin according to browser settings
• Added support for new 'raw' field to custom 'set-cmds' in smart-home page (device cards)
• Updated CLEXI client to v0.9.0 and added support for GPIO interface (read hardware buttons, control LEDs)
• Updated CLEXI help pop-up (remote terminal commands) to include new microphone settings and test functions

Updated Assist-server to v2.6.0:

• Introduced new 'shared-access' permission settings for accounts
• Allow 'receiver' field for remote-actions + check shared permissions when sending action to other user
• Created 'GeneralValue' parameter ("set AB to value XY") and implemented it to improve smart-home service (this will allow more general support for device-specific input)
• Introduced new 'Xtensions/TTS/settings.json' to configure TTS voices, improved setup and added MBROLA voices support
• Improved TTS pre-processing (e.g. better output for times)
• Reorganized and optimized geo-coder code and implemented Graphhopper API for geo-coding, location and directions search in addition to Google Maps
• ...

Updated WebSocket Chat-Server to v1.3.3:

• Added support for new 'sharedAccess' feature to manage and allow remote-actions between users
• Added endpoint 'refreshClientConnections' to trigger automatic client connection refresh (disconnect -> reconnect)
• ...

Updated Teach-Server to v2.2.3:

• Improved 'common.json' Teach-UI texts and configuration and added 'general_value' to smart-home command
• ...

Updated Core-tools to v2.2.10:

• Improved 'Connectors.apacheHttpGET' to be a bit more strict with timeouts and reconnects and to allow more custom settings
• ...

Make sure you don't miss this:

• Improved most of the SEPIA-Home install, setup and run scripts (including self-signed SSL certificate generation and handling)
• Improved DIY client (installation, logging, features) (installation, logging, features like LED controls etc.)
• Added SEPIA offline STT-Server install scripts (Linux) to SEPIA-Home and DIY client folders to get started more easily
• Adjusted server and DIY client for better compatibility with Debian 11/Raspberry Pi OS Bullseye
• ...

SEPIA-Home v2.6.0 with redesigned audio interface, new STT server, more powerful widgets and many more improvements

v2.6.0 - 2021.10.14

After almost one year in the making SEPIA-Home v2.6.0 has finally been released 🌠 and it is packed with amazing new features that will greatly enhance the overall voice-UI experience and pave the way for many new possibilities to customize and extend your own open-source assistant ✨ 😌 .

Heart of this update is the newly developed, modular SEPIA Web-Audio library that replaces the old audio interface and comes with state-of-the-art tools for audio recording and processing 🗣️ 🎶 🧑‍🔬 . Among other things the new library includes modules for wake-word recognition (via Porcupine v1.9), voice-activity-detection (VAD), audio resampling and the brand new version of the SEPIA Speech-To-Text (STT) server 🔥 . The new SEPIA STT-Server has been redesigned from scratch to support multiple, different STT engines (like Vosk), post processing of results (e.g. via text2num) and dynamic loading of ASR models while running on all major platforms, even Raspberry Pi 4! This elevates open-source speech recognition inside the SEPIA client to a whole new level 😎 🚀 .

Another important feature is the improved support for custom voice-widgets 👨‍🎨 📱 and the introduction of new media-player widgets 🧑‍🏭 📻 which will enable users to build own interfaces, dashboards, micro-apps (see clock demo) or music players just by creating a simple HTML file with a few lines of Javascript code 👩‍💻 and placing it inside the widgets folder. The SEPIA-Extensions repository has been updated to show some examples and collect future work (contributions are welcome ^^).

This release has a lot more to discover, so check-out the extended release notes or the "shorter" version below! Hope you enjoy the update ✌️ 😃.

Download SEPIA-Home - Download Android APK - See Twitter news feed

Release notes (short) - Extended version: here

Updated client to v0.24.0:

• Complete rework of audio system and switch to new SEPIA Web-Audio library:
  • Handle audio processing in 'AudioWorklets' and 'Worker' threads
  • Support for new SEPIA STT Server v2 with many new features (Docker container for all platforms available)
  • New WebRTC VAD (voice-activity-detection) module and experimental custom VAD module
  • Support for many new Porcupine wake-words like Computer, Jarvis, Alexa, Hey Siri, Hey Edison and many more
  • Support for external TTS server (Mary-TTS compatible API)
  • Added support for media-device selection (mic and speaker) and microphone test-page
  • New audio-effects library (e.g. for TTS filter effects etc.)
  • Support for MediaSession API
• Greatly improved features and support for custom voice-widgets (aka custom views/frames) including handling of speech events, input and animations (build your own voice interfaces)
• Completely new embedded media-player with custom widget support and powerful interface (build your own audio players)
• Improved Teach-UI commands overview + search field + support for improved 'music' service that works with new media-player widgets
• Implemented remote action 'notify' and updated 'broadcast' interface and button (broadcast TTS messages to clients with same account)
• New cards for audio, graphs (µPlot lazy lib.) and embedded content like YouTube + detection of YouTube URLs in chat
• Improved cards context-menus, e.g. to start media-player remotely on other connected clients
• Improved speech recognition support for Microsoft Edge and Apple's Safari browser (via Web-Speech API)
• Landscape mode for mobile screens (including orientation API support) + improved 'big-screen' mode
• New skins (+ improved older ones) 🎨 : 'Alabaster' (3 variants), 'Essential' (green + orange), 'Orange Style 2.0' (OS2)
• New avatars 👀 : 'The Dots', 'Classy', ILA O-Five (3 variants), 'S-Tech'
• Added ability to reload/refresh basic lists via context menu button
• Many style and UI improvements + updated icon-set
• Improved main menu (settings view) + updated tutorial
• Added button to export (show) client settings as JSON (handy for headless client setup)
• Made experimental languages accessible via language selector
• Improved service-worker handling + offline page and disabled service-worker by default (use new URL param. 'pwa' to enable)
• ...

Updated Control-HUB (admin-tools) to v1.4.1:

• Added 'get mediadevices' command to CLEXI help
• Updated icon set + improved service-worker and offline.html
• Replaced legacy STT page with new info page

Updated Assist-server to v2.5.2:

• Created new 'WebContent/widgets' folder and added default media-player (YouTube etc.) files
• Added new 'clock' demo view to 'WebContent/views' folder and updated old demo view
• Updated 'MusicSearch' service to support new media-player widgets and 'data' parameter
• Added new YouTubeAPI class and updated config and property files to support API key
• Tweaked YouTube web-search results
• Updated radio stations
• Updated news outlets
• Prepared open-liga worker and parameters for new season (Bundesliga)
• Added Porcupine wake-word files for HTML client to 'WebContent/files' folder
• Added support for optional 'delayUntilIdle' parameter of client_control_fun ACTION
• Updated 'ActionBuilder' class with more convenience methods
• Improved handling of list data and updates (this includes timers and alarms)
• ...

Updated WebSocket Chat-Server to v1.3.2:

• Added 'notify' as 'RemoteActionType'

Updated Teach-Server to v2.2.2:

• Added support for sorting commands by date
• Added Teach-UI service 'music' (music search)
• Allow custom (success) answers for music stream commands

Updated Core-tools to v2.2.9:

• Added fields to UserDataList 'checkable' item: 'itemId', 'priority' and 'dateAdded'
• Added optional 'delayUntilIdle' to client_control_fun ACTION
• Removed dependency 'google.guava' (use 'Apache Utils' and Java 8 classes instead)

Make sure you don't miss this:

• Improved Raspberry Pi DIY client (installation, logging, features)
• Added new sections to 'sepia-extensions' repository for custom media-player widgets and client-views
• Removed SEPIA reverse-proxy from SEPIA-Home package (use Nginx instead)
• In preparation: Updated SDK and Docker containers for SEPIA-Home
• Still on the list: Experimental Node.js support + Node-RED Nodes: (check 'Dev' branch, contributions welcome)

SEPIA-Home v2.5.1 with new client-to-client communication, custom HTML voice widgets, new style options, stability improvements and ...

v2.5.1 - 2020.10.21

On the paper its just a minor version jump from 2.5.0 but this release is actually packed with a lot of exciting new features, focused on the SEPIA client this time 😎. It comes with improved client stability and UX , especially when used as always-on, DIY smart speaker/display 🔮 , improved security, instant synchronization of timers ⏰ (and sync. warning for lists) across multiple clients (same user, different device ID), a lot of new status events (accessible via CLEXI remote terminal), new context menus for almost every result-card with buttons like 'play-on' for the radio-card to start the stream on a different device 📱 ↔️ 📻 and much more!
The power of 🖼️ custom view frames (HTML) has been greatly enhanced with new abilities and a deeper integration of speech input 🎤. Combined with the services SDK and a new command called 'frame_control' (see teach-interface) its a lot easier now to create your own "voice widgets" and UI 🧑‍🏭 🔠.
There is a brand new skin available as well in dark and light mode called 'Flaming Squirrel' 🐿️ 🔥 and it is possible now to select avatar and skin independently 🎨.

Besides many other updates another notable highlight is the improved Code-UI ⭐ of the SEPIA Control-HUB. You can now see all your uploaded services, open them, edit the code 👩‍💻 or simply remove them with one click . A new user-roles editor 📜 has been added as well .

Server improvements mainly focus on stability and support for all the new client features, but there have been some enhancements to the services as well. For example there are a few new news outlets like 'NY Times', 'The Guardian' or '1E9' 🗞️ and a new section for MAKER news including the 'Raspberry Pi' blog 🍇 ("show me Maker news").

There is more to discover, so check-out the extended release notes or the "shorter" version below! Hope you enjoy the update ✌️ 😃.

Download SEPIA-Home - Download Android APK - See Twitter news feed

[EDIT 2020.10.22]: Updated Android APK to v0.23.1 to hot-fix error messages for microphone issues

Release notes (short) - Extended version: here

Updated client to v0.23.0:

• Added support for PWA (progressive web app) feature of browsers/OS via new basic service worker and PWA manifest file
• Android 10 support aka set 'targetSdkVersion' to 29, updated Cordova to v9, android-platform plugin to v8.1.0 and several other plugins
• Implemented context menu for every result card, e.g. radio/music-stream (+ playlist button, if available), news (read or copy), weather, etc.
• Improved presentation of weather results and added more mini icons
• Greatly extended capabilities of custom view frames with proper integration of microphone, new scoped functions (e.g. 'handleSpeechToTextInput') and secure loading from SEPIA client (folder available in Android as well) or server (SEPIA web-server folder)
• New folders for HTML 'templates', 'custom-data' and 'local_data' + automatic file path expansion for the tags '<custom_data>/', '<local_data>/', '<app_data>/' (root folder), '<assist_server>/' (URL to SEPIA web-server), '<teach_server>/' and '<chat_server>/'
• Added example custom view to new 'xtensions/custom-data/' folder
• Support for new service-actions 'open_settings', 'frames_view_action', 'close_frames_view' and 'switch_stt_engine'
• Start and stop audio streams via new remote-action type 'audio_stream' + 'control' and changed data type 'music' to 'media'
• Added functions to get and show user clients and to send remote-actions via WebSocket chat-server
• Added 'connect', 'disconnect', 'wakeWordOn', 'wakeWordOff' and 'reload' to remote-action "hotkeys" (shortcuts: 'co', 'dc', 'ww', 'wm', 'F5')
• Automatically mark open lists as out-of-date if any active client changes the data
• New client event 'sepia_alarm_event' that will be broadcasted (to CLEXI) when an alarm is triggered/removed/stopped
• New client event 'sepia-audio-player-event' that will inform about audio streams (start/stop/URL etc.)
• Additional client events for 'sepia-account-error' and 'sepia-client-error' + 'sepia-state' event will now inform about 'connection' state (active, closed, etc.)
• Increased maximal number of custom buttons from 16 to 42 and in addition load custom buttons that were stored via assistant user (this way admins can create buttons for all users at once)
• Improved login-restore procedure on app start and added automatic login-retry after connection or server error
• Improved checks and handling of expired login tokens and prevent login-refresh try if client-ID was changed
• New touch-bar control style selectable via settings that uses only the bottom of the screen for swipe and automatically minimizes text input field (double-tap triggers back button)
• Added (very) basic big-screen mode to optionally remove size-limit of client window (settings)
• Interpret double-space as line-break in chat messages
• Fixed a bug that could sometimes crash the event queue after an empty speech result so that the client would not return to idle state
• Added an auto-reset trigger if client is in 'loading' state for more than 45s and (if enabled) made sure that the wake-word listener will restart reliably
• Added ability to define custom sound for mic activation confirm and default recorder buffer length (hidden in "Hey SEPIA" settings page)
• Added settings option to select avatar independently from skin, made possible by splitting CSS files for skin and avatar + added ability to load 'base' skins and avatars when adding new skins
• New skins "flaming squirrel" and "flaming squirrel dark"
• Added support for 'userPreferredSearchEngine' and search-engine selector to settings
• ...

Updated Control-HUB (admin-tools) to v1.4.0:

• Improved code-ui with new user custom-services manager to load, edit and delete previously uploaded services (source-code)
• New user-roles editor UI (replaces old, error-prone input field)
• Added new CLEXI remote terminal 'set' command examples ('micReset', 'connect', 'disconnect') and tweaked log colors
• Added button to client-connections page to show active clients via new 'getOwnClientConnections' chat-server endpoint
• Added statistics button to chat-server card in server overview page
• New room types 'entrance' and 'attic' for smart-home page
• Added support for PWA (progressive web app) feature of browsers/OS via new basic service worker and PWA manifest file
• ...

Updated Assist-server to v2.5.1:

• New news section 'maker' and new news outlets 'NY Times', 'The Guardian', '1E9', 'SPIEGEL Start' (replaces 'Bento'), 'Raspberry Pi' and 'SEPIA' + improved default selection for different sections
• Added SDK endpoint 'get-services' to load a list of user installed services
• Improved SDK service upload, store source code and load it via new '/get-service-source' endpoint
• New 'EventsBroadcaster' class, adaptations to 'RemoteActionEndpoint' and new method 'sendRemoteAction' (ServiceBuilder) for services
• New 'changeEvent' class and broadcaster to push e.g. user-data updates like changed lists and alarms directly to active clients
• Added new "custom view demo" (Xtensions/WebContent/views/custom-view-demo.html)
• Added new command 'frame_control' to create custom commands for custom view frames more easily
• Allow parameter extraction in services via named-groups in 'setCustomTriggerRegX'
• Added 'qwant' and 'ecosia' as default web search options and evaluate user preference (if given via client 'prefSearchEngine')
• Added new 'ActionBuilder' class to make usage in services easier
• Fixes and improvements for some NLU parameters and services (e.g. 'AlarmName', 'NewsSection', 'SportsTeam', 'OpenCustomLink')
• New 'Room' parameter types 'entrance' and 'attic'
• Fixed a bug in smart device type filter that could prevent automatic device loading from HUBs like 'FHEM' or 'openHAB'
• Sanitized device loading procedure for internal smart home HUB to improve timing for larger lists of items
• Added new 'validUntil' field to authentication endpoint and improved error code for expired tokens
• Added 'playlistURL' to card-data of each radio station if available
• Added CORS support for static file hosting + 2 new config options for assist.[].properties: 'enable_cors' and 'enable_file_cors'
• Updated Bundesliga service and teams for new season
• ...

Updated WebSocket Chat-Server to v1.3.1:

• Increased WebSocket maximal message size from 64 KB to 256 KB
• Added "broadcast to all devices" feature to remote-action handler and allow client-to-client actions for devices with same user
• Improved error messages for several handlers, e.g. 'SepiaAuthenticationHandler'
• Added new 'ClientManager' class and related endpoint '/getOwnClientConnections' to get a list of active clients for a specific user-id
• Added message data type 'ping' and new background service to test if a client is still alive and active (ClientPingRequestHandler)
• Added new statistics and endpoint (basically identical to the other servers)
• ...

Updated Teach-Server to v2.2.1:...

SEPIA-Home v2.5.0 with new Smart Home options (ioBroker, MQTT), new TTS voices, Wake-Words, improved Interfaces and much more

v2.5.0 - 2020.06.06

This is the biggest update yet and a milestone release for the SEPIA project since it introduces a new system to manage multiple 🏠 💡 Smart Home HUBs at the same time (including general MQTT and ioBroker support + openHAB + FHEM), it has more 🤖 💬 TTS options (Pico and MaryTTS), much improved usability (especially for the Teach-UI and Smart Home pages) and numerous other bigger and smaller improvements all over the place! 😀
There is a new skin as well 🎨 🖌️ that includes a unique avatar (making it a total of 3 now) 🚀 🌠 and there is even some code written in Kotlin this time 😁 .

Check-out the release notes below for all the details (but be sure to bring enough time ^^).

Download SEPIA-Home - Download Android APK - See Twitter news feed

Release notes

Updated client to v0.22.0:

• Major updates to Teach-UI to make creation of custom commands easier and more intuitive including examples and input pop-up for parameter data
• Added function to open Teach-UI via long-press on custom command button to edit command (uses new server endpoint, see below)
• Exported wake-word configuration to 'wakeWord.js' file (folder 'xtensions'), implemented switching of Porcupine engine (v1.4-1.6) and added all free wake-words (>40). Instructions included in folder.
• Updates for new weather service including new icons, styles and card updates
• Load new TTS voices list from SEPIA server (TTS endpoint) + smaller TTS fixes
• New skin 'historic future' as homage to 80s science fiction including new avatar for always-on mode
• Fixed bugs in 'geocoder' module and related errors in my-view page update
• Fixed a bug in speech recognition that could sometimes prevent the result from loading at first try
• Updated CLEXI lib to v0.8.2, added support for 'runtime-commands' (e.g to handle DIY client reboot and shutdown) and fixed a bug in CLEXI connection
• Added broadcasting of speech events via CLEXI connection (intended for DIY client e.g. to set LED status lights)
• Added support for integrated speech recognition in Firefox Nightly (currently requires activation of 'media.webspeech.recognition.enable' and 'media.webspeech.recognition.force_enable' flags in FF)
• Improved security by restricting access to certain SEPIA library functions via session token and by escaping all HTML code inside plain chat messages (includes adjustments to handle new chat server behavior)
• Improved 'inputControls' module, fixed gamepad support and remote trigger
• Improved handling of 'lastAudioStream' and stream title
• Added proper chat labels (indicators that show when messages were posted) for 'today' and improved 'UI.showInfo'
• Improved 'intent://' handling of Android inApp browser
• Added URL param 'autoSetup' (similar to headless-mode) to load settings.js at start and enter setup mode after 8s if no user is logged in
• Added basic web-worker interface that can be used to run code in background thread
• Optimized app start-up, initial page and skin loading
• Improved logger (for dev tools console)
• Smaller and bigger skin and style tweaks all over the place (skin upgrades, Firefox scrollbar support, etc.)

Updated Control-HUB (admin-tools) to v1.3.2:

• Several updates and improvements to 'smart home' page including full suport for new internal HUB, custom interfaces (including ioBroker and MQTT), smart device creation and more intuitive server settings handling
• Implemented new 'user list' endpoint and added request button to 'user management' page
• Added 'runtime-commands' support (new CLEXI feature) to 'client connection' page and updated CLEXI lib to v0.8.2
• More colors for CLEXI remote-terminal connection broadcasts + log filter options
• Added save file button to 'Code-UI' page and implemented auto-loading of class name on code paste event (if possible)
• Adjustments to handle all server changes (see below, e.g. camel-case in JSON etc.)
• Improved automatic hostname detection for public URLs during login
• Improved logger (for dev tools console)

Updated Assist-server to v2.5.0:

• Added completely new weather services with improved results, new icons and NO API KEY requirements
• Integrated TTS engines 'Pico' (3 voices) and 'Mary-TTS' (5 voices + X, requires server) + automatic handling of Mary-TTS server at SEPIA start
• Manage smart devices inside SEPIA and handle multiple smart HUB interfaces at the same time via the new interface 'SmartDevicesDb' using the new database indices 'smart-devices' and 'smart-interfaces' (implementation: 'SmartDevicesElasticsearch')
• Added new smart home HUB connectors (partially supporting 'SmartHomeHub') for ioBroker and MQTT
• Added new smart home HUB interface 'InternalHub' that stores smart devices inside the internal SEPIA DB and can connect to multiple instances of: openHAB, FHEM, ioBroker and MQTT to read/control devices
• Several improvements to 'SmartDevice' NLU parameter to find best device match for example extraction of known (buffered) device names
• Several improvements to 'SmartHomeHub' interface to support new implementations
• New types 'terrace' and 'balcony' for parameter 'Room' and tweaks for type 'other'
• Added GZIP header for FHEM interface to avoid (potential) bug with large 'getDevices' results
• Changed 'SmartHomeDevice' JSON fields to use camel-case consistently
• Improved smart home service to use all new features and fixed some bugs
• Updated news with sections for 'health' and 'corona' and added answer for devices without active display
• Created new 'ThreadManager' class to better handle all (most) active threads of the server and offer some convenience methods e.g. for parallel processing
• Added new worker type 'DuplexConnectionInterface' + MQTT implementation and improved existing workers
• Improved collection and presentation of server statistics to include smart home APIs, new workers and threads
• Added 'processTime' field to 'NluResult' JSON output with details about NLU and service time
• Updated 'ClientControls' service and 'ClientFunction' parameter to support new 'runtime-commands' feature (related to CLEXI updates)
• Added 'resume' to NLU parameter 'Action' and improved music and radio related services
• Extended 'UserManagementEndpoint' to accept action 'list' and return a basic overview of all user accounts
• Added general support for Kotlin and added 'com.willowtreeapps.fuzzywuzzy' (0.1.1) Kotlin library
• Added new new fuzzy match methods to 'StringCompare'
• Added class 'ParameterTools' to run performance critical methods via a profiler that will auto-skip slow calls
• Changed NLU sentence normalizers to keep '?' if its in the MIDDLE of a sentence
• Improved 'WebsearchBasic' service to handle devices that don't have an active display (via 'ENVIRONMENTS.deviceHasActiveDisplay')
• Improved 'OpenCustomLink' service security for links that contain Javascript code
• Improved RSS feed reader error correction
• Improved account data validation
• Many smaller tweaks to parameters, services and answers
• Code clean-ups for Java 11

Updated WebSocket Chat-Server to v1.3.0:

• Updated 'SepiaMqttClient' and 'SepiaMqttClientOptions' with new methods, e.g. for topic subscriptions and connection checks
• Introduced server-side HTML encoding for the 'text' field in 'SocketMessage' to prevent HTML injections by default
• NOTE: Due to the changes in 'SocketMessage' SEPIA services cannot use HTML content in plain answers anymore (use actions or cards instead)

Updated Teach-Server to v2.2.0:

• New endpoint 'getPersonalCommandsByIds'
• Restructuring of 'common.json' file used by Teach-UI to define shared parameters + general improvements and more info for commands.
• NOTE: Due to the format changes of 'common.json' old SEPIA clients (<0.22.0) need to be updated

Updated Core-tools to v2.2.6:

• Added new user roles: infant, kid, teen, elderly
• Extended 'AuthenticationInterface' and 'Elasticsearch' with methods for 'listUsers' and 'getDocuments'
• Extended 'ConnectionCheck' tool to optionally print error via 'printError' argument
• Optimized Http GET/POST/etc. methods in 'Connectors' to support GZIP encoding header, fixed some header issues and added timeout parameter
• Added 'CsvUtils' to tools to read CSV data
• Added 'deviceHasActiveDisplay' to 'ENVIRONMENTS'
• Added 'org.owasp.encoder' lib and new HTML encode and escape methods to 'Converters' class
• Added Apache Commons-IO lib
• Improvements to 'JSON', 'StringTools', 'URLBuilder', 'RuntimeInterface' and 'EsQueryBuilder'
• Updated jackson-databind to v2.9.10.3

Other servers, tools and common changes:

• Improved server setup, run and test scripts including support for Mary-TTS server
• Improved server update and backup scripts and added script to import backups
• Added script to import SSL certificates into Java truststore (Linux, beta, useful for self-signed SSL)
• Improved DIY client installation and setup (more info soon)
• Updated all servers and tools to use core-tools v2.2.6

SEPIA-Home v2.4.1 with remote terminal and new TTS endpoint to support DIY 'headless' clients

v2.4.1b - 2020.02.18 - updated: 2020.02.20

This release does some fine tuning of the smart home service, its parameters and interfaces (FHEM, openHAB) 🏠 💡 and introduces two very important new features required to build your own "headless" SEPIA client: a remote terminal for the client 📟 and a new TTS endpoint for the server 🤖 💬 to further reduce dependency on OS functionality.

If you like to customize commands you now have a new option to create your own server files for predefined small-talk, sentences and answers and carry them over from version to version (because they won't be overwritten). Check out this little tutorial.

In the skins section you will find a new entry called "ILA Legacy" that is special in two ways. First of all it is a tribute to ILA voice assistant the predecessor of SEPIA (that btw introduced offline TTS today as well ... just (almost) exactly 5 years ago ^_^) and secondly it is the first skin with a custom avatar for always on mode. Check it out! 😃

Oh and a quick reminder (because it has been asked a few times lately): Version v2.4.0 introduced a demo service for MQTT 📡 message broadcasting that can be found in the SDK and extensions repo.

Enjoy! 🙂

[EDIT] Updated with improved error messages about missing SSL and new self-signed SSL scripts for NGINX

Release notes

Updated client to v0.21.0:

• Introduced new headless mode (URL param. 'isHeadless=true') with support for 'settings.js' file and new remote commandline module (using CLEXI server)
• Client will automatically switch into setup state after few seconds when in headless mode and no login is given (+ audio notification ^^)
• Integrated new TTS engine that streams data from SEPIA server (switch via new voice engine selector in settings)
• Added new 'ILA-Legacy' skin with custom avatar for Always-On mode :-)
• Added new 'server-access' page for detailed connection configuration (accessible from login and settings, replaces hostname field)
• Improved automatic hostname recognition
• Created new icon selection popup and applied it to Teach-UI custom button field
• Added custom GPS location to device site settings
• Slightly improved visibility of missed chat messages
• Improved alarm sound player to better manage other audio sources
• New CSS options to better control status bar and navigation bar color in Android (and probably iOS)
• Fixed bugs in view scrolling and 'switchLanguage' service-action
• Added URL param. 'logout' to start client with automatic logout
• Improved Teach-UI start-up to allow more pre-filled teach fields
• Added more debug/help info for insecure-origin (SSL stuff) issues like microphone access restrictions
• Updated CLEXI lib to v0.8.1
• Added some IE11 polyfills

Updated Control-HUB (admin-tools) to v1.3.1:

• New 'Client Connections' page including remote terminal to access headless clients (via CLEXI)
• New 'Speech Synthesis' test page for SEPIA TTS endpoint called
• Smart home: better check of set-cmds entry for toggle button
• Smart home: added 'sunroom' to list of rooms
• Improved automatic hostname recognition for '[IP]:20726/sepia/...' origins
• Updated CLEXI lib to v0.8.1

Updated Assist-server to v2.4.1:

• The TTS module of the server has been updated and upgraded to support the 'espeak' engine (flite and picoTTS will follow soon). This means every client can stream TTS from the SEPIA server now if required. Please check the updated 'setup' scripts to install the packages!
• Improved DataLoader to allow custom command and answer files, e.g. chats_en_custom.txt in addition to chats_en.txt (folder: sepia-assist-server/Xtensions/Assistant/...). This way users can better manage global custom commands.
• Smart home interfaces (FHEM, openHAB, TestHub) will load devices correctly via unique ID now
• The smart home service will ask the user now if its ok to use the first result in case a device search has multiple matches (given a specific room)
• If a command uses a smart home device with specific name (tag) e.g. because it was defined via Teach-UI the tag will now be used properly to search the device (see Teach-UI examples)
• Added 'sunroom' type to Room parameter
• All brackets in the names of smart home devices will correctly be removed during voice output (use them e.g. to give your devices numbers)
• Fix for FHEM interface if CrfToken is deactivated in FHEM server
• Several smaller fixes/tweaks to the smart home interface (better 'state' and 'stateType' checks, "device with number X" works, correct handling of non-existing device types, etc.)
• Updated radio stations, fixed a bug and added "chill-out" to parameter 'MusicGenre'
• Improved output format of local time in answers
• Updated news outlets and removed "11FREUNDE" (feed does not exist anymore :-( )
• Added a README for the directory listing endpoint that can be found inside the web-content folder
• Improved Java 11 compatibility

Updated Core-tools to v2.2.5:

• Fixed a bug in RuntimeInterface
• Added new 'open_frames_view' action (to be used by services)
• Better error messages for Apache Http GET method
• Updated mp3spi lib to v1.9.5.4 to fix a Java 11 issue

Other servers and tools:

• Added scripts to install Java locally into SEPIA folder (~/SEPIA/java). Very useful for Windows systems.
• **Updated server scripts to offer more options, better overview and improved features (e.g. setup Nginx and TTS engine, handle errors better, etc.)
• Updated installation scripts and descriptions to get rid of some outdated info and stuff (e.g. promote Java 11, sudo warnings, etc.)
• Teach-Server v2.1.1 has some improved descriptions for Teach-UI commands
• Updated all servers and tools to use core-tools v2.2.5
• Updated Dockerfile (beta) and added new script to build the build-environment for SEPIA-Home
• Prepared everything for the release of the official "headless" Raspberry Pi SEPIA-Client

SEPIA-Home v2.4.0 with big smart home service upgrade and support for FHEM

v2.4.0a - 2019.12.30

This release is all about smart home 🤖 🏠 😎 .
The smart home NLU, interface and service have been upgraded to support 11 device types (lights, thermostats, roller shutter, sensors, etc.), 14 room types (living-room, office, garden, etc.), device numbers (Lamp 1, 2, 3, ...) and room numbers (bath 1, bath 2, room 300, ...). In addition the smart home page of the SEPIA Control-HUB has been upgraded with new device-cards and is now communicating with the smart home HUB via a new Assist-Server endpoint ('integrations') instead of calling it directly. The SEPIA client device can be configured to 'belong' to a certain room in your smart home so that a command like "lights" will refer to the same location. Last but not least SEPIA is supporting the open-source FHEM smart home HUB natively now (in addition to openHAB).
A few other notable updates (amongst many) are the 2 new NLU server endpoints 'understand' and 'interview' (you can try them via the Control-HUB assistant test page) and the new web-server endpoint 'web-content-index/*' that is able to generate a content list for web-server folders.
Oh and one more thing ... there is a new demo service for MQTT message broadcasting in the SDK and extensions repo 📡 🙂 . Enjoy!

Release notes

Updated client to v0.20.0:

• Integrated SEPIA Control HUB into settings frontpage of app (will show when user has certain role, e.g. 'tinkerer' or 'smarthomeadmin')
• New setting for preferred temperature unit (Celsius/Fahrenheit) that can be accessed in services via user account or device settings
• Added new page for device local site to set a specific location for the client like 'home:living-room' that can be read by any smart service on the server
• Load up to 16 custom command buttons to my-view by default (up from 10)
• Load services config of Teach-UI from new teach-server endpoint (and added smart home service)
• Added a help button for extended login box that redirects to SEPIA docs
• Auto-assign a 'dark-skin' or 'light-skin' class when selecting a skin to better handle certain CSS rules
• Basic post-message interface for apps that run inside IFrames (to be extended soon)
• Tweaked TTS voice selection indicator depending on platform
• Added 'env' parameter to launcher page

Updated Control-HUB (admin-tools) to v1.3.0:

• Added post-message interface to load tools in IFrame and login user (used for example in client app to show Control HUB and auto-login user)
• Smart home: Uses new 'integrations' endpoint of assist-server to communicate with HUBs instead of calling them from client (in addition: HUB host URLs must match server setting now to prevent unsafe HTTP calls)
• Smart home: Completely reworked device cards and interface (new options, hide/filter devices, auto-refresh, test button, etc. ...) and added Test-HUB for experiments
• Smart home: Support for a total of 11 device types (lights, shutters, thermostats, etc.) and 14 room types (bath, office, living-room, etc.)
• Smart home: Allow device names with numbers in brackets that will be skipped in assistant answers, e.g. name='Bed Light (2)' -> answer: 'your Bed Light is set to ...'
• Smart home: Added extended device settings for experts to fine-adjust state type and set commands
• Assistant testing: Added two new buttons to call 'understand' and 'interview' server endpoints
• Core settings: Show persistant settings as editable list
• STT: Added test button to extract personal commands as (anonymous) training data for language models

Updated Assist-server to v2.4.0:

• Added new server endpoints 'understand' (improved version of 'interpret') and 'interview' (shows either intent summary or indicates missing info including question)
• Completely reworked 'SmartHomeHub' interface to improve HUB communication and make it easier to integrate new HUBs (register methods, generalized set values, more sepia-xy tags, common search and filter methods, etc. ...)
• Added FHEM smart home HUB support (config name 'fhem') and a Test-HUB for experimenting (config name 'test')
• Improved 'SmartHomeHubConnector' service and whole smart home methods + NLU to handle a total of 11 device types (lights, shutters, thermostats, etc.) and 14 room types (bath, office, living-room, etc.)
• Added NLU support for device number and room number (e.g.: "set light 2 in bath 2 to 70%") and changed 'SmartDeviceValue' result type from 'Number' to own format
• Added optional reply parameter to smart home service
• Added settings for smart home HUB basic authentication ('smarthome_hub_auth_type', e.g. 'Basic' and 'smarthome_hub_auth_data', e.g. base64 encoded 'username:password')
• New server endpoint for integrations ('integrations//') and first implementation for direct smart home HUB communication (send commands to HUBs like FHEM directly without using the NLU + smart service)
• Questions inside a service can now use wildcards to access previously defined parameters from 'resultInfo' (example dialog: User: "Set light" - Sepia: "Set <1> to what?")
• Added new 'unit_pref_temp' (temperature unit) property to user account
• Services have access to new 'device local site' and 'user preferred temperature unit' (both used in smart home service for example)
• Added temperature convert method (C <-> F) to Number parameter and automatically convert between units in smart home service (depending on user pref. and device state type)
• Added 'ServiceRequirements' class and 'getRequirements' to 'ServiceInterface' to better handle SEPIA version conflicts in the future
• New server endpoint 'web-content-index/[web-server-folder]' and settings (see example folders in 'Xtensions/WebContent/') to generate directory listings of web-server content
• Added 'Size Radio' channel and fixed 'EgoFM' streams
• Support for Elasticsearch authorization via 'db_elastic_auth_type/data' setting
• Tweaked 'getConfig' server endpoint to show even less sensitive data from properties file (passwords and keys show as 'HIDDEN')
• Added device-ID to Elasticsearch mapping for personal commands
• Several code clean-ups, improvements, NLU tweaks and bug fixes

Updated WebSocket Chat-Server to v1.2.2:

• Added Eclipse Paho MQTT client and SEPIA classes (SepiaMqttClient, SepiaMqttClientOptions, SepiaMqttMessage)
• Support for Elasticsearch Authorization via 'db_elastic_auth_type/data' setting

Updated Teach-Server to v2.1.0:

• Support for Elasticsearch Authorization via 'db_elastic_auth_type/data' setting
• New endpoint 'getTeachUiServices' and services file (Xtensions/TeachUi/services/common.json) to be able to load Teach-UI configuration from server
• Added command 'smartdevice' to Teach-UI config with examples
• New endpoint 'getAllCustomSentencesAsTrainingData' to be able to extract data for speech recognition language model
• Optionally add device-ID to custom commands for future 'per-device' command selection

Updated Core-tools to v2.2.4:

• New user role 'smarthomeadmin' (user that has access to smart home HUB config) and 'tinkerer' (a user that likes to see more detailed configuration options ^_^)
• New CoreEndpoints method 'getWebContentIndex' to index web-server folder and build interactive file list
• Updated Connectors with more control of headers and added 'headers' map to 'HttpClientResult' for 'apacheHttpGET' method to get direct access to result headers
• Added support for 'Authorization' header to Elasticsearch class via 'auth_type' and 'auth_data' settings
• New method 'readFileModifyAndCache' in FileAndStreams class to read a file line-by-line, optionally modify each line and keep result in cache (handy for settings files etc.)
• New decimal format number converters 'stringToNumber' and 'numberToString' to read numbers and get a number string in specific format more easily (uses new common default decimal format)
• Added deviceId as field to Command and SentenceBuilder classes
• Added 'getKeys' and better 'prettyPrint' to JSON class
• New 'Is.typeEqual' and 'Is.typeEqualIgnoreCase' methods
• Added 'modifyThread' permission to SandboxSecurityPolicy to avoid errors with Paho MQTT client
• Tweaked 'httpSuccess' method in Connectors to optionally give cleaner result
• Updated fasterxml.jackson.core (again!) to apply latest security fix

Other tools:

• Updated SDK to v0.9.20 to support new SEPIA-Home release and added 'MqttDemo' service to demonstrate how to publish service results to a MQTT broker
• Updated Mesh-Node to v0.9.12 to include new core-tools v2.2.4

SEPIA-Home v2.3.1 with support for Custom Interpretation-Chain, NLU Servers, Python Bridge and more

v2.3.1a - 2019.10.14

This release fixes some smaller bugs and polishes off some rougher edges related to the chat improvements of the major v2.3.0 update but it also introduces two important new features namely the ability to configure the NLU interpretation chain of SEPIA and the option to add a custom NLU server to it that integrates seamlessly with the dialog management of SEPIA. There is an example given in the updated SDK that demonstrates how you can use the new SEPIA Python bridge (an early stage, lightweight Python server written with FastAPI) to upgrade your own services with custom Python code.
As usual there are plenty of other tweaks and optimizations so check out the full patch notes below:

Release notes

Updated client to v0.19.1:

• Custom-buttons (defined via Teach-UI) work properly now in group-chats
• Show a colored bell in AO mode to indicate 'you have a message in another channel'
• Fixed some bugs related to channel-history feature, e.g. a few seconds scrolling-lock and missing/wrong day tags in chat
• Improved audio-events tracking and handling in connection to 'hey SEPIA' wake-word
• Added option to allow/prevent wake-word while music is playing (default: prevent, to avoid audio artifacts in some mobile clients)
• Improved audio recorder performance and stability and fixed dynamic downsampling
• Fixed some issues with YouTube player (sometimes 'pause music' wasn't working)
• Added 'env' URL parameter to be able to set custom value for 'environment' variable (client info sent to server)
• Fixed deprecated code in iOS audio processing to make build process work again (iOS 12.4, Swift 5 - native ASR still broken, but restored open-source ASR support)
• Improved some error messages after failed login
• Prevent multiple queued follow-up messages of same 'type' (only one will show)
• Prevent chat names that look like user IDs to prevent accidental private messages to wrong receiver
• Prevent auto-scrolling of chat when hidden channel-status message was added
• Updated jquery to 3.4.1
• Added library for voice-activity-detection (VAD, though it is not used yet)

Updated Control-HUB (admin-tools) to v1.2.2:

• Updated smart home settings with new device and room options
• Implemented remote-action on 'assistant' page
• Added info message for new temporary 'login blocked' feature and fixed a bug with 'new login'
• Updated jquery to 3.4.1

Updated Assist-server to v2.3.1:

• Made NLU interpretation-chain configurable via properties file entry 'nlu_interpretation_chain'
• Introduced 'getWebApiResult' as new interpretation-step to easily integrate different NLU servers (example chain entry for SEPIA Python bridge: getPersonalCommand, WEB:http://127.0.0.1:20731/nlu/get_nlu_result, ...)
• Introduced abstract class 'WebApiParameter' to easily integrate custom parameter handlers that use different NLU servers
• Improved JSON import/export methods for 'NluInput', 'NluResult' and 'User' classes and added optional 'custom_data' fields to better serve new web API NLU options
• Introduced 'addCustomTriggerSentence' variations in 'ServiceInfo' to support predefined parameters (raw, normalized or extracted)
• Introduced optional input tags '<i_raw>', '<i_norm>' and '<i_ext>' to 'Interview' module to control how a predefined parameter should be handled
• Updated 'SmartDevice' and 'Room' parameters with new types (tv, fridge, oven, office, ...)
• Updated news outlets with 'Hackaday' RSS feed
• NLU tweaks ("pause music", "switch to german", ...)
• Fixed and improved 'remote-action' endpoint
• Added security feature to natively protect selected accounts from brute-force attacks by blocking login temporarily after too many failed attempts. Add accounts via properties file entry 'protected_accounts_list'
• Added option to deactivate security policy and sandbox via cmd arguments 'nosecuritypolicy' and 'nosandbox'
• Code clean-ups

Updated WebSocket Chat-Server to v1.2.1:

• Fixed and improved 'remote-action' handler
• Fixed a bug in sorting of channel history by timestamp

Updated Core-tools to v2.2.3:

• Redesigned ENVIRONMENTS class to better reflect possible client types
• Added method to get parameters from command-summary string
• Code tweaks

Updated SDK to v0.9.12:

• Support new SEPIA-Home release v2.3.1
• Added 'PythonBridgeDemo' service to demonstrate SEPIA Python bridge and new NLU web-API options
• Updated 'WorkoutHelperDemo' to demonstrate new options for 'addCustomTriggerSentence' with predefined parameters
• Updated readme to emphasize JDK dependency
• Updated FasterXML/jackson core to 2.9.10 to include latest security fix

Other tools:

• Updated Teach-Server to v2.0.4 to include core-tools v2.2.3 and fixed version number
• Updated Mesh-Node to v0.9.11 to include new core-tools v2.2.3 and fixed version number

SEPIA-Home v2.3.0 with new Messenger Features, Background Actions for Services and a lot more

v2.3.0a - 2019.09.08

The most striking changes in this release are probably the new messenger features! The users can now create new chat channels from inside the client, add other users directly or invite them via SEPIA universal link. Additionally channels with new messages are marked after login or whenever a message arrives while the user is online. By default the greatly improved chat-server will store the last 25 messages of every private group-chat channel.
Other noticeable features are: answer-text output in Always-On (AO) mode, the option to change the login password from inside the client, language switching via voice, background tasks and follow-up messages for services, language independent sharing of requests via URL, more tools for admins, improved support for small screens (useful for DIY RPi4 projects etc.) and much more!

If you need a detailed installation guide check THIS blog article, for the latest news and tips follow SEPIA on Twitter and be sure to stop by the official Google Play Store site if you are looking for the Android app (available in many countries).

Release notes

Updated client to v0.19.0:

• Partially reworked and greatly improved messenger features and UI to support channel create, join, invite (via URL), missed messages, history and more
• Added option to change login password from settings menu (via old password)
• Improved messaging between devices with same login but different device-ID
• Improved handling of SEPIA universal links when posted inside SEPIA chat channels
• Improved UX and security when interacting with SEPIA in group chats (public SEPIA messages will not automatically execute actions or play music anymore)
• Keep keyboard open in mobile apps when 'send'-button is pressed
• Forget last command after 60s (to prevent 'I just told you' kind of SEPIA comments after long idle time)
• Automatically select user-preferred color scheme (read from OS) and set light or dark skin when no skin was selected before
• Optimized 'switch-language' action
• Renamed 'saythis' button to 'broadcast'
• New 'updateData' message-event handler to support arbitrary data exchange with chat-server via WebSocket connection
• Fixed a rare crash due to outdated splash-screen plugin in Android
• Improved data store/load script to reduce number of writes
• Added onChatOutputHandler for views like AO-Mode (e.g. see SEPIA answer as text in AO)
• Added pause/resume client control for audio players and tweaked YouTube music to properly pause audio when STT is activated
• Added support for new input command 'i18n:XY' to dynamically set input language (e.g. 'i18n:de Guten Tag' will trigger the German 'Hello' even when app language is english)
• New share-menu activated by a long-press on the sender name in a chat message
• Made BLE Beacon remote URL more flexible
• Improved auto-scaling below 300px window width and tweaked tiny-mode
• New launcher.html page to configure launch-options and automatically redirect (handy for app in browser kiosk-mode)
• Reactivated 'application/ld+json' tag in index.html
• Set Android target SDK to 28 (Android 9.0) and improved support including new 'network security config'
• Minor UI, bug and style fixes

Updated Control-HUB (admin-tools) to v1.2.1:

• Core-settings page now allows to write assist-server settings (key-value pairs) directly to config file and reload server remotely. This e.g. allows to quickly add API keys etc.
• Smart-home page now allows to load smart-home HUB info data from server (via improved server-config endpoint)
• Added 'delete user' function and button to user-management page
• Fixed channel create function to support new chat-server version
• Get channel history statistics and force database clean-up from chat-settings page
• Style tweaks and more info texts
• Added correct version number to help-page

Updated Assist-server to v2.3.0:

• Introduced new SmartHomeHub interface and config settings to make integration of hubs like ioBroker, FHEM and HomeAssistant easier
• Adjusted openHAB integration to use new SmartHomeHub interface
• Updated radio-stations and news-outlets lists
• Fixed OpenLigaWorker and service to support new season of German Bundesliga
• Added background tasks and task-manager support (schedule, check, cancel, etc.) to services (via 'ServiceBuilder.runOnceInBackground') and improved data-storage for services (e.g. via 'ServiceBuilder.readServiceDataForUser')
• Improved and enabled (previously dormant) NLU-parameter 'Language' and tweaked dictionary service
• Added 'LanguageSwitcher' service to support commands like "switch language to German"
• Improved smart home device intent (NLU) to recognize devices more generally (e.g. "status of my XY 1" -> device=XY 1)
• Improved 'no_result' answer when using 'sentence_connect' command (e.g. via Teach-UI) and tweaked internal service redirect
• Tweaked 'reminders' NLU ("remind me to/remember ...")
• Added inputModifiers to InterpretatonChain and implemented new i18n-modifier to be able to switch input language on-the-fly (e.g to share SEPIA links independent of user language)
• Added meta data to OPEN_LINK command to better handle 'linkshare' (note: 'linkshare' is a special chat input cmd, e.g. type: "linkshare https://twitter.com/sepia_fw")
• Improved 'AuthEndpoint' to support password change via old password (or superuser) and tweaked 'delete' user procedure
• Improved 'ConfigServer' endpoint to support editing server settings file (assist.xy.properties) directly, e.g. via SEPIA Control-HUB
• Implemented server restart method and added endpoint option to 'ConfigServer' (it will try to shutdown client and workers as clean as possible and reload settings)
• Fixed code in GeoCoding and Weather API to make them more stable
• Adjustments in SEPIA WebSocket client to support chat-server update (including device-ID message routing)
• Updated and added new Elasticsearch database mappings for chat (channels, users, messages) and added test-method to server start that will add missing ES indices automatically

Updated WebSocket Chat-Server to v1.2.0:

• Re-organized and improved code, e.g. to get better access to all the message handlers (create channel, join, auth, etc.)
• New and improved methods/handlers/endpoints for channel create, join, delete
• New database interfaces and implementations (Elasticsearch) for channel-data (store/load created channels), channel-users (store/load missed events) and channel messages (store/load channel history)
• New settings in config file for database modules (in-memory and Elasticsearch) and options like 'store_messages_per_channel' or 'max_channels_per_user' etc.
• Automatic clean-up in background of old messages (e.g. when 'store_messages_per_channel' threshold is reached), see config option 'channel_clean_up_schedule_delay'
• Broadcast 'missed message' event to online users when in different channel and store events for offline users to inform them later on login
• Improved message routing by including sender and receiver device-IDs (especially useful when multiple devices run on same account, e.g. smart-speaker in living-room and kitchen etc.)
• Improved handling of channel-ID in combination with channel-name
• New 'updateData' message-event handler to support arbitrary data exchange with clients via WebSocket connection
• Added server-ID to each socket-message (in case we expand the cluster later)
• New endpoints 'getAvailableChannels', 'getChannelHistoryStatistic' and 'removeOutdatedChannelMessages'

Updated Core-tools to v2.2.2:

• Added run-single-task and schedule-task methods to 'ThreadManager'
• Added range-match to 'EsQueryBuilder' (e.g. to find all entries older than xy-timestamp)
• Created 'LruCache' class (last-recently-used cache)
• Updated javaspark up to v2.9.1
• Updated fasterxml.jackson.core to 2.9.9.2

All servers:

• 'localName' of server is now also used as server-ID and device-ID if one is required (e.g. for load-balancing of cluster or broadcasting info to chat channels)

Other tools:

• Added code for Microbit BLE Beacon remote
• Updated Teach-Server to v2.0.3 to include new core-tools v2.2.2 and thus javaspark v2.9.1 (no other code changes)
• Updated SDK to support new SEPIA-Home release and added 'WorkoutHelperDemo' service to demonstrate pro-active background messages and database access
• Updated reverse-proxy to v0.3.2 to include security fix for undertow-core v2.0.23
• Updated Mesh-Node to v0.9.10 to include new core-tools v2.2.2 and thus javaspark v2.9.1
• Updated browser extension to v0.6.2 to exclude localhost, local domains and SEPIA path from showing navbar