Release 2.5.0
Release notes - Hopsworks - Version 2.5.0
Bug
HOPSWORKS-2983 Projects can't be created without hops-system namespace in Kubernetes
HOPSWORKS-2980 Allow multiple model files in a model artifact
HOPSWORKS-2973 zip/unzip URI encoding calls system python, use hops-system python instead
HOPSWORKS-2965 Remove pathlib, confluent-kafka, hops-petastorm, opencv-python as dependencies for hops-util-py
HOPSWORKS-2963 NullPointerException when listing tags for model
HOPSWORKS-2962 Components in Serving UI are not aligned
HOPSWORKS-2953 init_prepared_statement throws NoneType exception
HOPSWORKS-2952 update hsfs examples with train_split property
HOPSWORKS-2951 hsml should also be monitored for minor releases
HOPSWORKS-2950 Remove user, project and installation section from hopsworks documentation
HOPSWORKS-2939 [tez] Tez master does not send application certificates with the create container request
HOPSWORKS-2936 Pin ipython to 7.31.0
HOPSWORKS-2935 jupyter_spec insufficient spark memory test should set python kernel to false
HOPSWORKS-2924 cloud-metrics-daemon ownership is incorrect
HOPSWORKS-2920 exclude org.slf4j:slf4j-log4j12 from zookeeper
HOPSWORKS-2893 SYNC_BASE_ENV should also check for conflicts
HOPSWORKS-2884 DefaultJobConfiguration update fails for PySpark/Spark JobType
HOPSWORKS-2873 Log4j2 vulnerability mitigation
HOPSWORKS-2869 Improve Spark/PySpark logging
HOPSWORKS-2863 parallel conversion of same notebook - add execution id to make name unique
HOPSWORKS-2856 OnlineFS user removal during project removal can result in collision
HOPSWORKS-2839 Fix batch serving vectors sql syntax
HOPSWORKS-2838 Training dataset get_query for offline storage not working on hive engine
HOPSWORKS-2826 OnlineFS replays data on cluster restart
HOPSWORKS-2818 Serialize Date objects correctly in Query Constructor
HOPSWORKS-2817 settings.updateOauthLogoutRedirectUri add a path to the uri
HOPSWORKS-2814 Enabling feature store service on an existing project doesn't create online users for existing members
HOPSWORKS-2813 Should be able to scale out a running serving
HOPSWORKS-2812 Kafka failing to query the database
HOPSWORKS-2809 Fix IllegalAnnotationsException that is polluting the log
HOPSWORKS-2805 Regex for validating library name does not allow square brackets
HOPSWORKS-2800 onlinefs service fails silently to start if Kafka has not fully started yet
HOPSWORKS-2781 Fix timestamp features for Hudi Feature Groups
HOPSWORKS-2779 OAuth callback redirect not working when user is already logged in
HOPSWORKS-2777 executions table grows infinitely and may kill RonDB when deleting a Job or Project
HOPSWORKS-2770 Feature Group clear content endpoint rethinking
HOPSWORKS-2721 Data validation inequality rules should compare columns but accept fractional values
HOPSWORKS-2698 Update hops::fuse_mnt for logging and bug fixes
HOPSWORKS-2592 Dataset href path is malformed
HOPSWORKS-2590 Storage connector should not set empty values
HOPSWORKS-2398 KVM ruby sorting int as string.
HOPSWORKS-2239 NullPointerException when creating a feature group with Null description
HOPSWORKS-2171 Online Feature Group creation without schema through UI fails with uninformative Error
HOPSWORKS-2097 Materialize font awesome icons
HOPSWORKS-2030 Hopsworks Airflow operator doesn't work with python jobs
HOPSWORKS-1867 flyway_schema_history engine should be ndbcluster instead of innodb
HOPSWORKS-1351 Services versions list needs to be updated
Improvement
HOPSWORKS-2955 default train_split property to "train"
HOPSWORKS-2948 Simplify getting shared feature store by name on HSFS
HOPSWORKS-2940 Use consul for livy.rsc.launcher.address configuration
HOPSWORKS-2933 Improve HSFS Python upload memory usage
HOPSWORKS-2926 Adding vscode specific folders to .gitignore
HOPSWORKS-2925 Move SqlCondition and SqlFilterLogic Enums to Persistance Package to make them reusable
HOPSWORKS-2916 use tf.io.parse_example instead of tf.io.parse_single_example
HOPSWORKS-2902 Fix for user management new UI
HOPSWORKS-2899 feature group data preview should default to online feature store
HOPSWORKS-2891 Add tags as a subresource for Models
HOPSWORKS-2888 Support for pyodbc in python environment
HOPSWORKS-2887 OAuth not working if 2fa set to mandatory
HOPSWORKS-2882 Vulnerability check failing in jenkins
HOPSWORKS-2880 Update prometheus version
HOPSWORKS-2872 Migrate auth and project
HOPSWORKS-2865 Disallow TLSv1 and TLSv1.1 at kagent web interface
HOPSWORKS-2853 Save OAuth id_token to be used for logout.
HOPSWORKS-2850 Fine-grained activity logging for validations
HOPSWORKS-2843 default data storage storage policy
HOPSWORKS-2842 Improve recent notebooks endpoint
HOPSWORKS-2836 Add retention policy for services logs
HOPSWORKS-2828 Support serving scikit-learn models and custom predictor scripts with KFServing
HOPSWORKS-2827 New module for managing model serving (HSML)
HOPSWORKS-2820 [consul-chef] Keep existing search domain in resolv.conf
HOPSWORKS-2819 Allow feature comparison in Query Filters
HOPSWORKS-2810 Fixes for oauth new UI
HOPSWORKS-2808 Fix home directory
HOPSWORKS-2804 Remove zookeeper rubygems dependency
HOPSWORKS-2803 Add prometheus rules for RonDb alerting
HOPSWORKS-2799 Use logging module in hops-util-py
HOPSWORKS-2797 Add model name and serving state filters to ServingService
HOPSWORKS-2784 bump up dependency versions to fix vulnerabilities
HOPSWORKS-2780 Configuration parameter for InitialLogFileGroup and InitialTablespace
HOPSWORKS-2774 Rest endpoints for configuring remote authentication
HOPSWORKS-2772 Alert on job fail not working
HOPSWORKS-2771 Include vnd.schemaregistry.v1+json header for kafka registry POST endpoints
HOPSWORKS-2766 Add template_fields to the Airflow HopsworksAbstractOperator
HOPSWORKS-2759 Set HttpOnly and Secure flags for cookies
HOPSWORKS-2745 New API for managing machine learning models
HOPSWORKS-2703 Improve documentation for model serving
HOPSWORKS-2633 Support for Istio authentication
HOPSWORKS-2623 Add Feature normalizer functions as in-built Transformation Functions to HSFS
HOPSWORKS-2617 add batch lookup API to retrieve batch of serving vectors from online feature store
HOPSWORKS-2504 Move feature descriptions from Hive Metastore to Hopsworks
HOPSWORKS-2395 Change s3 connector bucket name tooltip
HOPSWORKS-1882 Sparkmagic should send certificate to Livy
New Feature
HOPSWORKS-2919 Refresh online storage connector if connection gets dropped
HOPSWORKS-2917 Rest API for editing Hopsworks configuration
HOPSWORKS-2829 Expandable links provenance graph
HOPSWORKS-2811 Add ModelRegistry API to fully support shared models
HOPSWORKS-2802 Make it possible to manually set oauth session end point even if provider metadata end point is supported
HOPSWORKS-2750 Hopsworks GIT integration
HOPSWORKS-2680 Save code that it's used to create a feature group
HOPSWORKS-2382 Collect service logs on Elastic
HOPSWORKS-1960 Add support for downloading a model using the python API
Task
HOPSWORKS-2954 Add Old Documentation header to mkdocs.yml to point to readthedocs
HOPSWORKS-2930 Update schema and model_schema paths in test notebooks
HOPSWORKS-2929 Add zip package to hopsworks::install recipe
HOPSWORKS-2913 Bump Hops dependencies to 3.2.0.6
HOPSWORKS-2909 Reduce hopsworks-ear size
HOPSWORKS-2896 Run Airflow in Docker container
HOPSWORKS-2895 Add endpoint to generate elk service log user jwt token
HOPSWORKS-2894 adjust model_spec tests to check new input_example format
HOPSWORKS-2868 Model schema and input example should be expandable
HOPSWORKS-2860 Rename Signature to ModelSchema in ModelDTO
HOPSWORKS-2837 Extend query field size for SQL based on-demand feature group
HOPSWORKS-2835 Zookeeper recipe does not respect external_user option
HOPSWORKS-2833 Add documentation template
HOPSWORKS-2821 Add user scope API key for secrets APIs
HOPSWORKS-2776 Bump hops dependencies to 3.2.0.6-SNAPSHOT
HOPSWORKS-2773 Maven dependency refactoring and vulnerability fixes
HOPSWORKS-2769 Fix error management for python library installation
HOPSWORKS-2767 Update to Python 3.8
HOPSWORKS-1755 Upgrade PyTorch to 1.5
HOPSWORKS-1416 Support for bulk activating users in the Admin UI