docs: upgrade_0.8.0 (#3257)

CHANGELOG.md

@@ -1,5 +1,36 @@
 # Changelog
 
+## [0.8.0] - 2023-05-11
+
+### Features
+- Add a new synchronization tool to automatically synchronize data from online storage to offline storage (#3256 @vagetablechicken)
+- Support the new built-in functions `var_samp`, `var_pop`, `entropy`, `earth_distance`, `nth_value_where` and `add_months` (#3046 #3193 @aceforeverd)
+- Support batch read for openmldb-spark-connector (#3070 @tobegit3hub)
+- [Alpha] Support Kubernetes as a TaskManager backend for the offline engine (#3147 #3157 #3185 @tobegit3hub)
+- Support LAST JOIN over WHERE clause (#3134 @aceforeverd)
+- Support LAST JOIN in WINDOW UNION clause (#3205 @aceforeverd)
+- Supports a decimal place as the second parameter in the function `round` (#3221 @aceforeverd)
+- Support Amazon S3 as an offline data source (#3229 #3261 @tobegit3hub)
+- Enhance the diagnostic tool (#3224 #3208 @zhangziheng01233)
+- Add the new option `SKIP_INDEX_CHECK` to skip index checking when deploying SQLs (#3241 @dl239)
+- Support symbolic paths for offline table (#3235 @tobegit3hub)
+- Improve the documents (#3104 #2993 @selenachenjingxin, #3113 #3118 #3239 @tobegit3hub, #3150 #3184 #3237 #3255 @aceforeverd, #3160 #3195 #3197 #3223 @lumianph, #3192 #3215 @haseeb-xd, #3201 #3220 #3232 #3236 #3254 @vagetablechicken, #3213 @alexab612, #3189 #3199 @TanZiYen)
+- Other minor features (#3115 #3143 #3182 @tobegit3hub, #2818 #3123 @aceforeverd, #3128 #3127 @dl239)
+
+### Bug Fixes
+- There is curator conflict problem when executing offline SQLs under certain circumstances. (#3090 @tobegit3hub)
+- `CREATE TABLE ... LIKE HIVE ...` statement execution fails if there is no database in a table name. (#3063 @tobegit3hub)
+- The CLI shows `success` even though `CREATE TABLE ... LIKE ...` execution fails. (#3080 @tobegit3hub)
+- `SELECT ... INTO ...` statement execution fails in the offline mode if there is no source table. (#3116 @tobegit3hub)
+- Compilation fails when `SELECT * ` over two LAST JOIN. (#3117 @aceforeverd)
+- The syncing job thread will enter an infinite loop if querying `JOB_INFO` fails. (#3169 @vagetablechicken)
+- SQLs deployment fails if there are more than one conditions on a JOIN statement. (#3196 @vagetablechicken)
+- Registered tables cannot be fetched when enabling SparkSQL for the offline engine (#3057 @tobegit3hub)
+- Other minor bug fixes (#3097 #3095 @dl239, #3109 #3141 #3162 #3234 @aceforeverd, #3096 #3112 @tobegit3hub, #3231 #3251 @vagetablechicken)
+
+### Code Refactoring
+#3188 @tobegit3hub
+
 ## [0.7.3] - 2023-03-22
 
 ### Features
@@ -546,6 +577,7 @@ Removed
 - openmldb-0.2.0-linux.tar.gz targets on x86_64
 - aarch64 artifacts consider experimental
 
+[0.8.0]: https://github.com/4paradigm/OpenMLDB/compare/v0.7.3...v0.8.0
 [0.7.3]: https://github.com/4paradigm/OpenMLDB/compare/v0.7.2...v0.7.3
 [0.7.2]: https://github.com/4paradigm/OpenMLDB/compare/v0.7.1...v0.7.2
 [0.7.1]: https://github.com/4paradigm/OpenMLDB/compare/v0.7.0...v0.7.1

demo/java_quickstart/demo/pom.xml

@@ -29,7 +29,7 @@
     <dependency>
       <groupId>com.4paradigm.openmldb</groupId>
       <artifactId>openmldb-jdbc</artifactId>
-      <version>0.7.3</version>
+      <version>0.8.0</version>
     </dependency>
     <dependency>
       <groupId>org.testng</groupId>

demo/jd-recommendation/README.md

@@ -11,7 +11,7 @@ For full instructions, please refer to [website page](http://openmldb.ai/docs/zh
 ## OpenMLDB Feature Extraction
 
 ```
-docker run -dit --name=openmldb --network=host -v $demodir:/work/oneflow_demo 4pdosc/openmldb:0.7.3 bash
+docker run -dit --name=openmldb --network=host -v $demodir:/work/oneflow_demo 4pdosc/openmldb:0.8.0 bash
 docker exec -it openmldb bash
 ```
 

demo/predict-taxi-trip-duration/README.md

@@ -28,7 +28,7 @@ w2 as (PARTITION BY passenger_count ORDER BY pickup_datetime ROWS_RANGE BETWEEN
 
 **Start docker**
 ```
-docker run -it 4pdosc/openmldb:0.7.3 bash
+docker run -it 4pdosc/openmldb:0.8.0 bash
 ```
 **Initialize environment**
 ```bash
@@ -138,7 +138,7 @@ python3 predict.py
 **Start docker**
 
 ```bash
-docker run -it 4pdosc/openmldb:0.7.3 bash
+docker run -it 4pdosc/openmldb:0.8.0 bash
 ```
 **Initialize environment**
 

demo/talkingdata-adtracking-fraud-detection/README.md

@@ -15,7 +15,7 @@ We recommend you to use docker to run the demo. OpenMLDB and dependencies have b
 **Start docker**
 
 ```
-docker run -it 4pdosc/openmldb:0.7.3 bash
+docker run -it 4pdosc/openmldb:0.8.0 bash
 ```
 
 #### Run locally

docs/en/deploy/compile.md

@@ -7,25 +7,25 @@
 This section describes the steps to compile and use OpenMLDB inside its official docker image [hybridsql](https://hub.docker.com/r/4pdosc/hybridsql).
 The docker image has packed required tools and dependencies, so there is no need to set them up separately. To compile without the official docker image, refer to the section [Detailed Instructions for Build](#detailed-instructions-for-build) below.
 
-Keep in mind that you should always use the same version of both compile image and [OpenMLDB version](https://github.com/4paradigm/OpenMLDB/releases). This section demonstrates compiling for [OpenMLDB v0.7.3](https://github.com/4paradigm/OpenMLDB/releases/tag/v0.7.3) under `hybridsql:0.7.3` ，If you prefer to compile on the latest code in `main` branch, pull `hybridsql:latest` image instead.
+Keep in mind that you should always use the same version of both compile image and [OpenMLDB version](https://github.com/4paradigm/OpenMLDB/releases). This section demonstrates compiling for [OpenMLDB v0.8.0](https://github.com/4paradigm/OpenMLDB/releases/tag/v0.8.0) under `hybridsql:0.8.0` ，If you prefer to compile on the latest code in `main` branch, pull `hybridsql:latest` image instead.
 
 1. Pull the docker image
 
    ```bash
-    docker pull 4pdosc/hybridsql:0.7
+    docker pull 4pdosc/hybridsql:0.8
    ```
 
 2. Create a docker container with the hybridsql docker image
 
    ```bash
-   docker run -it 4pdosc/hybridsql:0.7 bash
+   docker run -it 4pdosc/hybridsql:0.8 bash
    ```
 
-3. Download the OpenMLDB source code inside the docker container, and setting the branch into v0.7.3
+3. Download the OpenMLDB source code inside the docker container, and setting the branch into v0.8.0
 
    ```bash
    cd ~
-   git clone -b v0.7.3 https://github.com/4paradigm/OpenMLDB.git
+   git clone -b v0.8.0 https://github.com/4paradigm/OpenMLDB.git
    ```
 
 4. Compile OpenMLDB
@@ -151,7 +151,7 @@ The built jar packages are in the `target` path of each submodule. If you want t
 1. Downloading the pre-built OpenMLDB Spark distribution:
 
 ```bash
-wget https://github.com/4paradigm/spark/releases/download/v3.2.1-openmldb0.7.3/spark-3.2.1-bin-openmldbspark.tgz
+wget https://github.com/4paradigm/spark/releases/download/v3.2.1-openmldb0.8.0/spark-3.2.1-bin-openmldbspark.tgz
 ```
 
 Alternatively, you can also download the source code and compile from scratch:

docs/en/deploy/install_deploy.md

@@ -9,7 +9,7 @@
   * The number of cores is recommended to be no less than 4 cores. If the CPU does not support the AVX2 instruction set in the Linux environment, the deployment package needs to be recompiled from the source code.
 
 ## Deployment Package
-The precompiled OpenMLDB deployment package is used by default in this documentation ([Linux](https://github.com/4paradigm/OpenMLDB/releases/download/v0.7.3/openmldb-0.7.3-linux.tar.gz) , [macOS](https://github.com/4paradigm/OpenMLDB/releases/download/v0.7.3/openmldb-0.7.3-darwin.tar.gz)), the supported operating system requirements are: CentOS 7, Ubuntu 20.04, macOS >= 10.15. If the user wishes to compile by himself (for example, for OpenMLDB source code development, the operating system or CPU architecture is not in the support list of the precompiled deployment package, etc.), the user can choose to compile and use in the docker container or compile from the source code. For details, please refer to our [compile documentation](compile.md).
+The precompiled OpenMLDB deployment package is used by default in this documentation ([Linux](https://github.com/4paradigm/OpenMLDB/releases/download/v0.8.0/openmldb-0.8.0-linux.tar.gz) , [macOS](https://github.com/4paradigm/OpenMLDB/releases/download/v0.8.0/openmldb-0.8.0-darwin.tar.gz)), the supported operating system requirements are: CentOS 7, Ubuntu 20.04, macOS >= 10.15. If the user wishes to compile by himself (for example, for OpenMLDB source code development, the operating system or CPU architecture is not in the support list of the precompiled deployment package, etc.), the user can choose to compile and use in the docker container or compile from the source code. For details, please refer to our [compile documentation](compile.md).
 
 ## Configure Environment (Linux)
 
@@ -91,9 +91,9 @@ OpenMLDB standalone version needs to deploy a nameserver and a tablet. The names
 ### Download the OpenMLDB Release Package
 
 ```
-wget https://github.com/4paradigm/OpenMLDB/releases/download/v0.7.3/openmldb-0.7.3-linux.tar.gz
-tar -zxvf openmldb-0.7.3-linux.tar.gz
-cd openmldb-0.7.3-linux
+wget https://github.com/4paradigm/OpenMLDB/releases/download/v0.8.0/openmldb-0.8.0-linux.tar.gz
+tar -zxvf openmldb-0.8.0-linux.tar.gz
+cd openmldb-0.8.0-linux
 ```
 
 ### Configuration
@@ -163,9 +163,9 @@ Environment Requirements:
 ### Download the OpenMLDB Deployment Package
 
 ```
-wget https://github.com/4paradigm/OpenMLDB/releases/download/v0.7.3/openmldb-0.7.3-linux.tar.gz
-tar -zxvf openmldb-0.7.3-linux.tar.gz
-cd openmldb-0.7.3-linux
+wget https://github.com/4paradigm/OpenMLDB/releases/download/v0.8.0/openmldb-0.8.0-linux.tar.gz
+tar -zxvf openmldb-0.8.0-linux.tar.gz
+cd openmldb-0.8.0-linux
 ```
 
 ### Configuration
@@ -175,7 +175,7 @@ which are listed below.
 
 | Environment Variables             | Default Values                                          | Definitons                                                                                            |
 |-----------------------------------|---------------------------------------------------------|-------------------------------------------------------------------------------------------------------|
-| OPENMLDB_VERSION                  | 0.7.3                                                   | OpenMLDB version                                                                                      |
+| OPENMLDB_VERSION                  | 0.8.0                                                   | OpenMLDB version                                                                                      |
 | OPENMLDB_MODE                     | standalone                                              | standalone or cluster mode                                                                            |
 | OPENMLDB_HOME                     | root directory of the release folder                    | openmldb root path                                                                                    |
 | SPARK_HOME                        | $OPENMLDB_HOME/spark                                    | the root path of openmldb spark release. if not exists, download from online                          |
@@ -311,10 +311,10 @@ Deploy the Zookeeper cluster [refer to here](https://zookeeper.apache.org/doc/r3
 #### 1. Download the OpenMLDB Deployment Package
 
 ```
-wget https://github.com/4paradigm/OpenMLDB/releases/download/v0.7.3/openmldb-0.7.3-linux.tar.gz
-tar -zxvf openmldb-0.7.3-linux.tar.gz
-mv openmldb-0.7.3-linux openmldb-tablet-0.7.3
-cd openmldb-tablet-0.7.3
+wget https://github.com/4paradigm/OpenMLDB/releases/download/v0.8.0/openmldb-0.8.0-linux.tar.gz
+tar -zxvf openmldb-0.8.0-linux.tar.gz
+mv openmldb-0.8.0-linux openmldb-tablet-0.8.0
+cd openmldb-tablet-0.8.0
 ```
 
 #### 2. Modify the Configuration File: conf/tablet.flags
@@ -361,10 +361,10 @@ Repeat the above steps to deploy multiple tablets.
 #### 1. Download the OpenMLDB Deployment Package
 
 ```
-wget https://github.com/4paradigm/OpenMLDB/releases/download/v0.7.3/openmldb-0.7.3-linux.tar.gz
-tar -zxvf openmldb-0.7.3-linux.tar.gz
-mv openmldb-0.7.3-linux openmldb-ns-0.7.3
-cd openmldb-ns-0.7.3
+wget https://github.com/4paradigm/OpenMLDB/releases/download/v0.8.0/openmldb-0.8.0-linux.tar.gz
+tar -zxvf openmldb-0.8.0-linux.tar.gz
+mv openmldb-0.8.0-linux openmldb-ns-0.8.0
+cd openmldb-ns-0.8.0
 ```
 
 #### 2. Modify the Configuration File: conf/nameserver.flags
@@ -411,10 +411,10 @@ Before running, make sure that the OpenMLDB cluster has been started, otherwise
 #### 1. Download the OpenMLDB Deployment Package
 
 ```
-wget https://github.com/4paradigm/OpenMLDB/releases/download/v0.7.3/openmldb-0.7.3-linux.tar.gz
-tar -zxvf openmldb-0.7.3-linux.tar.gz
-mv openmldb-0.7.3-linux openmldb-apiserver-0.7.3
-cd openmldb-apiserver-0.7.3
+wget https://github.com/4paradigm/OpenMLDB/releases/download/v0.8.0/openmldb-0.8.0-linux.tar.gz
+tar -zxvf openmldb-0.8.0-linux.tar.gz
+mv openmldb-0.8.0-linux openmldb-apiserver-0.8.0
+cd openmldb-apiserver-0.8.0
 ```
 
 #### 2. Modify the Configuration File: conf/apiserver.flags
@@ -454,12 +454,12 @@ TaskManager can be deployed in single server. You can deploy multiple instances
 #### 1. Download the OpenMLDB Spark Distribution that is Optimized for Feature Engineering
 
 ```
-wget https://github.com/4paradigm/spark/releases/download/v3.2.1-openmldb0.7.3/spark-3.2.1-bin-openmldbspark.tgz
+wget https://github.com/4paradigm/spark/releases/download/v3.2.1-openmldb0.8.0/spark-3.2.1-bin-openmldbspark.tgz
 tar -zxvf spark-3.2.1-bin-openmldbspark.tgz
-wget https://github.com/4paradigm/OpenMLDB/releases/download/v0.7.3/openmldb-0.7.3-linux.tar.gz
-tar -zxvf openmldb-0.7.3-linux.tar.gz
-mv openmldb-0.7.3-linux openmldb-taskmanager-0.7.3
-cd openmldb-taskmanager-0.7.3
+wget https://github.com/4paradigm/OpenMLDB/releases/download/v0.8.0/openmldb-0.8.0-linux.tar.gz
+tar -zxvf openmldb-0.8.0-linux.tar.gz
+mv openmldb-0.8.0-linux openmldb-taskmanager-0.8.0
+cd openmldb-taskmanager-0.8.0
 ```
 
 #### 2. Modify the Configuration File conf/taskmanager.properties

docs/en/quickstart/java_sdk.md

@@ -9,12 +9,12 @@ Configure maven pom
 <dependency>
     <groupId>com.4paradigm.openmldb</groupId>
     <artifactId>openmldb-jdbc</artifactId>
-    <version>0.7.3</version>
+    <version>0.8.0</version>
 </dependency>
 <dependency>
     <groupId>com.4paradigm.openmldb</groupId>
     <artifactId>openmldb-native</artifactId>
-    <version>0.7.3</version>
+    <version>0.8.0</version>
 </dependency>
 ```
 ### Package Installation on Mac
@@ -24,15 +24,15 @@ Configure maven pom
 <dependency>
     <groupId>com.4paradigm.openmldb</groupId>
     <artifactId>openmldb-jdbc</artifactId>
-    <version>0.7.3</version>
+    <version>0.8.0</version>
 </dependency>
 <dependency>
     <groupId>com.4paradigm.openmldb</groupId>
     <artifactId>openmldb-native</artifactId>
-    <version>0.7.3-macos</version>
+    <version>0.8.0-macos</version>
 </dependency>
 ```
-Note that since `openmldb-native` contains the C++ static library compiled by OpenMLDB, by default it is a Linux's static library. On macOS, the version of the above openmldb-native needs to be changed to `0.7.3-macos`, and the version of openmldb-jdbc remains unchanged.
+Note that since `openmldb-native` contains the C++ static library compiled by OpenMLDB, by default it is a Linux's static library. On macOS, the version of the above openmldb-native needs to be changed to `0.8.0-macos`, and the version of openmldb-jdbc remains unchanged.
 
 The macOS native relase only supports macos-12. If you want use in macos-11 or macos 10.15, you should build openmldb-native from source in macos-11/macos-10.15, see [Build Java SDK](../deploy/compile.md#build-java-sdk-with-multi-processes) for details.
 

docs/en/quickstart/openmldb_quickstart.md

@@ -17,7 +17,7 @@ If you wan to compile and install it by yourself, you can refer to our [installa
 Pull the image (image download size is about 1GB, after decompression is about 1.7 GB) and start the docker container:
 
 ```bash
-docker run -it 4pdosc/openmldb:0.7.3 bash
+docker run -it 4pdosc/openmldb:0.8.0 bash
 ```
 
 ```{important}

docs/en/reference/ip_tips.md

@@ -38,12 +38,12 @@ Expose the port through `-p` when starting the container, and the client can acc
 
 The stand-alone version needs to expose the ports of three components (nameserver, tabletserver, apiserver):
 ```
-docker run -p 6527:6527 -p 9921:9921 -p 8080:8080 -it 4pdosc/openmldb:0.7.3 bash
+docker run -p 6527:6527 -p 9921:9921 -p 8080:8080 -it 4pdosc/openmldb:0.8.0 bash
 ```
 
 The cluster version needs to expose the zk port and the ports of all components:
 ```
-docker run -p 2181:2181 -p 7527:7527 -p 10921:10921 -p 10922:10922 -p 8080:8080 -p 9902:9902 -it 4pdosc/openmldb:0.7.3 bash
+docker run -p 2181:2181 -p 7527:7527 -p 10921:10921 -p 10922:10922 -p 8080:8080 -p 9902:9902 -it 4pdosc/openmldb:0.8.0 bash
 ```
 
 ```{tip}
@@ -57,7 +57,7 @@ If the OpenMLDB service process is distributed, the "port number is occupied" ap
 #### Host Network
 Or more conveniently, use host networking without port isolation, for example:
 ```
-docker run --network host -it 4pdosc/openmldb:0.7.3 bash
+docker run --network host -it 4pdosc/openmldb:0.8.0 bash
 ```
 But in this case, it is easy to find that the port is occupied by other processes in the host. If occupancy occurs, change the port number carefully.
 

docs/en/use_case/JD_recommendation_en.md

@@ -52,7 +52,7 @@ Oneflow-serving：https://github.com/Oneflow-Inc/serving/tree/ce5d667468b6b3ba66
 Pull the OpenMLDB docker image and run.
 
 ```bash
-docker run -dit --name=openmldb --network=host -v $demodir:/work/oneflow_demo 4pdosc/openmldb:0.7.3 bash
+docker run -dit --name=openmldb --network=host -v $demodir:/work/oneflow_demo 4pdosc/openmldb:0.8.0 bash
 docker exec -it openmldb bash
 ```
 

docs/en/use_case/airflow_provider_demo.md

@@ -34,7 +34,7 @@ For the newest version, please visit [GitHub example_dags](https://github.com/4p
 - Please project the previously downloaded files to the path `/work/airflow/dags`, where Airflow will access for the DAG.
 
 ```
-docker run -p 8080:8080 -v `pwd`/airflow_demo_files:/work/airflow/dags -it 4pdosc/openmldb:0.7.3 bash
+docker run -p 8080:8080 -v `pwd`/airflow_demo_files:/work/airflow/dags -it 4pdosc/openmldb:0.8.0 bash
 ```
 
 #### 0.3 Download and Install the Airflow and the Airflow OpenMLDB Provider 

docs/en/use_case/dolphinscheduler_task_demo.md

@@ -33,7 +33,7 @@ In addition to the feature engineering done by OpenMLDB, the prediction also req
 
 The demo can run on MacOS or Linux, the OpenMLDB docker image is recommended. We'll start OpenMLDB and DolphinScheduler in the same container, expose the DolphinScheduler web port:
 ```
-docker run -it -p 12345:12345 4pdosc/openmldb:0.7.3 bash
+docker run -it -p 12345:12345 4pdosc/openmldb:0.8.0 bash
 ```
 
 ```{attention}

docs/en/use_case/kafka_connector_demo.md

@@ -22,7 +22,7 @@ For OpenMLDB Kafka Connector implementation, please refer to [extensions/kafka-c
 This article will start the OpenMLDB in docker container, so there is no need to download the OpenMLDB separately. Moreover, Kafka and connector can be started in the same container. We recommend that you save the three downloaded packages to the same directory. Let's assume that the packages are in the `/work/kafka` directory.
 
 ```
-docker run -it -v `pwd`:/work/kafka --name openmldb 4pdosc/openmldb:0.7.3 bash
+docker run -it -v `pwd`:/work/kafka --name openmldb 4pdosc/openmldb:0.8.0 bash
 ```
 
 ### Steps

docs/en/use_case/lightgbm_demo.md

@@ -13,7 +13,7 @@ Note that: (1) this case is based on the OpenMLDB cluster version for tutorial d
 - Pull the OpenMLDB docker image and run the corresponding container:
 
 ```bash
-docker run -it 4pdosc/openmldb:0.7.3 bash
+docker run -it 4pdosc/openmldb:0.8.0 bash
 ```
 
 The image is preinstalled with OpenMLDB and preset with all scripts, third-party libraries, open-source tools and training data required for this case.

docs/en/use_case/pulsar_connector_demo.md

@@ -29,7 +29,7 @@ Only OpenMLDB cluster mode can be the sink dist, and only write to online storag
 
 We recommend that you use ‘host network’ to run docker. And bind volume ‘files’ too. The sql scripts are in it.
 ```
-docker run -dit --network host -v `pwd`/files:/work/pulsar_files --name openmldb 4pdosc/openmldb:0.7.3 bash
+docker run -dit --network host -v `pwd`/files:/work/pulsar_files --name openmldb 4pdosc/openmldb:0.8.0 bash
 docker exec -it openmldb bash
 ```
 ```{note}

docs/en/use_case/talkingdata_demo.md

@@ -13,7 +13,7 @@ It is recommended to run this demo in Docker. Please make sure that OpenMLDB and
 **Start the OpenMLDB Docker Image**
 
 ```
-docker run -it 4pdosc/openmldb:0.7.3 bash
+docker run -it 4pdosc/openmldb:0.8.0 bash
 ```
 
 #### 1.1.2 Run Locally