Use Airflow, Kafka, Spark, and Cassandra to establish a data pipeline.
It will fetch daily trading information of Taiwan stocks from TWSE API every day.
-
Airflow: do fetch task.
-
Kafka: we can see the stock information.
-
Cassandra: finally we store data here.
