Skip to content

Latest commit

 

History

History
179 lines (133 loc) · 6.85 KB

TODO.md

File metadata and controls

179 lines (133 loc) · 6.85 KB
Raw

TODO

version 2.0.0

  • Clear all Flask extensions and implement only the code without any external packages

Version 1.0.0

Pending

  • Changing base code from Flask to FastAPI and use Alembic and SQLAlchemy/Tortoise for database migration and database ORM
  • Changing psycogpg2 to psycogpg3 (alias name is psycopg)

Version 0.3.0.b

Pending

  • Adding async function with Quart (It should be possible to migrate to Quart from Flask)
    • Quart has add_background_task method (migrate from threading)
    • Support asyncpg
  • Adding another database types like SQLite, MySQL, or SQL Server

Version 0.3.0

  • Reduce extension code which does not use or useless

    • Reduce static file
    • Reduce Flask's extension packages

  • Design Pages and Capabilities flexible.

  • Fixing #? change transaction table to **partition table

  • Adding Data Quality Views in Database

  • Add Pydantic for type validator and receive JSON data

  • Adding swagger for API documents

  • Adding Flask Extensions for control I/O if application

  • Change physical of transaction table from default to partition by range of date

  • Change controlling thread for run data pipeline to Flask-Executor

  • Add adjustment flow for imp_mix_max_mos_criteria_fc/rdc/dc values

  • Add Pull and Push Models on Control Object for any control tables

  • Change the objects file from legacy slot classes to Pydantic class supported

  • Change trigger value from list to tuple object

  • Change swagger from flasgger to flask-swagger-ui

  • Fix merge PUT and DELETE methods together with same endpoint

    • Add module for delete data from platform with payload

Version 0.2.0

  • Adding and Fixing #? Ingestion
    • Add updatable merge from payload
    • Add updatable common from payload
  • Change level of utils directory to inside core directory
  • Adding unittests for core directory
    • Add validators test
    • Add statements test
  • Documents rename from DAF to DFA

Version 0.1.0

  • Adding unittests
    • Add to core directory
  • Fixing #? bug of re-create function when start this application at the first time
  • Compare Dataclasses vs Attrs vs Pydantic
    • Change the base file from legacy slot classes to Pydantic models
      • Change base file to validators and statements files
      • Add load object for loading yaml file
    • Add status Enum object

Version 0.0.1

  • Adding Migrate table properties process

  • Adding Load file from local to target table

  • Change app framework from function to class instance framework code

  • Add frontend for monitoring and operation data pipelines

    • Add Home page
    • Adding Flask-Login for login and register process
    • Add Login and Register page

  • Adding Web catalogs for initialize authentication process

    • Add User registration
    • Add Role assignment
    • Add Group assignment
    • Add Policy assignment
    • Mapping all relation together

Note:
The Version 0.*.* still support requirement from pre-version 1.*.* and pre-Version 2.*.*.
We can deploy only the backend mode (API mode) by this command; python manage.py --api=True.

Pre Version 2.1.0

  • Add foreign key feature in table configuration
  • Adding initial data with Json files with key file or files

Pre Version 2.0.4

  • Add new table (ai_report_forecast_brand) for sync data to forecast adjust dashboard
  • Add new table (ai_report_article_listing_master) for channel value mapping to platform
  • Fixing #? condition for rounding max_qty value to integer type
  • Change adjustment logic for product class and min/max value
  • Adding ctr_task_schedule for control trigger for the data pipeline running
    • Add schedule, trigger, for run data pipeline from updated S3 files trigger
    • Add schedule, cron job, for run data pipeline with manual value
  • Adding ingest action mode, update, for update data to target table
  • Adding ctr_s3_logging for keep log from AWS Batch Job which ingest data from S3 to Database with src_ prefix tables
  • Fixing #? logic of calculation rolling data for product class
  • Fixing #? logic of filter condition data for min max
  • Adding ai_article_vendor_master
  • Adding filter cat_mch3_code from source file to article master table
  • Adding refresh table (vacuum) process in retention module

Pre Version 2.0.3

  • Add new source file from AWS S3 (sales_order)
  • Re-structure of actual_sales_transaction
  • Add new field for keep frequency for product_class_freq
  • Change logic for get default product_class with product_class_freq only
  • Change logic of min/max value with custom Month Of Supply (MOS) such as {A: 2, B: 2, C: 1} receive with imp_min_max_mos_adjust_fc/rdc/dc tables
  • Change table_type of replenishment tables from transaction to report

Pre Version 2.0.2

  • Adding Flask-APSchedule for re-create tables, which solving table storage
  • Adding Ingestion Module for put data from platform in merge case
  • Adding Analytic a table dependency from catalog config
  • Change data type from double precision to numeric

Pre Version 2.0.1

  • Adding Flask-APSchedule for control retention module and task checker
  • Adding Ingestion Module for put data from platform in common case
    • Adding GET request for check background task heartbeat after ingest data
  • Adding RDC hierarchy level between DC and Franchise
  • Change catalog config format from create statement to mapping features
  • Fixing #? adjustment and article replacement logic
  • Fixing #? allocate, proportion, and accumulate logic

Pre Version 2.0.0

  • Adding GET request for check background task heartbeat after run data pipeline
  • Fixing #? the normal flow was changed to foreground task function
  • Adding support background task
  • Adding support task monitoring with logging/process table in database
  • Adding component which keep framework and analytic routes

Note:
pre-version from 1.x.x to 2.x.x was change application file skeleton*

Pre Version 1.1.1

  • Adding close_running function for server down scenario
  • Adding GET request for shutdown framework application
  • Fixing #? GET request for health check without and include headers
  • Adding logging handler
  • Fixing #? config *.yaml file can keep more than one table process
  • Fixing #? order quantity report logic

Pre Version 1.1.0

  • Adding support filter process name in node function
  • Adding support pipeline function for run data with multi-table processes

Pre Version 1.0.0

  • Initial release
  • Original creation, 2021-10-01