Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature store preview #226

Merged
merged 238 commits into from
Nov 15, 2023
Merged

Feature store preview #226

merged 238 commits into from
Nov 15, 2023

Conversation

KshitizLohia
Copy link
Member

@KshitizLohia KshitizLohia commented Jun 12, 2023

Description

  1. Creation of feature store entities as a stack
  2. Spark data-frame in and spark data-frame out interface with spark magic
  3. Pandas data-frame in and pandas data-frame out interface.
  4. Lineage tree in the notebook using graphviz

Documentation

https://feature-store-accelerated-data-science.readthedocs.io/en/latest/

Integrations

  • great expectations: Feature store supports great expectations validations for validation of data as this library seems to be mature, could be run with minimal additional dependencies and there’s a possibility to scale with apache spark.

  • mlm: Feature store supports mlm profiler for statistics of data as this library could be run with minimal additional dependencies and there’s a possibility to scale with apache spark.

  • plotly: Feature store uses plotly to showcase histograms and charts for each feature within a feature group or dataset.

  • delta-spark: Feature store uses delta for each commit to allow time travel of queries and to allow capabilities like schema enforcement and schema evolution.

Unit tests

Screenshot 2023-06-15 at 11 38 16 PM

Integration tests

Screenshot 2023-06-29 at 2 29 46 PM

@oracle-contributor-agreement oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Jun 12, 2023
@KshitizLohia KshitizLohia added the do not merge for any issue that isn't ready for merging yet label Jun 12, 2023
@github-actions
Copy link

📌 Cov diff with main:

No success to gather report. 😿

📌 Overall coverage:

No success to gather report. 😿

@github-actions
Copy link

⚠️ This PR changed setup.py file. ⚠️

  • PR Creator must update 📃 THIRD_PARTY_LICENSES.txt, if any 📚 library added/removed in setup.py.
  • PR Approver must confirm 📃 THIRD_PARTY_LICENSES.txt updated, if any 📚 library added/removed in setup.py.

@KshitizLohia KshitizLohia added draft enhancement New feature or request labels Jun 14, 2023
@github-actions
Copy link

📌 Cov diff with main:

No success to gather report. 😿

📌 Overall coverage:

No success to gather report. 😿

@github-actions
Copy link

⚠️ This PR changed setup.py file. ⚠️

  • PR Creator must update 📃 THIRD_PARTY_LICENSES.txt, if any 📚 library added/removed in setup.py.
  • PR Approver must confirm 📃 THIRD_PARTY_LICENSES.txt updated, if any 📚 library added/removed in setup.py.

@github-actions
Copy link

📌 Cov diff with main:

No success to gather report. 😿

📌 Overall coverage:

No success to gather report. 😿

@github-actions
Copy link

⚠️ This PR changed pyproject.toml file. ⚠️

  • PR Creator must update 📃 THIRD_PARTY_LICENSES.txt, if any 📚 library added/removed in pyproject.toml.
  • PR Approver must confirm 📃 THIRD_PARTY_LICENSES.txt updated, if any 📚 library added/removed in pyproject.toml.

@github-actions
Copy link

📌 Cov diff with main:

Coverage-0%

📌 Overall coverage:

Coverage-62.07%

@github-actions
Copy link

⚠️ This PR changed pyproject.toml file. ⚠️

  • PR Creator must update 📃 THIRD_PARTY_LICENSES.txt, if any 📚 library added/removed in pyproject.toml.
  • PR Approver must confirm 📃 THIRD_PARTY_LICENSES.txt updated, if any 📚 library added/removed in pyproject.toml.

@github-actions
Copy link

📌 Cov diff with main:

Coverage-0%

📌 Overall coverage:

Coverage-62.06%

Copy link

github-actions bot commented Nov 7, 2023

⚠️ This PR changed pyproject.toml file. ⚠️

  • PR Creator must update 📃 THIRD_PARTY_LICENSES.txt, if any 📚 library added/removed in pyproject.toml.
  • PR Approver must confirm 📃 THIRD_PARTY_LICENSES.txt updated, if any 📚 library added/removed in pyproject.toml.

Copy link

github-actions bot commented Nov 7, 2023

📌 Cov diff with main:

Coverage-0%

📌 Overall coverage:

Coverage-26.50%

Copy link

github-actions bot commented Nov 8, 2023

⚠️ This PR changed pyproject.toml file. ⚠️

  • PR Creator must update 📃 THIRD_PARTY_LICENSES.txt, if any 📚 library added/removed in pyproject.toml.
  • PR Approver must confirm 📃 THIRD_PARTY_LICENSES.txt updated, if any 📚 library added/removed in pyproject.toml.

Copy link

github-actions bot commented Nov 8, 2023

📌 Cov diff with main:

No success to gather report. 😿

📌 Overall coverage:

No success to gather report. 😿

1 similar comment
Copy link

github-actions bot commented Nov 8, 2023

📌 Cov diff with main:

No success to gather report. 😿

📌 Overall coverage:

No success to gather report. 😿

liudmylaru
liudmylaru previously approved these changes Nov 13, 2023
mrDzurb
mrDzurb previously approved these changes Nov 14, 2023
@harsh97 harsh97 dismissed stale reviews from mrDzurb and liudmylaru via 24aed7f November 15, 2023 09:25
Copy link

⚠️ This PR changed pyproject.toml file. ⚠️

  • PR Creator must update 📃 THIRD_PARTY_LICENSES.txt, if any 📚 library added/removed in pyproject.toml.
  • PR Approver must confirm 📃 THIRD_PARTY_LICENSES.txt updated, if any 📚 library added/removed in pyproject.toml.

Copy link

📌 Cov diff with main:

Coverage-0%

📌 Overall coverage:

Coverage-61.99%

@mrDzurb mrDzurb merged commit 2f9be39 into main Nov 15, 2023
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request OCA Verified All contributors have signed the Oracle Contributor Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants