Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CUR proxy + CUR 2.0 - Major v4.0.0 #807

Merged
merged 105 commits into from
Nov 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
105 commits
Select commit Hold shift + click to select a range
cac1ad6
add cur proxy
iakov-aws Apr 30, 2024
9a5a6fc
wip
iakov-aws Apr 30, 2024
542bb20
wip
iakov-aws Apr 30, 2024
73f90b8
add more
iakov-aws Apr 30, 2024
c9dcbf5
wip
iakov-aws May 1, 2024
73ab2e3
wip
iakov-aws May 1, 2024
ea3ab1c
more fixes
iakov-aws May 2, 2024
5759257
add maps
iakov-aws May 6, 2024
47786f9
fix propagation of fields
iakov-aws May 6, 2024
30df100
add proxy cur command
iakov-aws May 6, 2024
1b52025
add sql diff and some refactoring
iakov-aws May 11, 2024
2f6e5c1
Merge branch 'main' into cur-proxy
iakov-aws May 29, 2024
7f81681
refactore account map
iakov-aws May 30, 2024
79b8de1
refactoring for account map handling
iakov-aws Jun 1, 2024
52e950e
set verbosity
iakov-aws Jun 2, 2024
4de86b5
minor fixes
iakov-aws Jun 2, 2024
16b5c2b
minor fixes
iakov-aws Jun 2, 2024
164c6e3
add doc
iakov-aws Jun 2, 2024
c3cdd3e
fix update procedure
iakov-aws Jun 3, 2024
215e9d3
add account_map_cur2.sql
iakov-aws Jun 3, 2024
f76e818
add plugin dashboard deployment
iakov-aws Jun 20, 2024
6478374
add plugin dashboard deployment
iakov-aws Jun 20, 2024
63f00e5
add plugin dashboard deployment
iakov-aws Jun 20, 2024
7268ee6
Update cid-plugin.yml
iakov-aws Jun 21, 2024
bf875e5
Merge branch 'main' into cur-proxy
iakov-aws Jun 22, 2024
5f33014
allow cur database to be different from the main one
iakov-aws Jun 23, 2024
63b04a4
minor fixes
iakov-aws Jun 23, 2024
26c725c
wip
iakov-aws Jun 25, 2024
7ba2640
filter glue catalogs
iakov-aws Jun 25, 2024
65dfe5d
filter glue catalogs
iakov-aws Jun 25, 2024
0006aab
fix account map
iakov-aws Jun 25, 2024
42eb8d9
fix the cur fetching
iakov-aws Jun 25, 2024
b33fff8
wip
iakov-aws Jun 25, 2024
6ede8f1
Merge
iakov-aws Jun 25, 2024
0ce186b
wip
iakov-aws Jun 25, 2024
fb5c228
wip
iakov-aws Jun 25, 2024
b8a2279
Merge branch 'main' into cur-proxy
iakov-aws Jun 28, 2024
e57b4e6
Updated permissions and crawler check logic (#872)
petrokashlikov Jun 29, 2024
981cbdf
Merge branch 'main' into cur-proxy
iakov-aws Jul 2, 2024
edb0d60
avoid sql issues
iakov-aws Jul 2, 2024
b69bfbc
fix cli help
iakov-aws Jul 2, 2024
3124f05
remove a warning and add done at the end of proxy command
iakov-aws Jul 3, 2024
2432b67
add sp/ri columns
iakov-aws Jul 4, 2024
a9b6cd6
WIP (#859)
petrokashlikov Jul 11, 2024
cbb363d
cleanup non ri or non sp views as with proxy we do not need them anymore
iakov-aws Jul 19, 2024
f65eb73
typo
iakov-aws Jul 19, 2024
120c9af
merge
iakov-aws Jul 26, 2024
9f7d002
switch all foundationalviews to cur2
iakov-aws Jul 27, 2024
a52db35
fix merge errors
iakov-aws Jul 27, 2024
246d99a
fix sql
iakov-aws Jul 27, 2024
2cd37d3
minor restyling of sql
iakov-aws Jul 27, 2024
68dc7e0
fixes for trend
iakov-aws Jul 27, 2024
11fbc99
rename ri-sp to default
iakov-aws Jul 27, 2024
8595faf
aling cur2 parameters
iakov-aws Jul 27, 2024
992b6d6
support mulitdb
iakov-aws Jul 31, 2024
438b9b8
merge
iakov-aws Aug 1, 2024
0e78f22
Merge branch 'main' into cur-proxy
iakov-aws Aug 1, 2024
7dcffb8
fix map update
iakov-aws Aug 12, 2024
1a2bd1b
Merge branch 'main' into cur-proxy
iakov-aws Aug 16, 2024
cb558b5
update marketplace
iakov-aws Aug 21, 2024
0278aaa
update kpi
iakov-aws Aug 21, 2024
ed3abb8
updates
iakov-aws Aug 21, 2024
6cedd84
add tags and refactor
iakov-aws Aug 22, 2024
903f9b3
reduse verbosity
iakov-aws Aug 22, 2024
c4faa51
fix cur1 to cur2 update and add some tracing
iakov-aws Aug 22, 2024
046bff3
adjustt time dates
iakov-aws Aug 23, 2024
274c612
fix ri-sp mapping
iakov-aws Aug 23, 2024
4b424ad
refactoring and fixes
iakov-aws Aug 23, 2024
4c9a29b
refactoring to speed up deployment
iakov-aws Aug 24, 2024
a791e75
fix test
iakov-aws Aug 25, 2024
73d6afc
fix test
iakov-aws Aug 25, 2024
05a390b
fix extended-support-cost-projection
iakov-aws Sep 4, 2024
64bae55
Merge branch 'main' into add-cfn-plugins
iakov-aws Sep 4, 2024
087e5fc
Merge branch 'main' into add-cfn-plugins
iakov-aws Sep 4, 2024
a59bb01
fix cfn
iakov-aws Sep 4, 2024
8067f97
wip
iakov-aws Sep 5, 2024
6b3e005
Merge branch 'main' into cur-proxy
iakov-aws Sep 5, 2024
83eae05
revert to string
iakov-aws Sep 18, 2024
4c6580d
revert savings_plan_end_time
iakov-aws Sep 18, 2024
0d46aa1
merge
iakov-aws Sep 18, 2024
1141b7a
fix mkp formatting
iakov-aws Sep 23, 2024
0d09fa6
add data provider management
iakov-aws Sep 24, 2024
28c517c
Merge branch 'main' into cur-proxy-and-dataprovider-managment
iakov-aws Sep 24, 2024
16b1dfc
more changes to support data providers and cur 2.0
iakov-aws Sep 24, 2024
4bb1fa6
Merge branch 'main' into cur-proxy
iakov-aws Sep 24, 2024
da415d5
Merge branch 'cur-proxy-and-dataprovider-managment' into cur-proxy
iakov-aws Sep 24, 2024
bc359a9
bump version
iakov-aws Sep 24, 2024
499bb38
better wording for cur legacy path
iakov-aws Sep 25, 2024
7ab19dc
create region map
iakov-aws Sep 25, 2024
0ce7dc7
Merge branch 'main' into cur-proxy
iakov-aws Sep 25, 2024
05e088c
better error management
iakov-aws Sep 29, 2024
88ddaea
support more then 254 tags
iakov-aws Sep 29, 2024
80387c2
make the right partitions
iakov-aws Sep 29, 2024
b35ac1f
make linter happier
iakov-aws Sep 29, 2024
0ff02bb
Merge branch 'main' into add-cfn-plugins
iakov-aws Sep 29, 2024
eb322dd
merge
iakov-aws Oct 11, 2024
2a19f1c
minor change to view to trigger update
iakov-aws Oct 11, 2024
688c9a3
avoid using the same bucket
iakov-aws Oct 11, 2024
6604099
Merge branch 'main' into cur-proxy
iakov-aws Oct 20, 2024
5943ab6
allow keeping legacy cur
iakov-aws Oct 20, 2024
17fa7a2
merge
iakov-aws Oct 24, 2024
9be9dfc
merge
iakov-aws Nov 5, 2024
0f362dd
Merge branch 'add-cfn-plugins' into cur-proxy
iakov-aws Nov 5, 2024
32dcdef
make linter happier
iakov-aws Nov 5, 2024
2748973
make tests more robust
iakov-aws Nov 6, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 28 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ See more dashboards on the [workshop page](https://catalog.workshops.aws/awscid/
pip3 install --upgrade cid-cmd
```

#### Dasbhoard Deployment
#### Dashboard Deployment

```bash
cid-cmd deploy
Expand All @@ -71,6 +71,7 @@ Update dashboard and all dependencies (Datasets and Athena View). WARNING: this
```bash
cid-cmd update --force --recursive
```

#### Show Dashboard Status
Show dashboards status

Expand All @@ -80,8 +81,6 @@ cid-cmd status
[<img width="558" alt="status" src="https://github.com/aws-samples/aws-cudos-framework-deployment/assets/82834333/cae2015f-0f81-4593-80b3-c67ec1200fcd">](https://www.youtube.com/watch?v=ivr1MoGaApM)




#### Share QuickSight resources
```bash
cid-cmd share
Expand All @@ -95,18 +94,43 @@ cid-cmd init-qs
```

#### Initialize CUR
One time action to initialize Athena table and Crawler from s3 with CUR data.
One time action to initialize Athena table and Crawler from s3 with CUR data. Currently only CUR1 supported.

```bash
cid-cmd init-cur
```

#### Create CUR proxy
There are 2 CUR formats that `cid-cmd` supports. CUR1 and CUR2 have slightly different fields structure. Each dashboard can be developed using CUR2 or CUR1, but on installation `cid-cmd` will deploy a special Athena View "CUR proxy". This View will play a role of compatibility layer between the CUR version used by Dashboard and available CUR. This View can be extended as new dashboard request CUR fields.

Please note that Cost Allocation Tags and Cost Categories are not present in CUR Proxy by default. You will need to add these fields by modifying CUR Proxy in Athena or using `cid-cmd` tool:

For CUR2 proxy (using CUR1 data as source)
```bash
cid-cmd create-cur-proxy -vv \
--cur-version '2' \
--cur-table-name 'mycur1' \
--athena-workgroup 'primary' \
--fields "resource_tags['user_cost_center'],resource_tags['user_owner']"
```

For CUR1 proxy (using CUR2 data as source)
```bash
cid-cmd create-cur-proxy -vv \
--cur-version '1' \
--cur-table-name 'mycur2' \
--athena-workgroup 'primary' \
--fields "resource_tags_user_cost_center,resource_tags_user_owner"
```


#### Delete Dashboard and all dependencies unused by other
Delete Dashboards and all dependencies unused by other CID-managed dashboards.(including QuickSight datasets, Athena views and tables)
```bash
cid-cmd delete
```


#### Delete Command Options:
```
--dashboard-id TEXT QuickSight dashboard id
Expand Down
Loading
Loading