How to reuse S3 File assets?

[wangzhihao]

We have some s3 file assets like metastore-sync.sh, some jars. The jars can be up to 200MB.

Currently we upload them every time when a job is submit. There are two drawbacks.

1. It wastes time since on the most of time the files are the same.
2. It wastes storage in a long term since the uploaded files are not either cleaned up or overriden.

We currently use CDK assets for the upload. We can not use CDK resources for this end since CDK/Cloudformation only support S3 Bucket level as the resource.

CDK assets allows users to manually specify what assets to not rebuild. See link1 and link2. But not very helpful to us.
CDK assets also can't garbage collect itself. See here.

One promising way is to use Cloud Formation custom resources to define one for s3 object. Here is an example.

[wangzhihao]

We are to garbage collect the resources by #94