Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Manage packages #58

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Manage packages #58

wants to merge 4 commits into from

Conversation

muryanto1
Copy link
Member

Adding 'remove_anaconda_pkgs.py' script for removing anaconda packages that are older than certain time.

$ python ./remove_anaconda_pkgs.py -h
usage: remove_anaconda_pkgs.py [-h] [-c CHANNEL] [-p PACKAGES] [-R REGEX]
                               [-o OLDER_THAN] [-d]

remove your anaconda pkgs that are older than the specified constraint(s)

optional arguments:
  -h, --help            show this help message and exit
  -c CHANNEL, --channel CHANNEL
                        conda channel, ex: cdat/label/unstable (default: None)
  -p PACKAGES, --packages PACKAGES
                        conda packages, ex: 'thermo cdutil' (default: None)
  -R REGEX, --regex REGEX
                        regex to match with substring in the pkg version, ex:
                        '.*2016.*', '.*(2016|2017).*' (default: None)
  -o OLDER_THAN, --older_than OLDER_THAN
                        look for packages older than specified constraints:
                        days=<num>, hours=<num>, minutes=<num> or weeks=<num>
                        (default: None)
  -d, --dryrun          dry run do not actually do anything, just list out the
                        packages to be removed (default: False)


$ python ./remove_anaconda_pkgs.py -c cdat/label/linatest -o days=120 -d
CMD: conda search --override -c cdat/label/linatest
Loading channels: ...working... done
# Name                  Version           Build  Channel
cdutil          8.0.2018.07.30.15.26.ga076047          py36_0  cdat/label/linatest
cdutil          8.0.2018.08.03.08.35.g90c6425          py36_0  cdat/label/linatest
cdutil          8.0.2018.08.03.09.05.g2b08380          py36_0  cdat/label/linatest
cdutil          8.0.2018.08.03.09.36.g5f4fb99          py36_0  cdat/label/linatest
cdutil          8.0.2018.08.03.09.56.g8805ee1          py36_0  cdat/label/linatest
cdutil          8.0.2018.08.03.14.01.g298416a          py36_0  cdat/label/linatest
cdutil          8.0.2018.08.03.16.41.g3da5cf0          py36_0  cdat/label/linatest
cdutil          8.0.2018.08.03.17.03.gfdbaac2          py36_0  cdat/label/linatest
dv3d            8.0.2018.09.18.15.46.g0c84513          py27_0  cdat/label/linatest
dv3d            8.0.2018.09.18.15.53.g0c84513          py36_0  cdat/label/linatest
thermo          8.0.2019.02.15.22.33.g8fdc229            py_0  cdat/label/linatest

Going to remove: cdat/cdutil/8.0.2018.07.30.15.26.ga076047
Going to remove: cdat/cdutil/8.0.2018.08.03.08.35.g90c6425
Going to remove: cdat/cdutil/8.0.2018.08.03.09.05.g2b08380
Going to remove: cdat/cdutil/8.0.2018.08.03.09.36.g5f4fb99
Going to remove: cdat/cdutil/8.0.2018.08.03.09.56.g8805ee1
Going to remove: cdat/cdutil/8.0.2018.08.03.14.01.g298416a
Going to remove: cdat/cdutil/8.0.2018.08.03.16.41.g3da5cf0
Going to remove: cdat/cdutil/8.0.2018.08.03.17.03.gfdbaac2
Going to remove: cdat/dv3d/8.0.2018.09.18.15.46.g0c84513
Going to remove: cdat/dv3d/8.0.2018.09.18.15.53.g0c84513


$ python ./remove_anaconda_pkgs.py -R ".*2018.*" -c cdat/label/linatest -d
CMD: conda search --override -c cdat/label/linatest
Loading channels: ...working... done
# Name                  Version           Build  Channel
cdutil          8.0.2018.07.30.15.26.ga076047          py36_0  cdat/label/linatest
cdutil          8.0.2018.08.03.08.35.g90c6425          py36_0  cdat/label/linatest
cdutil          8.0.2018.08.03.09.05.g2b08380          py36_0  cdat/label/linatest
cdutil          8.0.2018.08.03.09.36.g5f4fb99          py36_0  cdat/label/linatest
cdutil          8.0.2018.08.03.09.56.g8805ee1          py36_0  cdat/label/linatest
cdutil          8.0.2018.08.03.14.01.g298416a          py36_0  cdat/label/linatest
cdutil          8.0.2018.08.03.16.41.g3da5cf0          py36_0  cdat/label/linatest
cdutil          8.0.2018.08.03.17.03.gfdbaac2          py36_0  cdat/label/linatest
dv3d            8.0.2018.09.18.15.46.g0c84513          py27_0  cdat/label/linatest
dv3d            8.0.2018.09.18.15.53.g0c84513          py36_0  cdat/label/linatest
thermo          8.0.2019.02.15.22.33.g8fdc229            py_0  cdat/label/linatest

Going to remove: cdat/cdutil/8.0.2018.07.30.15.26.ga076047
Going to remove: cdat/cdutil/8.0.2018.08.03.08.35.g90c6425
Going to remove: cdat/cdutil/8.0.2018.08.03.09.05.g2b08380
Going to remove: cdat/cdutil/8.0.2018.08.03.09.36.g5f4fb99
Going to remove: cdat/cdutil/8.0.2018.08.03.09.56.g8805ee1
Going to remove: cdat/cdutil/8.0.2018.08.03.14.01.g298416a
Going to remove: cdat/cdutil/8.0.2018.08.03.16.41.g3da5cf0
Going to remove: cdat/cdutil/8.0.2018.08.03.17.03.gfdbaac2
Going to remove: cdat/dv3d/8.0.2018.09.18.15.46.g0c84513
Going to remove: cdat/dv3d/8.0.2018.09.18.15.53.g0c84513

@muryanto1 muryanto1 requested a review from doutriaux1 February 26, 2019 00:23
@doutriaux1
Copy link
Contributor

@muryanto1 thanks for this, few question, what about packages with multiple channels? e.g "main" and "nightly", is there a way for us to know this?

@doutriaux1
Copy link
Contributor

if there's no way it's ok, let's make sure we do not have any nightly/main or nightly/v80 or this sort anywhere. Also we might want our default to ask before removing any file.

Muryanto added 2 commits February 26, 2019 09:11
adding a prompt before really removing
244d8f1
add support for specifying multiple channels, and by default it will …
376dc49 
…prompt user before really removing, unless '--no_prompt' option is specified
@muryanto1
Copy link
Member Author

I added support for specifying multiple channels, and by default it will prompt user before really removing, unless '--no_prompt' option is specified.

@xylar
Copy link
Collaborator

xylar commented Feb 26, 2019

I don't know if you guys are aware of this already, but some package versions have been removed that some E3SM software relies on (specifically cdp 1.4.1). I would recommend being very careful about removing packages from the anaconda channel.
cc @zshaheen @sterlingbaldwin

@zshaheen
Copy link
Contributor

@xylar, yes I've noticed that. My plan is to release a version of e3sm_diags that's:

  • built with noarch, so it'll support Python 3.7, like how you've ask for here.
  • Use >= signs so it'll have future compatibility and work better in the unified env.

@muryanto1
Copy link
Member Author

I don't know if you guys are aware of this already, but some package versions have been removed that some E3SM software relies on (specifically cdp 1.4.1). I would recommend being very careful about removing packages from the anaconda channel.
cc @zshaheen @sterlingbaldwin

I have been testing the script that I added using my cdat/label/linatest channel...so I do not know what happened to those deleted cdp packages.

@xylar
Copy link
Collaborator

xylar commented Feb 26, 2019

I have been testing the script that I added using my cdat/label/linatest channel...so I do not know what happened to those deleted cdp packages.

Okay, no problem. It may be unrelated. I just wanted to be sure you were careful about which packages get deleted because obviously some over-zealous removal has happened recently.

@xylar
Copy link
Collaborator

xylar commented Feb 26, 2019

I opened #59 so it doesn't get confused with this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@doutriaux1 doutriaux1 Awaiting requested review from doutriaux1

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@muryanto1 @doutriaux1 @xylar @zshaheen