Recommended virtual environment methods, tools

[williamhobbs]

Are there any plans in the works to update documentation related to virtual environments? Existing documentation favors Anaconda/conda [1] or references Astropy [1, 2], and Astropy recommends conda, virtualenvwrapper, or pipenv.

Conda:
Maybe it's just me, but Anaconda's terms are confusing for commercial users. It seems like Anaconda requires a paid license for commercial use [3], and conda is BSD-3, but I'm not sure how to install conda without Anaconda. Also, see confusing answers on the internet, like here https://stackoverflow.com/questions/74762863/are-conda-miniconda-and-anaconda-free-to-use-and-open-source. For now, I've decided not to use it at all, and I worry that this might be a hurdle for other potential commercial users.

pipenv: (recommended by Atropy)
I don't know much about it, but I've found a number of online discussions where people say it is (was?) deprecated, stale, or simply no longer "favored".

What about venv?:
It looks like venv is officially recommended by Python/PyPA for Python 3.3, and comes with a standard Python installation [4]. I don't know anything about the auther of this article [5], but they seem to make a compelling argument for venv. I've been using venv for a while now (which definitely doesn't make it an automatic good idea...) and it seems to be working ok. Should venv be referenced somewhere in the pvlib documentation?

[1] https://pvlib-python.readthedocs.io/en/latest/user_guide/installation.html#set-up-a-virtual-environment
[2] https://pvlib-python.readthedocs.io/en/latest/contributing.html#the-basics
[3] https://legal.anaconda.com/policies/en/
[4] https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/#installing-virtualenv
[5] https://www.bitecode.dev/p/why-not-tell-people-to-simply-use

[mikofski]

Sounds like a good idea to update docs.

I mostly use mamba-forge. No fuss no mess.

I also use the built in venv Python module with official Python but not for really complicated lib heavy packages on Windows for which reliable binaries are not yet available, like libGEOS

[williamhobbs]

Ah! Another option! 🙂

Would you recommend mamba-forge or venv for someone just getting started with python, pvlib, and virtual environments?

[williamhobbs]

Just to add some more to my confusion, this link you provided

I mostly use mamba-forge...

starts with

We recommend that you start with the Mambaforge distribution.

which now takes me to the top of the miniforge page, as the subsection is hidden under a message that mambaforge is discouraged as of this month (sept 2023):

Full text is:

Mambaforge (Discouraged as of September 2023)
With the release of Miniforge3-22.3.1-0, that incorporated the changes in conda-forge/miniforge#277, the packages and configuration of Mambaforge and Miniforge3 are now identical. The only difference between the two is the name of the installer and, subsequently, the default installation directory.

Given its wide usage, there are no plans to deprecate Mambaforge. If at some point we decide to deprecate Mambaforge, it will be appropriately announced and communicated with sufficient time in advance.

As of September 2023, the new usage of Mambaforge is thus discouraged. Bug reports specific to Mambaforge will be closed as won't fix.

It seems like the latest miniforge should be fine, as it gives a user the ability to use conda or mamba, is BSD-3, and lets you install from conda-forge, but it still can all feel a little overwhelming...

[mikofski]

Mambaforge, zero doubt. What I love:

• installed locally, usually in your home folder
• uses mamba instead of conda, but subcommands are identical
• Easy installation of difficult libraries so cross platform deployment is a snap
• Defaults to community supported conda-forge channel, but b/c functions same as conda any channel including “default” could be used

[adriesse]

I am not as aware of these issues as I should be, so thanks for the discussion. I did find the following in the license agreement:

The Free Offerings are made available to You, and Community Users, at the Free Subscription level strictly for internal: (i) Personal Use, (ii) Educational Use, (iii) Open-Source Use, and/or (iv) Small Business Use.

Looks like I should be ok. Last new install I did was mini-conda with the mamba solver, which worked ok. I was thinking that we could create/publish a yaml file that covers 99% of anticipated pvlib user needs as an alternative to the full anaconda.

[williamhobbs]

The way I read Anaconda’s definitions, I’m not sure if, e.g., national labs can use the free version, since they are not small businesses and “education use” sounds like it is only for instructional settings.

Many pvlib users probably do fall in the small business category (<200 employees).

venv also has the advantage of being included with a standard python install, so it’s one less installer that might require IT support for organization with strict admin rules. It looks to me like mamba requires a separate installation.

[mikofski]

Here’s the problem with venv: even though it’s included in Python from https://python.org that version of Python still requires admin access to install, and all venv gets you is the ability to make virtual environments which you already get if using any of the conda variants.

I think there may be some confusion between the licensing terms from https://anaconda.com for use of the “default” channel versus the use of conda the virtual environment and package manager which is free an open source, see: https://docs.conda.io/en/latest/license.html

Therefore what I believe your main concern is about getting packages without violating Anaconda’s terms of use, not the use of conda itself. Therefore this discussion might really be more about which package index or conda channel to use.

The solution to this is just use the community supported conda-forge channel: https://conda-forge.org/ which is the default if you install python with the simple mambaforge installer, no admin rights needed

When using venv you typically use the python package index (aka PyPI) which has the added challenge of not having binary wheels for all packages and platforms which is a common blocker for many users even experienced Pythonistas. This is the main hurdle that conda solves b/c all packages from conda-forge automatically take care of most required binaries.

[williamhobbs]

Thanks, @mikofski, that is very helpful. I think my concerns were 1) how to get conda without Anaconda, and 2) how to use conda to get packages without violating Anaconda’s terms. It seems like you’ve answered that with 1) mamba and 2) conda-forge.

But I don’t think I’ve ever heard of mamba before this discussion, and only sort-of understood conda-forge. So another vote from me (I think I’ve voted twice now, don’t tell…) for updated documentation. Copy-pasting your comment might go a long way 😀.

On the topic of needing admin rights to install the python.org version of Python, we already have it in an “IT approved” software list, so can install it on company workstations with no problems. Adding something new and less common (like a mamba installer) could be a challenge. I don’t know if other organizations could have the same challenge, though.

maybe there are two “recommended” options: if you can get python.org python, and only that, here’s how to use venv…. Or, if you can get mamba, here’s how to use it with conda-forge and avoid Anaconda TOS challenges… (and maybe a third, if you can use Anaconda, like @adriesse, …).

[kandersolar]

FWIW although the anaconda/conda-forge indexes used to be the only reasonable option for getting a lot of fundamental data/numerics packages (esp on windows), it's a lot easier than it used to be to get everything from PyPI nowadays. That said I still like conda/mamba for creating new environments, but I almost exclusively use pip to install packages into them.

But it seems like pvlib is one of many projects with an imperfect "getting started with python" page, and it's a shame for all of us to reinvent that wheel and/or reference each other. I wish some collaborative like scientific-python.org would make something like that which all of us could reference.

[williamhobbs]

I still like conda/mamba for creating new environments, but I almost exclusively use pip to install packages into them.

Good to know that PyPI with pip for installation works well for you. It's been working pretty well for me.

[jules-ch]

When using OS with package manager namely OSX and Linux. You can easily install almost everything with PyPI these days. It's also the safest option I'd say.
Conda has made the lives of many datascientist or Windows users easier especially with scientific packages publishing to conda-forge.