Setup - Unable to setup via pip on Windows machine and few other installation questions

[csharma2309]

Topic

FarmVibes.AI setup

Ask away!

@rafaspadilha @renatolfc - Is having a Linux based system such as Ubuntu or WSL is a hard requirement?

I have a windows 11 machine and I have been un-successful

• installing vibe_core package through pip install src/vibe_core command, It usually stuck at the step, unable to build wheel for vibe-core error
• At the same time, I was not able to create a venv using micromamba, it also fails at the same step with similar error.

I use Python on Windows for many projects and I have never been limited by this. Due to some security reasons it is impossible for me to get WSL on my corporate machine. So I have live with this limitation for the time being.

So, there are two scenarios:-

• Installing a VM - Do we need a Linux or WSL to work through this? Because, how would one connect to Jupyter SERVER running on a VM through their local machines if one is unable to install these python modules or notebook related environment on our local machine?
• Via AKS - Even to connect to a AKS backed installation, how would one connect to its Jupyter server via a notebook running on ones local if they are unable to install?
  - In your documentation to use Cloud Shell, I was able to install it via this approach, however problem comes if I lose this cloud shell because of hibernation/termination of shell. Since the installation process install a lot of configuration files to work with client = get_default_vibe_client("remote") like calls, once a shell is lost you basically lose all configuation you got during installation thereby rendering and connecting with installation tricky and with workarounds.

On a side note to clear some doubts:-

How would you connect to let say Jupyter server running on either types(VM or AKS) if one is unsuccessful to install required packages? Can you please help/guide how would the process looks like?

Regards,
Chetan

[renatolfc]

Hello @csharma2309.

Do we need a Linux or WSL to work through this?

No. But building local clusters on Windows hosts is not supported.

I use Python on Windows for many projects and I have never been limited by this.

In the following, I'm using Python installed with winget install python and I'm also assuming a PowerShell session. Please adapt the steps accordingly.

It might be the case that your local copy of pip needs upgrading. I'd recommend creating a new virtualenv (with python -m venv venv), activating it (with venv\Scripts\Activate.ps1) and upgrading pip (with pip install --upgrade pip).

After you've done that, please try installing vibe_core with

pip install -e "git+https://github.com/microsoft/farmvibes-ai.git#egg=vibe_core&subdirectory=src/vibe_core"

I've just followed the above steps in a brand new machine and the farmvibes-ai command was installed successfully. Here's a step-by-step log of the above instructions:

...

---

In general, if you are having issues with the steps we shared, it helps a lot to share logs or screenshots or some sort of description of what exactly you tried and how it fails fails.

From the description of your error, I am assuming your local pip copy is too old to support installs with pyproject.toml files (which we use in our setup process).

[csharma2309]

Thanks @renatolfc for providing detailed steps. I followed these steps and I was able to install farmvibes-ai and it was accessible.

I also upgraded before running these commands. I have tried many solutions through stackoverflow before posting here. None of them worked.

Now other question is now when I am installing a particular env to run a notebook via micromamba, I am getting following errors.

  copying lib\cartopy\tests\mpl\baseline_images\mpl\test_ticks\yticks_cylindrical.png -> build\lib.win-amd64-cpython-38\cartopy\tests\mpl\baseline_images\mpl\test_ticks
  copying lib\cartopy\tests\mpl\baseline_images\mpl\test_ticks\yticks_no_transform.png -> build\lib.win-amd64-cpython-38\cartopy\tests\mpl\baseline_images\mpl\test_ticks
  creating build\lib.win-amd64-cpython-38\cartopy\tests\mpl\baseline_images\mpl\test_web_services
  copying lib\cartopy\tests\mpl\baseline_images\mpl\test_web_services\wms.png -> build\lib.win-amd64-cpython-38\cartopy\tests\mpl\baseline_images\mpl\test_web_services
  copying lib\cartopy\tests\mpl\baseline_images\mpl\test_web_services\wmts.png -> build\lib.win-amd64-cpython-38\cartopy\tests\mpl\baseline_images\mpl\test_web_services
  creating build\lib.win-amd64-cpython-38\cartopy\data
  creating build\lib.win-amd64-cpython-38\cartopy\data\raster
  creating build\lib.win-amd64-cpython-38\cartopy\data\raster\natural_earth
  copying lib\cartopy\data\raster\natural_earth\50-natural-earth-1-downsampled.png -> build\lib.win-amd64-cpython-38\cartopy\data\raster\natural_earth
  copying lib\cartopy\data\raster\natural_earth\images.json -> build\lib.win-amd64-cpython-38\cartopy\data\raster\natural_earth
  creating build\lib.win-amd64-cpython-38\cartopy\data\raster\sample
  copying lib\cartopy\data\raster\sample\Miriam.A2012270.2050.2km.jpg -> build\lib.win-amd64-cpython-38\cartopy\data\raster\sample
  copying lib\cartopy\data\raster\sample\Miriam.A2012270.2050.2km.README.txt -> build\lib.win-amd64-cpython-38\cartopy\data\raster\sample
  creating build\lib.win-amd64-cpython-38\cartopy\data\netcdf
  copying lib\cartopy\data\netcdf\HadISST1_SST_update.nc -> build\lib.win-amd64-cpython-38\cartopy\data\netcdf
  copying lib\cartopy\data\netcdf\HadISST1_SST_update.README.txt -> build\lib.win-amd64-cpython-38\cartopy\data\netcdf
  creating build\lib.win-amd64-cpython-38\cartopy\data\shapefiles
  creating build\lib.win-amd64-cpython-38\cartopy\data\shapefiles\gshhs
  copying lib\cartopy\data\shapefiles\gshhs\README.TXT -> build\lib.win-amd64-cpython-38\cartopy\data\shapefiles\gshhs
  creating build\lib.win-amd64-cpython-38\cartopy\data\shapefiles\gshhs\c
  copying lib\cartopy\data\shapefiles\gshhs\c\GSHHS_c_L1.dbf -> build\lib.win-amd64-cpython-38\cartopy\data\shapefiles\gshhs\c
  copying lib\cartopy\data\shapefiles\gshhs\c\GSHHS_c_L1.shp -> build\lib.win-amd64-cpython-38\cartopy\data\shapefiles\gshhs\c
  copying lib\cartopy\data\shapefiles\gshhs\c\GSHHS_c_L1.shx -> build\lib.win-amd64-cpython-38\cartopy\data\shapefiles\gshhs\c
  creating build\lib.win-amd64-cpython-38\cartopy\data\shapefiles\gshhs\l
  copying lib\cartopy\data\shapefiles\gshhs\l\GSHHS_l_L2.dbf -> build\lib.win-amd64-cpython-38\cartopy\data\shapefiles\gshhs\l
  copying lib\cartopy\data\shapefiles\gshhs\l\GSHHS_l_L2.shp -> build\lib.win-amd64-cpython-38\cartopy\data\shapefiles\gshhs\l
  copying lib\cartopy\data\shapefiles\gshhs\l\GSHHS_l_L2.shx -> build\lib.win-amd64-cpython-38\cartopy\data\shapefiles\gshhs\l
  creating build\lib.win-amd64-cpython-38\cartopy\tests\lakes_shapefile
  copying lib\cartopy\tests\lakes_shapefile\ne_110m_lakes.dbf -> build\lib.win-amd64-cpython-38\cartopy\tests\lakes_shapefile
  copying lib\cartopy\tests\lakes_shapefile\ne_110m_lakes.shp -> build\lib.win-amd64-cpython-38\cartopy\tests\lakes_shapefile
  copying lib\cartopy\tests\lakes_shapefile\ne_110m_lakes.shx -> build\lib.win-amd64-cpython-38\cartopy\tests\lakes_shapefile
  copying lib\cartopy\io\srtm.npz -> build\lib.win-amd64-cpython-38\cartopy\io
  running build_ext
  building 'cartopy.trace' extension
  creating build\temp.win-amd64-cpython-38
  creating build\temp.win-amd64-cpython-38\Release
  creating build\temp.win-amd64-cpython-38\Release\lib
  creating build\temp.win-amd64-cpython-38\Release\lib\cartopy
  "C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.33.31629\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -I. -I./lib/cartopy -IC:\Users\chetan.sharma\AppData\Locj
  trace.cpp
  lib/cartopy/trace.cpp(767): fatal error C1083: Cannot open include file: 'geos_c.h': No such file or directory
  error: command 'C:\\Program Files\\Microsoft Visual Studio\\2022\\Professional\\VC\\Tools\\MSVC\\14.33.31629\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for cartopy
Successfully built vibe-core
Failed to build cartopy
ERROR: Could not build wheels for cartopy, which is required to install pyproject.toml-based projects
critical libmamba pip failed to install packages

[renatolfc]

Hi. That new error is a bit more complicated. What notebook env is that?

[renatolfc]

Hello @csharma2309,

I found the root cause of the issue, and it is due to us specifying a pip dependency when it should have been a conda-forge dependency when on Windows.

I'll submit a patch later, but right now, to fix your environment, save the following patch to a file (for example fix.diff) and apply it with git apply fix.diff.

This should fix your environment while we don't open a PR with the fix.

diff --git a/notebooks/env.yaml b/notebooks/env.yaml
index 6fc0228d..c4bc145e 100644
--- a/notebooks/env.yaml
+++ b/notebooks/env.yaml
@@ -4,6 +4,7 @@ channels:
   - defaults
 dependencies:
   - python==3.8.*
+  - cartopy~=0.21.0
   - geopandas~=0.11.1
   - ipython~=8.5.0
   - ipywidgets~=8.0.2
@@ -19,5 +20,4 @@ dependencies:
   - scikit-image~=0.20.0
   - pip:
       - ../src/vibe_core
-      - cartopy~=0.21.0
       - xarray~=2022.10.0

[chetan2309]

@renatolfc - thanks for the update. I will try this tomorrow and will update this thread! Much appreciate your help

[rafaspadilha]

@chetan2309, any update on this end?