SAM Does not build the application, returns "PythonPipBuilder:ResolveDependencies failed

[Hanna-Hinn-imVitro]

I have a project where I upload a domain using SAM Build.
I have packages that are inside helpers that uses setuptools.
I am using a virtual environment using pyenv , python version 3.8.17 // sam version : 1.82.0
Note:** If I run the requirements.txt theirs no issue, even the pip install that is generated by the sam build it works fine

For each lambda I have a requirements file for example:
-e ./helpers/adapters
boto3==1.14.47
dataclasses-json==0.5.2
requests==2.23.0
python_dynamodb_lock==0.9.1
pycryptodome==3.16.0
numpy==1.20.1

If I keep the adapters I get error :

  error: subprocess-exited-with-error                                                                                                                                                                
                                                                                                                                                                                                     
  × python setup.py egg_info did not run successfully.                                                                                                                                               
  │ exit code: 1                                                                                                                                                                                     
  ╰─> [1 lines of output]                                                                                                                                                                            
      ERROR: Can not execute `setup.py` since setuptools is not available in the build environment.                                                                                                  
      [end of output]                                                                                                                                                                                
                                                                                                                                                                                                     
  note: This error originates from a subprocess, and is likely not a problem with pip.                                                                                                               
error: metadata-generation-failed   

If I delete the adapter library I get :

2024-04-30 17:43:19,683 | PythonPipBuilder:ResolveDependencies failed
Traceback (most recent call last):
  File "/home/hanna/.pyenv/versions/3.8.17/envs/testing/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 51, in execute
    package_builder.build_dependencies(
  File "/home/hanna/.pyenv/versions/3.8.17/envs/testing/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 156, in build_dependencies
    self._dependency_builder.build_site_packages(requirements_path, artifacts_dir_path, scratch_dir_path)
  File "/home/hanna/.pyenv/versions/3.8.17/envs/testing/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 256, in build_site_packages
    wheels, packages_without_wheels = self._download_dependencies(scratch_directory, requirements_filepath)
  File "/home/hanna/.pyenv/versions/3.8.17/envs/testing/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 280, in _download_dependencies
    deps = self._download_all_dependencies(requirements_filename, directory)
  File "/home/hanna/.pyenv/versions/3.8.17/envs/testing/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 362, in _download_all_dependencies
    self._pip.download_all_dependencies(requirements_filename, directory)
  File "/home/hanna/.pyenv/versions/3.8.17/envs/testing/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 710, in download_all_dependencies
    raise NoSuchPackageError(str(package_name))
aws_lambda_builders.workflows.python_pip.packager.NoSuchPackageError: Could not satisfy the requirement: dataclasses-json==0.5.2

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/hanna/.pyenv/versions/3.8.17/envs/testing/lib/python3.8/site-packages/aws_lambda_builders/workflow.py", line 369, in run
    action.execute()
  File "/home/hanna/.pyenv/versions/3.8.17/envs/testing/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 57, in execute
    raise ActionFailedError(str(ex))
aws_lambda_builders.actions.ActionFailedError: Could not satisfy the requirement: dataclasses-json==0.5.2

Can anyone please help?

[mildaniel]

Hi @Hanna-Hinn-imVitro, can you provide a minimal sample application for us to reproduce the issue?

[Hanna-Hinn]

Hey, here's the minimal sample application :
Here's the project structure

|── helpers
|  ├── adapters
|  |  ├── adapters
|  |  ├── adapters.egg-info
|  |  ├── setup.py
|  |  |
|── domain
|  ├── bin
|  ├── tests
|  ├── src
|  |  ├── functions
|  |   ├── function1
|  |   | ├── __init__.py
|  |   | ├── handler.py
|  |   | ├── requirements.txt
|  |   ├── __init__.py
|  |  ├── __init__.py
|  |
|  ├── tests
|  ├── template.yaml
|  ├── ... other files

example on the requirements:

-e ../helpers/adapters
dataclasses-json==0.5.2
pycryptodome==3.16.0

Here's the overall requirements.txt for the whole project:

dataclasses-json==0.5.2
pytest==6.1.2
-e helpers/adapters
numpy==1.20.1
boto3==1.16.25
moto==1.3.16
dynamodb-encryption-sdk==1.2.0
pytest-mock==3.5.1
requests-mock==1.8.0
python_dynamodb_lock==0.9.1
pandas==1.3.4
opencv-python-headless==4.5.5.62
pillow==9.0.1
pycryptodome==3.16.0
aiohttp==3.8.1
aiohttp-retry==2.5.2
requests==2.25.1
aws-lambda-powertools[all]==2.8.0
pytest-cov==4.0.0
pydantic==1.10.13
asynctest==0.13.0
pytest-asyncio==0.20

The project is a bit old. I have a theory regarding some dependency version between 2 libraries but I can not figure it out :(. tbh I do not know how exactly can I help you reproduce the issue.

[mildaniel]

Hi @Hanna-Hinn, I did some more digging and this is what I found.

1. For the local dependencies, make sure that the library is being passed into the CodeURI. SAM CLI will move the CodeURI into a temporary scratch directory to build, and if the local dependency is outside of that, relative paths outside of the CodeURI won't be found. You can see an example of local dependencies being used in the Lambda Builders integration tests.
2. For the rest of the dependencies, there was a bug fixed in version 1.95.0 of SAM CLI that should address the issue you're having, try updating to the latest version and seeing if that helps. However, there does seem to be an issue with one of the dependencies of dataclasses-json, depending on the version of pip you are using. stringcase==1.2.0 uses a deprecated pip feature which prevents Lambda Builders from building a wheel. There isn't much we can do for it since it doesn't contain any wheels we can use and we can't change the pip behaviour.

Let us know if this helps.

[Hanna-Hinn]

Okay thank you so much 🙏

[Hanna-Hinn]

Hey, So I migrated the project to python3.11 and upgraded some packages versions and the issue was gone, but I do not know how or why it was gone.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.