Skip to content
This repository has been archived by the owner on Jun 18, 2019. It is now read-only.

Error executing action install on resource 'python_package[setuptools]' #133

Open
f9n opened this issue Oct 5, 2018 · 28 comments
Open

Error executing action install on resource 'python_package[setuptools]' #133

f9n opened this issue Oct 5, 2018 · 28 comments

Comments

@f9n
Copy link

f9n commented Oct 5, 2018

$ chef generate cookbook install-python
$ cd install-python
$ vim recipes/default.rb
python_runtime '3'
$ vim .kitchen.yml
...
platforms:
  - name: ubuntu-16.04
...
$ vim metadata.rb   # Add poise python
...
depends 'poise-python'
...
$ kitchen converge ubuntu
.....
Installing chef 
       installing with dpkg...
       Selecting previously unselected package chef.
(Reading database ... 65145 files and directories currently installed.)
       Preparing to unpack .../cache/chef_14.5.33-1_amd64.deb ...
       Unpacking chef (14.5.33-1) ...
       Setting up chef (14.5.33-1) ...
       Thank you for installing Chef!
       Transferring files to <default-ubuntu-1604>
       Starting Chef Client, version 14.5.33
       Creating a new client identity for default-ubuntu-1604 using the validator key.
       resolving cookbooks for run list: ["install-python::default"]
       Synchronizing Cookbooks:
         - poise-python (1.7.0)
         - install-python (0.1.0)
         - poise-languages (2.1.2)
         - poise (2.8.2)
         - poise-archive (1.5.0)
       Installing Cookbook Gems:
       Compiling Cookbooks...
       Converging 1 resources
       Recipe: install-python::default
         * python_runtime[3] action install
           * poise_languages_system[python3.5] action install
             - install version 3.5.2-2ubuntu0~16.04.4 of package python3.5-dev
           * apt_package[python3.5, python3.5-dev] action nothing (skipped due to action :nothing)
         
         * python_runtime_pip[3] action install
           - Bootstrapping pip latest from https://bootstrap.pypa.io/get-pip.py
         * python_package[setuptools] action install
           
           ================================================================================
           Error executing action `install` on resource 'python_package[setuptools]'
           ================================================================================
           
           Mixlib::ShellOut::ShellCommandFailed
           ------------------------------------
           Expected process to exit with [0], but received '1'
           ---- Begin output of ["/usr/bin/python3.5", "-", "setuptools"] ----
           STDOUT: 
           STDERR: The directory '/home/vagrant/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
           Traceback (most recent call last):
             File "<stdin>", line 42, in <module>
           AttributeError: type object 'InstallRequirement' has no attribute 'from_line'
           ---- End output of ["/usr/bin/python3.5", "-", "setuptools"] ----
           Ran ["/usr/bin/python3.5", "-", "setuptools"] returned 1
           
           Cookbook Trace:
           ---------------
           /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:207:in `tap'
           /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:207:in `language_command_shell_out!'
           /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:222:in `block in language_command_mixin'
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:329:in `pip_command'
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:353:in `pip_outdated'
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:221:in `check_package_versions'
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:200:in `load_current_resource'
           /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
           /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:50:in `action_install'
           
           Resource Declaration:
           ---------------------
           # In /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb
           
           138:         python_package 'setuptools' do
           139:           parent_python new_resource
           140:           version setuptools_version if setuptools_version.is_a?(String)
           141:         end
           142:       end
           
           Compiled Resource:
           ------------------
           # Declared in /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:138:in `install_setuptools'
           
           python_package("setuptools") do
             package_name "setuptools"
             action [:install]
             default_guard_interpreter :default
             declared_type :python_package
             cookbook_name "install-python"
             parent_python python_runtime[3]
             timeout 900
           end
           
           System Info:
           ------------
           chef_version=14.5.33
           platform=ubuntu
           platform_version=16.04
           ruby=ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]
           program_name=/opt/chef/bin/chef-client
           executable=/opt/chef/bin/chef-client
           
         
         ================================================================================
         Error executing action `install` on resource 'python_runtime[3]'
         ================================================================================
         
         Mixlib::ShellOut::ShellCommandFailed
         ------------------------------------
         python_package[setuptools] (/tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb line 138) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
         ---- Begin output of ["/usr/bin/python3.5", "-", "setuptools"] ----
         STDOUT: 
         STDERR: The directory '/home/vagrant/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
         Traceback (most recent call last):
           File "<stdin>", line 42, in <module>
         AttributeError: type object 'InstallRequirement' has no attribute 'from_line'
         ---- End output of ["/usr/bin/python3.5", "-", "setuptools"] ----
         Ran ["/usr/bin/python3.5", "-", "setuptools"] returned 1
         
         Cookbook Trace:
         ---------------
         /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:207:in `tap'
         /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:207:in `language_command_shell_out!'
         /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:222:in `block in language_command_mixin'
         /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:329:in `pip_command'
         /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:353:in `pip_outdated'
         /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:221:in `check_package_versions'
         /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:200:in `load_current_resource'
         /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
         /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:50:in `action_install'
         
         Resource Declaration:
         ---------------------
         # In /tmp/kitchen/cache/cookbooks/install-python/recipes/default.rb
         
           7: python_runtime '3'
         
         Compiled Resource:
         ------------------
         # Declared in /tmp/kitchen/cache/cookbooks/install-python/recipes/default.rb:7:in `from_file'
         
         python_runtime("3") do
           action [:install]
           updated true
           updated_by_last_action true
           default_guard_interpreter :default
           subresources [python_runtime_pip[3], python_package[setuptools], python_package[wheel]]
           declared_type :python_runtime
           cookbook_name "install-python"
           recipe_name "default"
           get_pip_url "https://bootstrap.pypa.io/get-pip.py"
           pip_version true
           setuptools_version true
           version "3"
           virtualenv_version true
           wheel_version true
         end
         
         System Info:
         ------------
         chef_version=14.5.33
         platform=ubuntu
         platform_version=16.04
         ruby=ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]
         program_name=/opt/chef/bin/chef-client
         executable=/opt/chef/bin/chef-client
         
       
       Running handlers:
       [2018-10-05T12:52:36+00:00] ERROR: Running exception handlers
       [2018-10-05T12:52:36+00:00] ERROR: Running exception handlers
       Running handlers complete
       [2018-10-05T12:52:36+00:00] ERROR: Exception handlers complete
       [2018-10-05T12:52:36+00:00] ERROR: Exception handlers complete
       Chef Client failed. 3 resources updated in 22 seconds
       [2018-10-05T12:52:36+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
       [2018-10-05T12:52:36+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
       [2018-10-05T12:52:36+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
       [2018-10-05T12:52:36+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
       [2018-10-05T12:52:36+00:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: python_runtime[3] (install-python::default line 7) had an error: Mixlib::ShellOut::ShellCommandFailed: python_package[setuptools] (/tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb line 138) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
       ---- Begin output of ["/usr/bin/python3.5", "-", "setuptools"] ----
       STDOUT: 
       STDERR: The directory '/home/vagrant/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
       Traceback (most recent call last):
         File "<stdin>", line 42, in <module>
       AttributeError: type object 'InstallRequirement' has no attribute 'from_line'
       ---- End output of ["/usr/bin/python3.5", "-", "setuptools"] ----
       Ran ["/usr/bin/python3.5", "-", "setuptools"] returned 1
       [2018-10-05T12:52:36+00:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: python_runtime[3] (install-python::default line 7) had an error: Mixlib::ShellOut::ShellCommandFailed: python_package[setuptools] (/tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb line 138) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
       ---- Begin output of ["/usr/bin/python3.5", "-", "setuptools"] ----
       STDOUT: 
       STDERR: The directory '/home/vagrant/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
       Traceback (most recent call last):
         File "<stdin>", line 42, in <module>
       AttributeError: type object 'InstallRequirement' has no attribute 'from_line'
       ---- End output of ["/usr/bin/python3.5", "-", "setuptools"] ----
       Ran ["/usr/bin/python3.5", "-", "setuptools"] returned 1
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>>     Converge failed on instance <default-ubuntu-1604>.  Please see .kitchen/logs/default-ubuntu-1604.log for more details
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

I don't understand, it worked in the past.

@devsibwarra
Copy link

https://pypi.org/project/pip/#history shows 18.1 was released a couple hours ago. Rolling back to 18.0 via pip install pip==18.0 resolved this error for me

@knightorc
Copy link

Interestingly enough. Does not impact CentOS 6.x but does impact CentOS 7.x

@fozboz
Copy link

fozboz commented Oct 5, 2018

Doing

python_runtime '3' do
  pip_version '18.0'
end

does not work because of pypa/pip#5857 which is probably the cause of original issue, given that they occurred at the same time.

@nxf5025
Copy link

nxf5025 commented Oct 5, 2018

Any workaround we can implement into the actual cookbook?

@coderanger
Copy link
Member

You can try setting the get_pip_url property to an older version of the script, either one you host yourself or one of the legacy scripts like https://bootstrap.pypa.io/3.3/get-pip.py.

@jeffbyrnes
Copy link

This works for me:

python_runtime '3' do
  get_pip_url 'https://github.com/pypa/get-pip/raw/f88ab195ecdf2f0001ed21443e247fb32265cabb/get-pip.py'
  pip_version '18.0'
end

@coderanger
Copy link
Member

Hopefully this will be fixed upstream shortly. I'll monitor the upstream ticket and see how it goes.

@aungger
Copy link

aungger commented Oct 5, 2018

I am getting the same thing for python_runtime '2', any tips for this? plz and ty!

@coderanger
Copy link
Member

@aungger Same fix, the current default get-pip.py script is broken.

@eriepasquare
Copy link

if your dependency on poise-python is through a wrapped cookbook and you can't change the pip_version or get_pip_url parameters of the resource... we had success implementing the following workaround in our wrapper cookbook:

edit_resource(:python_runtime, '2') do
  pip_version '18.0'
  get_pip_url 'https://github.com/pypa/get-pip/raw/f88ab195ecdf2f0001ed21443e247fb32265cabb/get-pip.py'
end

@nkovacne
Copy link

nkovacne commented Oct 8, 2018

If you're using a virtualenv resource, you also needed to set the pip_version and get_pip_url in the python_virtualenv resource:

python_virtualenv '/opt/venv' do
  pip_version '18.0'
  get_pip_url 'https://github.com/pypa/get-pip/raw/f88ab195ecdf2f0001ed21443e247fb32265cabb/get-pip.py'
  action :create
end

@hartmantis
Copy link
Contributor

Update: They merged a fix to get-pip but it seems to only resolve the part where get_pip_url had to be specified. So

python_virtualenv '/opt/venv' do
  pip_version '18.0'
end

works now, but blows up without the pip_version when it tries to install setuptools.

@lopezm1
Copy link

lopezm1 commented Oct 8, 2018

confirmed, overriding the attribute will force poise-python to use pip 18.0 and our cookbooks now work as expected

"poise-python": {
      "options": {
        "pip_version": "18.0"
      }	      
}

hartmantis added a commit to hartmantis/poise-python that referenced this issue Oct 8, 2018
It [moved](pypa/pip@a5a07fe) the from_line method in a backwards-incompatible way. This should resolve the remainder of poise#133.

My Python is pretty atrophied, but I tested this against the following recipe and it converged successfully.

```ruby
apt_update 'default'

python_runtime '2'

python_virtualenv '/tmp/piplatest' do
  python '2'
end

python_virtualenv '/tmp/pip18' do
  python '2'
  pip_version '18.0'
end

python_virtualenv '/tmp/pip10' do
  python '2'
  pip_version '10.0.1'
end

python_virtualenv '/tmp/pip9' do
  python '2'
  pip_version '9.0.3'
end
```
@nikbhadane
Copy link

This issue still appears in wrapper cookbook.
Can someone please help?

@jmd9019
Copy link

jmd9019 commented Oct 9, 2018

python_runtime '3' failing on ubuntu 18
`
* python_runtime[3] action install

     Running handlers:
   [2018-10-09T08:45:32+00:00] ERROR: Running exception handlers
   [2018-10-09T08:45:32+00:00] ERROR: Running exception handlers
     Running handlers complete
   [2018-10-09T08:45:32+00:00] ERROR: Exception handlers complete
   [2018-10-09T08:45:32+00:00] ERROR: Exception handlers complete
     Chef Client failed. 2 resources updated in 10 seconds
   [2018-10-09T08:45:32+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
   [2018-10-09T08:45:32+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
   [2018-10-09T08:45:32+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
   [2018-10-09T08:45:32+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
   [2018-10-09T08:45:32+00:00] FATAL: PoiseLanguages::Error: Unable to find a candidate package for version "3". Please set package_name provider option for python_runtime[3].
   [2018-10-09T08:45:32+00:00] FATAL: PoiseLanguages::Error: Unable to find a candidate package for version "3". Please set package_name provider option for python_runtime[3].

`

@mike10010100
Copy link

@lopezm1 Good find! This was the quickest and easiest way to get around this error for me.

@lopezm1
Copy link

lopezm1 commented Oct 9, 2018

This issue still appears in wrapper cookbook.
Can someone please help?

I was having this issue if a previous chef run was cached. Have you tried clearing the cache or running the cookbook on a fresh server?

@nikbhadane
Copy link

I was having this issue if a previous chef run was cached. Have you tried clearing the cache or running the cookbook on a fresh server?

Right, By overiding the pip_version in attribute file of wrapper cookbook resolving the issue now.
like

override['poise-python']['options']['pip_version'] = '9.0.3'

thanks for the help.

lukeseawalker added a commit to lukeseawalker/aws-parallelcluster-cookbook that referenced this issue Oct 11, 2018
The setup_python recipe doesn't work with the latest version of pip
18.1 https://pypi.org/project/pip/18.1/#history
Issue is open at poise/poise-python#133
The workaround consists in overriding pip version to 18.0

Signed-off-by: Luca Carrogu <carrogu@amazon.com>
lukeseawalker added a commit to lukeseawalker/aws-parallelcluster-cookbook that referenced this issue Oct 12, 2018
The setup_python recipe doesn't work with the latest version of pip
18.1 https://pypi.org/project/pip/18.1/#history
Issue is open at poise/poise-python#133
The workaround consists in overriding pip version to 18.0

Signed-off-by: Luca Carrogu <carrogu@amazon.com>
lukeseawalker added a commit to aws/aws-parallelcluster-cookbook that referenced this issue Oct 12, 2018
The setup_python recipe doesn't work with the latest version of pip
18.1 https://pypi.org/project/pip/18.1/#history
Issue is open at poise/poise-python#133
The workaround consists in overriding pip version to 18.0

Signed-off-by: Luca Carrogu <carrogu@amazon.com>
@jharbott
Copy link

Just a note to some folks for whom the workarounds above might show no effect: If pip==18.1 is already installed (and your resource is using the provider: system option), the broken script is executed with that version and fails before pip can be downgraded. So in that case you may need to run pip install pip=18.0 manually once as additional workaround.

Let's hope the patch in #134 gets merged soon. A long-term solution that does not depend on any pip internals would be even nicer.

openstack-gerrit pushed a commit to openstack-archive/openstack-chef that referenced this issue Oct 15, 2018
There is a bug in the poise-python cookbook because it depends on
internal features of pip and these internals have changed in pip 18.1,
see [0]. We have a workaround in the openstack-common cookbook proposed,
but that doesn't work if the server has pip 18.1 pre-installed, as seems
to be the case for the infra-provided CI image. So as a further
workaround we need to downgrade the pre-installed pip version for the
time being.

[0] poise/poise-python#133

Depends-On: https://review.openstack.org/610427
Change-Id: I52514691d3f271579628b62539222f7307957ba0
openstack-gerrit pushed a commit to openstack-archive/openstack-chef that referenced this issue Oct 19, 2018
There is a bug in the poise-python cookbook because it depends on
internal features of pip and these internals have changed in pip 18.1,
see [0]. We have a workaround in the openstack-common cookbook proposed,
but that doesn't work if the server has pip 18.1 pre-installed, as seems
to be the case for the infra-provided CI image. So as a further
workaround we need to downgrade the pre-installed pip version for the
time being.

[0] poise/poise-python#133

Depends-On: https://review.openstack.org/610427
Change-Id: I52514691d3f271579628b62539222f7307957ba0
(cherry picked from commit 973119b)
openstack-gerrit pushed a commit to openstack-archive/openstack-chef that referenced this issue Oct 19, 2018
There is a bug in the poise-python cookbook because it depends on
internal features of pip and these internals have changed in pip 18.1,
see [0]. We have a workaround in the openstack-common cookbook proposed,
but that doesn't work if the server has pip 18.1 pre-installed, as seems
to be the case for the infra-provided CI image. So as a further
workaround we need to downgrade the pre-installed pip version for the
time being.

[0] poise/poise-python#133

Depends-On: https://review.openstack.org/610427
Change-Id: I52514691d3f271579628b62539222f7307957ba0
(cherry picked from commit 973119b)
@adamfeuer
Copy link

adamfeuer commented Oct 31, 2018

As a workaround, since we need this to work to deploy our software using poise-python, I forked into our own copy of this repo, used Rake to build a version from the tip of master that has the fix for using pip 18.1, and saved it to a branch (just a convenient place to put the release). I put this line in my Berksfile:

cookbook "poise-python", "1.7.1", git:'https://github.com/twistle/poise-python.git', branch: 'fix/pip-18-1-problems'

Then when 1.7.1 is really released, I will switch back to using the version from the Chef Supermarket.

Posting this here in case others might find it helpful.

You're welcome to use this or fork it and use it from your own repo.

@sunilkumarmohanty
Copy link

sunilkumarmohanty commented Nov 6, 2018

when is the 1.7.1 version getting released?

@kpmueller
Copy link

Why is 1.7.1 not out yet? :(

@coderanger
Copy link
Member

I explained this on another issue, but my CI system is pretty much busted so I have no safe way to vet a release. No one seems to be willing to pay for the time it would take to fix and I’m no longer doing pro-bono Chef work when I can avoid it.

@adamfeuer
Copy link

@coderanger How could we collaborate to help...? At Twistle we're using this code as part of our chef build for a production project, so maybe there's a way to help. It seems like others are in similar positions. Would you be willing to point me at the other issue where you explained the problem?

@adamfeuer
Copy link

@coderanger P.S. I get not wanting to do volunteer work using chef. I feel the same way! 🙂

@coderanger
Copy link
Member

I’m on my phone so easier to just summarize here again: the first issue is that since my last release, Travis CI has cut the maximum time for builds from two hours to one, which means my tests get killed. But I’m not inclined to fix that because the second issue is my (very generous) infrastructure sponsor will sadly be terminating their open source sponsorship program so the infra that my tests actually run on will go away in a few more weeks. So to get my CI back into a healthy state would require a pretty major rebuild, probably switching providers as well as reworking things to not use the dedicated infra (which will make the tests even slower).

@ramereth
Copy link

@coderanger I might be able to help with this via the OSUOSL. PM me on slack if you're interested.

@adamfeuer
Copy link

@ramereth Cool!

@coderanger I might be able to help with moving things over to a new provider if you need more help.

Maybe there are others that could pitch in too?

wosc added a commit to wosc/pharos-deployment that referenced this issue Dec 26, 2018
Also note, until poise/poise-python@861bf06 is
released, you once have to manually `apt install python3.6-distutils python3.6-venv`
shapiy added a commit to shareaholic/poise-python that referenced this issue Jan 15, 2019
This branch and fork to be removed when poise-python is released.

poise#133 (comment)
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests