Can't sync pipenv

[petitsurfeur]

Ubuntu 20.04.1 LTS, Docker version 19.03.8 and Python 3.8.5 installed

When I run
╰─$ pipenv sync --dev
Installing dependencies from Pipfile.lock (c916dd)…
Ignoring importlib-metadata: markers 'python_version < "3.8"' don't match your environment
An error occurred while installing psycopg2==2.8.5! Will try again.
Ignoring importlib-metadata: markers 'python_version < "3.8"' don't match your environment
🐍 ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 105/105 — 00:00:33
Installing initially–failed dependencies…
Collecting psycopg2==2.8.5▉▉▉▉▉▉▉▉▉▉▉ 0/1 — 00:00:00
Using cached psycopg2-2.8.5.tar.gz (380 kB)

ERROR: Command errored out with exit status 1:
 command: /home/cybitnap/.local/share/virtualenvs/JARR-u_PnIgzS/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ky16yyj2/psycopg2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-ky16yyj2/psycopg2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-ky16yyj2/psycopg2/pip-egg-info
     cwd: /tmp/pip-install-ky16yyj2/psycopg2/
Complete output (23 lines):
running egg_info
creating /tmp/pip-install-ky16yyj2/psycopg2/pip-egg-info/psycopg2.egg-info
writing /tmp/pip-install-ky16yyj2/psycopg2/pip-egg-info/psycopg2.egg-info/PKG-INFO
writing dependency_links to /tmp/pip-install-ky16yyj2/psycopg2/pip-egg-info/psycopg2.egg-info/dependency_links.txt
writing top-level names to /tmp/pip-install-ky16yyj2/psycopg2/pip-egg-info/psycopg2.egg-info/top_level.txt
writing manifest file '/tmp/pip-install-ky16yyj2/psycopg2/pip-egg-info/psycopg2.egg-info/SOURCES.txt'

Error: pg_config executable not found.

pg_config is required to build psycopg2 from source.  Please add the directory
containing pg_config to the $PATH or specify the full executable path with the
option:

    python setup.py build_ext --pg-config /path/to/pg_config build ...

or with the pg_config option in 'setup.cfg'.

If you prefer to avoid building psycopg2 from source, please install the PyPI
'psycopg2-binary' package instead.

For further information please check the 'doc/src/install.rst' file (also at
<https://www.psycopg.org/docs/install.html>).

----------------------------------------

ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

☤ ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/1 — 00:00:01

Trying to install psycopg2
$ pip3 install psycopg2
Collecting psycopg2
Using cached psycopg2-2.8.6.tar.gz (383 kB)
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-8d1ctuq7/psycopg2/setup.py'"'"'; file='"'"'/tmp/pip-install-8d1ctuq7/psycopg2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-8d1ctuq7/psycopg2/pip-egg-info
cwd: /tmp/pip-install-8d1ctuq7/psycopg2/
Complete output (23 lines):
running egg_info
creating /tmp/pip-install-8d1ctuq7/psycopg2/pip-egg-info/psycopg2.egg-info
writing /tmp/pip-install-8d1ctuq7/psycopg2/pip-egg-info/psycopg2.egg-info/PKG-INFO
writing dependency_links to /tmp/pip-install-8d1ctuq7/psycopg2/pip-egg-info/psycopg2.egg-info/dependency_links.txt
writing top-level names to /tmp/pip-install-8d1ctuq7/psycopg2/pip-egg-info/psycopg2.egg-info/top_level.txt
writing manifest file '/tmp/pip-install-8d1ctuq7/psycopg2/pip-egg-info/psycopg2.egg-info/SOURCES.txt'

Error: pg_config executable not found.

pg_config is required to build psycopg2 from source.  Please add the directory
containing pg_config to the $PATH or specify the full executable path with the
option:

    python setup.py build_ext --pg-config /path/to/pg_config build ...

or with the pg_config option in 'setup.cfg'.

If you prefer to avoid building psycopg2 from source, please install the PyPI
'psycopg2-binary' package instead.

For further information please check the 'doc/src/install.rst' file (also at
<https://www.psycopg.org/docs/install.html>).

----------------------------------------

ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

I don't know what to do

[jaesivsm]

Hello,

Yeah indeed, something might be missing from documentation because the psycopg2 isn't installable as is.
You'll need to install the debian packet for it postgresql-server-dev-all. You'll also need libxml2-dev (which has nothing to do with the current issue but it's also a system dependency).

TL;DR: Run sudo apt install libxml2-dev postgresql-server-dev-all :)

[petitsurfeur]

Thanks, I was able to sync

2nd error :

╭─cybitnap@pp-ubuntu-02 /opt/Git_Repos/JARR
╰─# make start-env COMPOSE_FILE=Dockerfiles/prod.yml
pipenv run docker-compose --project-name jarr --file Dockerfiles/prod.yml up -d
ERROR: build path /opt/Git_Repos/JARR/**Dockerfiles/Dockerfiles/front** either does not exist, is not accessible, or is not a valid URL.
make: *** [Makefile:52: start-env] Error 1

I modified the prod.yml (line 21)
context: ./front

3th error : Service "jarr-server" uses an undefined network "prometheus" >> I commented this in prod.yml

4th error and I don't know what to do :

╭─cybitnap@pp-ubuntu-02 /opt/Git_Repos/JARR ‹master*›
╰─$ make start-env COMPOSE_FILE=Dockerfiles/prod.yml
pipenv run docker-compose --project-name jarr --file Dockerfiles/prod.yml up -d
Building jarr-front
Traceback (most recent call last):
  File "/home/cybitnap/.local/share/virtualenvs/JARR-u_PnIgzS/bin/docker-compose", line 8, in <module>
    sys.exit(main())
  File "/home/cybitnap/.local/share/virtualenvs/JARR-u_PnIgzS/lib/python3.8/site-packages/compose/cli/main.py", line 72, in main
    command()
  File "/home/cybitnap/.local/share/virtualenvs/JARR-u_PnIgzS/lib/python3.8/site-packages/compose/cli/main.py", line 128, in perform_command
    handler(command, command_options)
  File "/home/cybitnap/.local/share/virtualenvs/JARR-u_PnIgzS/lib/python3.8/site-packages/compose/cli/main.py", line 1077, in up
    to_attach = up(False)
  File "/home/cybitnap/.local/share/virtualenvs/JARR-u_PnIgzS/lib/python3.8/site-packages/compose/cli/main.py", line 1058, in up
    return self.project.up(
  File "/home/cybitnap/.local/share/virtualenvs/JARR-u_PnIgzS/lib/python3.8/site-packages/compose/project.py", line 548, in up
    svc.ensure_image_exists(do_build=do_build, silent=silent, cli=cli)
  File "/home/cybitnap/.local/share/virtualenvs/JARR-u_PnIgzS/lib/python3.8/site-packages/compose/service.py", line 367, in ensure_image_exists
    self.build(cli=cli)
  File "/home/cybitnap/.local/share/virtualenvs/JARR-u_PnIgzS/lib/python3.8/site-packages/compose/service.py", line 1086, in build
    build_output = builder.build(
  File "/home/cybitnap/.local/share/virtualenvs/JARR-u_PnIgzS/lib/python3.8/site-packages/docker/api/build.py", line 148, in build
    raise TypeError("You must specify a directory to build in path")
TypeError: You must specify a directory to build in path
make: *** [Makefile:52: start-env] Error 1

[jaesivsm]

Looks like the example prod.yml isn't working out of the box.
I've started a fix, you can check it out there.
Replaying the same command with that file instead should get the environment started, but it will build the whole thing as the file is intended to be an example (I should definitely make that more explicit).

For production I suggest you to use the images server and worker from dockerhub by replacing the build section in the compose file by image: jaesivsm/jarr-worker (or server).
You'll have to build the front though as it needs a special environment variable at build time (PUBLIC_URL and REACT_APP_API_URL which are the full URLs (shema included) at which the UI is supposed to be server).

A new release with a way more complete documentation should be release in a not so far future.

[petitsurfeur]

Hi

Thanks for your answer. I tried again with your prod.yml file

╭─root@pp-ubuntu-02 /opt/Git_Repos/JARR ‹master*›
╰─# make start-env COMPOSE_FILE=Dockerfiles/prod.yml
**pipenv run docker-compose --project-name jarr --file Dockerfiles/prod.yml up -d**
Building jarr-worker
Step 1/5 : FROM jarr-base:latest
ERROR: Service 'jarr-worker' failed to build: pull access denied for jarr-base, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
make: *** [Makefile:52: start-env] Error 1

I tried the containers but I failed
I will wait the new release and documentation and will let you know :)

[jaesivsm]

Yeah I definitely have to update the documentation. You can take a look at the fixing-docker-files branch and try running the same command in the mean time.

git checkout fixing-docker-files
make start-env COMPOSE_FILE=Dockerfiles/prod.yml

It runs for me, so I'm pretty sure it should be alright for you.
If not, you can try building the base image by yourself with make build-base

Thanks for testing and keeping me up to date so far :)

[petitsurfeur]

Hi

Sorry, I'm not an expert :)
I did :

rm JARR -Rf
git clone https://github.com/jaesivsm/JARR.git
cd JARR

╰─# git checkout fixing-docker-files
Branch 'fixing-docker-files' set up to track remote branch 'fixing-docker-files' from 'origin'.
Switched to a new branch 'fixing-docker-files'

╰─# pipenv sync --dev
Installing dependencies from Pipfile.lock (dac0fc)...
🐍 ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 57/57 — 00:00:41
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
All dependencies are now up-to-date!

╰─# make start-env COMPOSE_FILE=Dockerfiles/prod.yml
pipenv run docker-compose --project-name jarr --file Dockerfiles/prod.yml up -d
Building jarr-worker
Step 1/5 : FROM jarr-base:latest
ERROR: Service 'jarr-worker' failed to build : pull access denied for jarr-base, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
make: *** [Makefile:52: start-env] Error 1

I tried

╰─# apt install postgresql
╰─# make create-db COMPOSE_FILE=Dockerfiles/prod.yml
pipenv run docker-compose --project-name jarr --file Dockerfiles/prod.yml exec postgresql su postgres -c
"createuser jarr --no-superuser --createdb --no-createrole"
ERROR: No such service: postgresql
make: *** [Makefile:66: create-db] Error 1

╰─# systemctl status postgresql.service
● postgresql.service - PostgreSQL RDBMS
Loaded: loaded (/lib/systemd/system/postgresql.service; enabled; vendor preset: enabled)
Active: active (exited) since Fri 2021-01-15 18:35:17 CET; 2min 36s ago

[jaesivsm]

Indeed, stuff were missing from that example. Thanks for the report !
I have updated both the example and the docker-compose file. I also renamed it to emphasized that it's really just an example. It should work out of the box though now (except the last part).
Enjoy !

[petitsurfeur]

╰─# cp prod-example.yml prod.yml
╰─# make start-env COMPOSE_FILE=Dockerfiles/prod.yml
make: *** No rule to make target 'start-env'. Stop.

Sorry, I really want to try ant install but I can't help

[jaesivsm]

Haha, don't worry about it. From your first command I can guess you're actually in the ./Dockerfiles folder.

The trick is, make is pretty straightforward and you have to be in the same directory than the Makefile for it to work.
All the available make <whatever> are just script described in that file.

So basically you need to get one directory up.