Docker Troubleshooting
Here are some common problems we've encountered while setting up the dev environment along with potential solutions. We've ordered this list roughly based on when you might observe the problem in the setup process.
You can always ask questions in Slack #interns. We are here to help!
| City ID | Database Name | Database User |
|---|---|---|
| seattle-wa | sidewalk-seattle | sidewalk_seattle |
| columbus-oh | sidewalk-columbus | sidewalk_columbus |
| cdmx | sidewalk-cdmx | sidewalk_cdmx |
| spgg | sidewalk-spgg | sidewalk_spgg |
| pittsburgh-pa | sidewalk-pittsburgh | sidewalk_pittsburgh |
| newberg-or | sidewalk-newberg | sidewalk_newberg |
| washington-dc | sidewalk | sidewalk |
| chicago-il | sidewalk-chicago | sidewalk_chicago |
| amsterdam | sidewalk-amseterdam | sidewalk_amsterdam |
| la-piedad | sidewalk-la-piedad | sidewalk_la_piedad |
| oradell-nj | sidewalk-oradell | sidewalk_oradell |
| validation-study | sidewalk_validation | sidewalk_validation |
| zurich | sidewalk-zurich | sidewalk_zurich |
| taipei | sidewalk-taipei | sidewalk_taipei |
| new-taipei-tw | sidewalk-new-taipei | sidewalk_new_taipei |
| keelung-tw | sidewalk-keelung | sidewalk_keelung |
| auckland | sidewalk-auckland | sidewalk_auckland |
| cuenca | sidewalk-cuenca | sidewalk_cuenca |
| crowdstudy | sidewalk-crowdstudy | sidewalk_crowdstudy |
| burnaby | sidewalk-burnaby | sidewalk_burnaby |
| teaneck-nj | sidewalk-teaneck | sidewalk_teaneck |
| walla-walla-wa | sidewalk-walla-walla | sidewalk_walla_walla |
| st-louis-mo | NA | sidewalk_st_louis |
If you see an error like:
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 9; 2615 2200 SCHEMA public postgres
pg_restore: error: could not execute query: ERROR: schema "public" already exists
Command was: CREATE SCHEMA public;
This is actually the one error that you can simply ignore! :) It doesn't have any effect.
If you see an error like:
Execution exception[[NoSuchElementException: None.get]]
This is most likely because the data from the database is missing and you'd need to import the sql dump. The schema import that's a part of init script only sets the schema and does not import the data.
If you see an error like:
sh: 1: grunt: not found
This is a known problem (#1517). You can fix it by running npm install grunt from inside the Docker shell.
If you run make import-dump db=sidewalk-seattle (or similar) and you see errors like this:
could not execut query: ERROR role "sidewalk_seattle" does not exist
could not execut query: ERROR role "saugstad" does not exist
or like this:
pg_restore: [archiver (db)] Error while PROCESSING TOC:
pg_restore: [archiver (db)] Error from TOC entry 9; 2615 189957 SCHEMA sidewalk sidewalk_spgg
pg_restore: [archiver (db)] could not execute query: ERROR: role "sidewalk_spgg" does not exist
then run the following line of code for each role that does not exist:
docker exec -it projectsidewalk-db psql -c "CREATE ROLE sidewalk_seattle SUPERUSER LOGIN ENCRYPTED PASSWORD 'sidewalk';" -U postgres -d sidewalk-seattle
For example, to add the role saugstad, you would type:
docker exec -it projectsidewalk-db psql -c "CREATE ROLE saugstad SUPERUSER LOGIN ENCRYPTED PASSWORD 'sidewalk';" -U postgres -d sidewalk
If you try to load the webpage and it says Database 'default' is in an inconsistent state and the error specifically mentions the following line:
INSERT INTO mission_type (mission_type) VALUES ('labelmapValidation');
then you should edit the conf/evolutions/default/68.sql file, add the following line after the !Ups line:
SELECT setval('mission_type_mission_type_id_seq', (SELECT MAX(mission_type_id) from mission_type));
then edit the conf/evolutions/default/72.sql file, adding the following line after the !Ups line:
SELECT setval('tag_tag_id_seq', (SELECT MAX(tag_id) from "tag"));
and click "Mark it resolved". The webpage should then load normally. Once it loads, remove the lines you just added and refresh. The website should still work normally after you remove the line.
If it is still showing a similar error, I'd try to run the command above that corresponds to the error message, and I'd run that command manually on the database. You should be able to run it like this (outside the docker shell):
docker exec -it projectsidewalk-db psql -c "<command>" -U postgres -d sidewalk
After running that command, try clicking "mark as resolved". If it still doesn't work, I'd run the command again, and click "mark as resolved" again. Hopefully the error message at least changes, which would mean that you're making progress. You may have to try a few times with different commands, but doing this should usually resolve this error :)
HOWEVER if the error message references
INSERT INTO gsv_data VALUES ( 'tutorial', 13312, 6656, 512, 512, '2014-05', 1, '', FALSE, now() );
Then you should edit conf/evolutions/73.sql, changing it to
INSERT INTO gsv_data VALUES ( 'tutorial', 13312, 6656, 512, 512, '2014-05', '', FALSE, now(), 1 );
But you'll just have to leave the file like this from now on! But do not commit it ever :)
If you see an error like:
sh: 1: /opt/import-dump.sh: not found
This is a known problem (#1527). The issue is your files somehow have Windows style line endings, which breaks the shell scripts. If you are on Linux or Windows, you can fix it by running the following lines from outside the Docker shell:
sed -i "s/\r//g" db/init.sh
sed -i "s/\r//g" db/import-dump.sh
If you are on Mac then you will want to run this instead:
ex -bsc '%!awk "{sub(/\r/,\"\")}1"' -cx db/init.sh
ex -bsc '%!awk "{sub(/\r/,\"\")}1"' -cx db/import-dump.sh
You will then want to run docker exec -it projectsidewalk-db sh -c "/opt/init.sh" manually before running your make import-dump command again.
For Windows: If you continue to get this error, run the following lines outside the Docker shell:
(Get-Content -Raw "db\import-dump.sh").Endswith("`r`n")
(Get-Content -Raw "db\init.sh").Endswith("`r`n")
Which should both return False.
If they return False and you still receive the error, run the following lines:
docker ps -a
docker stop CONTAINER_ID
docker rm CONTAINER_ID
docker images
docker rmi IMAGE_ID (Make sure to do the ID of projectsidewalk/web)
docker rmi IMAGE_ID (Make sure to do the ID of projectsidewalk/db)
And then run the make dev command again.
If they return 'True' and you still receive the error, then the previous sed commands did not convert your line endings from Windows style to Unix style, so you should run:
(Get-Content -Raw "db\import-dump.sh").Replace("`r`n","`n") | Set-Content "db\import-dump.sh" -Force -NoNewLine
(Get-Content -Raw "db\init.sh").Replace("`r`n","`n") | Set-Content "db\init.sh" -Force -NoNewLine
Which should fix the line endings in the case that the sed commands did not. Now run:
(Get-Content -Raw "db\import-dump.sh").Endswith("`r`n")
(Get-Content -Raw "db\init.sh").Endswith("`r`n")
And make sure these are both False and continue as described above.
If you execute npm start but run into this error:
[info] play - Application started (Dev)
[error] application -
! @7h87h0af1 - Internal server error, for (GET) [/] ->
play.api.Application$$anon$1: Execution exception[[FileNotFoundException: google_maps_api_key.txt (No such file or directory)]]
at play.api.Application$class.handleError(Application.scala:296) ~[play_2.10-2.3.8.jar:2.3.8]
at play.api.DefaultApplication.handleError(Application.scala:402) [play_2.10-2.3.8.jar:2.3.8]
...
Make sure you put the Google Maps API key (received from Mikey) into the root directory of your project with the file named exactly google_maps_api_key.txt.
If it seems like commands that you run using make are just not working, first try reinstalling make. If the issue persists, one workaround has been to look at the Makefile and manually run each command that make is an alias for whenever you need to run something. For example, if you wanted to run make ssh target=web, you would instead run docker exec -it projectsidewalk-web /bin/bash (you can see where the comes from by looking at the Makefile).
If you are having trouble connecting to the database, one potential issue might be that your docker container for the database might not have the configs set correctly for listening addresses. ssh into the db container and edit the /var/lib/postgresql/data/postgresql.conf file, setting listen_addresses = '*'.
If you see an error like this on Windows:
gpg: keyserver receive failed: Server indicated a failure
your system is treating this warning as a fatal error and is not completing the build. To fix this, add this line to the Dockerfile at line 2:
ENV APT_KEY_DONT_WARN_ON_DANGEROUS_USAGE=1
After restarting laptop, you run > make dev but receive an error like this:
> make dev
projectsidewalk-db is up-to-date
Starting projectsidewalk-db ... done
ERROR: Cannot create container for service web: Conflict. The container name "/projectsidewalk-web" is already in use by container "abeedfd010f3bbf9e03d9725b69ab9560b796e9b7e48bef8ff656243fb40a494". You have to remove (or rename) that container to be able to reuse that name.
make: *** [docker-run] Error 1
Mikey suggests removing that docker container: > docker container rm <container_id>. So, in the above error case, we would run > docker container rm /projectsidewalk-web. This worked for me (Jon).
If you are using WSL with Windows, trying running wsl --shutdown in the command prompt. Docker should ask if you want to restart WSL. Click restart. Otherwise, just restart Docker manually and connect again to the WSL Distribution.
