Skip to content

Commit

Permalink
docs: Add documentation on how to create a Docker image, closes #234
Browse files Browse the repository at this point in the history
  • Loading branch information
jpmckinney committed Jul 16, 2024
1 parent b229169 commit ee4e036
Show file tree
Hide file tree
Showing 3 changed files with 83 additions and 2 deletions.
4 changes: 2 additions & 2 deletions docs/contributing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Contributing

These docs describe how to setup and contribute to Scrapyd.

Reporting Issues & Bugs
Reporting issues & bugs
-----------------------

Issues should be reported to the Scrapyd project `issue tracker`_ on GitHub
Expand Down Expand Up @@ -48,7 +48,7 @@ And their unit-tests are in::

scrapyd/tests/test_scheduler.py

Installing Locally
Installing locally
------------------

To install a locally edited version of Scrapyd onto the system to use and test, inside the project root run:
Expand Down
80 changes: 80 additions & 0 deletions docs/deploy.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,83 @@ Deploying your project
======================

Deploying your project involves eggifying it and uploading the egg to Scrapyd via the `addversion.json <https://scrapyd.readthedocs.org/en/latest/api.html#addversion-json>`_ webservice. You can do this manually, but the easiest way is to use the `scrapyd-deploy` tool provided by `scrapyd-client <https://github.com/scrapy/scrapyd-client>`_ which will do it all for you.

.. _docker:

Create a Docker image
---------------------

If you prefer to create a Docker image, you can copy this ``Dockerfile`` template into your Scrapy project, and adapt it.

.. code-block:: dockerfile
# Build an egg of your project.
FROM python as build-stage
RUN pip install --no-cache-dir scrapyd-client
WORKDIR /workdir
COPY . .
RUN scrapyd-deploy --build-egg=myproject.egg
# Build the image.
FROM python:alpine
# Install Scrapy dependencies - and any others for your project.
RUN apk --no-cache add --virtual build-dependencies \
gcc \
musl-dev \
libffi-dev \
libressl-dev \
libxml2-dev \
libxslt-dev \
&& pip install --no-cache-dir \
scrapyd \
&& apk del build-dependencies \
&& apk add \
libressl \
libxml2 \
libxslt
# Mount two volumes for configuration and runtime.
VOLUME /etc/scrapyd/ /var/lib/scrapyd/
COPY ./scrapyd.conf /etc/scrapyd/
RUN mkdir -p /src/eggs/myproject
COPY --from=build-stage /workdir/myproject.egg /src/eggs/myproject/1.egg
EXPOSE 6800
ENTRYPOINT ["scrapyd", "--pidfile="]
Where your ``scrapy.cfg`` file, used by ``scrapyd-deploy``, might be:

.. code-block:: ini
[settings]
default = myproject.settings
[deploy]
url = http://localhost:6800
project = myproject
And your ``scrapyd.conf`` file might match the :ref:`config-example`, except:

.. code-block:: ini
[scrapyd]
bind_address = 0.0.0.0
logs_dir = /var/lib/scrapyd/logs
items_dir = /var/lib/scrapyd/items
dbs_dir = /var/lib/scrapyd/dbs
eggs_dir = /src/eggs
...
1 change: 1 addition & 0 deletions docs/news.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ Added
- Respond to HTTP ``OPTIONS`` method requests.
- Add environment variables to override common options. See :doc:`config`.
- Add documentation on how to add webservices (endpoints). See :ref:`config-services`.
- Add documentation on how to create a Docker image. See :ref:`docker`.

Changed
~~~~~~~
Expand Down

0 comments on commit ee4e036

Please sign in to comment.