Merge pull request #366 from ELVIS-Project/devel_docs

Merge devel_docs to master

VERSION_HISTORY.md

@@ -7,6 +7,7 @@ the top of the file.
     - Importing no longer uses/stores cached pickle files.
     - WorkflowManager.output() produces many table-format results if "count frequency" is False.
     - WorkflowManager.output() changes the column name for table-like outputs, according to the experiment.
+    - Greatly improved API documentation.
 * 2.2.1:
     - set music21 version to 2.1.2 exclusively
 * 2.2.0:

doc/api/conf.py

@@ -50,8 +50,8 @@
 master_doc = 'index'
 
 # General information about the project.
-project = 'VIS'
-copyright = '2013 to 2014 Christopher Antila, CC-BY-SA 4.0'
+project = 'VIS Framework'
+copyright = '2013 to 2015 Christopher Antila, CC-BY-SA 4.0'
 
 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the
@@ -116,10 +116,10 @@
 
 # The name for this set of Sphinx documents.  If None, it defaults to
 # "<project> v<release> documentation".
-#html_title = None
+html_title = '{} Documentation (Version {})'.format(project, release)
 
 # A shorter title for the navigation bar.  Default is the same as html_title.
-html_short_title = "VIS API (v%s)" % release
+html_short_title = 'VIS API ({})'.format(release)
 
 # The name of an image file (relative to this directory) to place at the top
 # of the sidebar.
@@ -196,8 +196,8 @@
 # Grouping the document tree into LaTeX files. List of tuples
 # (source start file, target name, title, author, documentclass [howto/manual]).
 latex_documents = [
-  ('index', 'vis.tex', u'VIS Framework Documentation',
-   u'Christopher Antila', 'manual'),
+  ('index', 'vis-framework.tex', 'VIS Framework Documentation',
+   'Christopher Antila', 'manual'),
 ]
 
 # The name of an image file (relative to this directory) to place at the top of

doc/api/index.rst

@@ -7,49 +7,57 @@
     :alt: Logo for the VIS Music Analysis Framework
     :width: 100 %
 
-This is the documentation for the VIS music analysis framework, made available as free software according to the terms of the Affero General Public Licence, version 3 or any later. VIS is a Python package that uses the music21 and pandas libraries to build a ridiculously flexible and preposterously easy system for writing computer music analysis programs. The developers hope to lower the barrier to empirical music analysis to the point that North American music theorists feel compelled to use techniques common among scientists since the 1980s. The VIS framework was produced by the McGill University team of the ELVIS Project. Learn more about the project at our website, http://elvisproject.ca.
+This is the API documentation for the VIS Framework for Music Analysis.
+The VIS Framework is a Python package that uses music21 and pandas to build a flexible and easy system for writing computer music analysis programs.
+By providing a free-and-open solution (AGPLv3+) we hope to lower the barrier to empirical music analysis.
+The VIS Framework was produced at McGill University as part of the ELVIS Project (http://elvisproject.ca).
+VIS has become famous through cameo appearances on television programmes such as "The Amazing Race" and "CSI: Miami," plus lead roles at such academic conferences as the International Society for Music Information Retrieval and the Society for Music Theory.
 
 About This Documentation
 ------------------------
 
-Who Should Read This Documentation
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-**If you are a programmer** who wishes to use the ``vis`` Python objects directly, to use the framework as part of another project, or to add functionality to your own instance of the ``vis`` Web app, you should read this document.
-
-**If you are not a programmer**, this document is probably not for you, since we assume readers are fluent with Python. You may still wish to read the :ref:`design_principles` section to get an idea of how to use our framework, but it is still quite technical. If you wish to learn about computer-driven music analysis, if you are looking for help using the ELVIS counterpoint Web app, or if you are looking for research findings produced by the ELVIS Project, you will find these resources on the `ELVIS website <http://elvisproject.ca>`_ when they become available.
-
-What This Documentation Covers
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-The VIS API documentation explains the framework's architecture, how to install and use the framework, and how to use the built-in analyzer modules.
+The API is written for programmers interested in symbolic music information retrieval (i.e., music theory analysis performed by computers).
+Musicologists and programmers alike may access our Counterpoint Web App software from `counterpoint.elvisproject.ca <https://counterpoint.elvisproject.ca>`_, and some of our research findings from the `ELVIS Project's website <https://elvisproject.ca/research/>`_.
 
-How to Use This Documentation
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-**If you are new to VIS**, we recommend you read :ref:`design_principles` and continue through both tutorials.
+The rest of the API discusses the VIS Framework's architecture, how to install an use the Framework, and how to implement additional program modules for your own use.
 
-**If you forgot the basics**, we recommend you read this paragraph. The VIS framework uses two models (:class:`~vis.models.indexed_piece.IndexedPiece` and :class:`~vis.models.aggregated_pieces.AggregatedPieces`) to fetch results for one or multiple pieces, respectively. Use :meth:`metadata` to fetch metadata and :meth:`get_data` to run analyses and fetch results. Call :meth:`get_data` with a list of analyzers to run, and a dictionary with the settings they use. Learn what an analyzer does, what it requires for input, and what settings it uses by reading the class documentation. Learn the output format of an analyzer by reading the :meth:`run` documentation.
+One-Paragraph Introduction
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-**If you have a question about an analyzer**, learn what it does, what it requires for input, and what settings it uses by reading the class documentation. Learn the output format by reading the :meth:`run` documentation.
+The VIS Framework uses two data models (:class:`~vis.models.indexed_piece.IndexedPiece` and :class:`~vis.models.aggregated_pieces.AggregatedPieces`) to fetch results for one or multiple pieces, respectively.
+Call their :meth:`~vis.models.indexed_piece.IndexedPiece.get_data` method with a list of analyzer classes to run, and a dictionary with their settings.
+After you develop an analysis workflow, add it to the :class:`~vis.workflow.WorkflowManager` for more consistent operation.
 
-.. important:: Remember to have a lot of fun.
+Table of Contents
+-----------------
 
-Use the ``vis`` Framework
--------------------------
+Learn about and Install the VIS Framework
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 .. toctree::
     :maxdepth: 2
 
     about
+    install_and_test
+
+Programming Tutorials for VIS
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+.. toctree::
+    :maxdepth: 1
+
+    tutorial-melodic_ngrams
+    tutorial-workflowmanager
 
 
 API Specification
------------------
+^^^^^^^^^^^^^^^^^
 .. toctree::
-   :maxdepth: 3
+   :maxdepth: 5
 
    modules
 
 
 Indices and Tables
-------------------
+^^^^^^^^^^^^^^^^^^
 * :ref:`genindex`
 * :ref:`modindex`
 * :ref:`search`

doc/api/install_and_test.rst

@@ -0,0 +1,147 @@
+
+.. _install_and_test:
+
+Install the Framework
+---------------------
+
+You must install the VIS Framework before you use it.
+Our recommended installation method depends on how you plan to use VIS.
+
+If you plan to use VIS as a program or a library, you should install the Framework for *deployment*.
+This includes users who will conduct their own analyses with the built-in modules, and users who will host an instance of the `Counterpoint Web App <https://github.com/ELVIS_Project/web-vis>`_.
+A deployment installation will always be "stable" software, meaning that the software has been tested more thoroughly, and we can offer a higher assurance of correctness.
+
+If you plan to use VIS as a framework, you should install the Framework for *development*.
+This includes users who will write their own analyzers for the VIS Framework, and users who will modify existing modules.
+A development installation allows you to use the most up-to-date versions of VIS, and even to suggest your improvements for inclusion in a VIS release, although these versions may crash more often and be less correct than deployment versions.
+
+You may safely install the VIS Framework multiple times on the same computer, including a mixture of deployment and development installations.
+
+Python 3
+^^^^^^^^
+
+With the release of VIS Framework 2.1, VIS can run and is tested with Python 2.7, 3.3, and 3.4.
+We recommend a Python 3 release for reasons of future compatibility.
+The music21 developers have announced plans to drop support for Python 2.7 in calendar year 2016, at which point VIS will also drop support for Python 2.7.
+Python 3.4 offers several major enhancements over Python 3.3, including improved speed of execution, so we recommend you use Python 3.4 if possible.
+
+Install for Deployment
+^^^^^^^^^^^^^^^^^^^^^^
+You must install the VIS Framework before you use it.
+If you will not write extensions for the Framework, you may use ``pip`` to install the package from the Python Package Index (PyPI---`https://pypi.python.org/pypi/vis-framework/ <https://pypi.python.org/pypi/vis-framework/>`_). Run this command::
+
+    $ pip install vis-framework
+
+You may also wish to install some or all of the optional dependencies:
+
+    * ``numexpr`` and ``bottleneck``, which speed up ``pandas``.
+    * ``openpyxl``, which allows ``pandas`` to export Excel-format spreadsheets.
+    * ``cython`` and ``tables``, which allow ``pandas`` to export HDF5-format binary files.
+
+You may install optional dependencies in the same ways as VIS itself. For example::
+
+    $ pip install numexpr bottleneck
+
+Install for Development
+^^^^^^^^^^^^^^^^^^^^^^^
+If you wish to install the VIS Framework for development work, we recommend you clone our Git repository from https://github.com/ELVIS-Project/vis/, or even make your own fork on GitHub.
+You may also wish to checkout a particular version for development with the "checkout" command, as ``git checkout tags/vis-framework-1.2.3`` or ``git checkout master``.
+
+If you installed git, but you need help to clone a repository, you may find useful information in the `git documentation <http://git-scm.com/book/en/Git-Basics-Getting-a-Git-Repository#Cloning-an-Existing-Repository>`_.
+
+After you clone the VIS repository, you should install its dependencies (listed in the "requirements.py" file), for which we recommend you use ``pip``.
+From the main VIS directory, run ``pip install -r requirements.txt`` to automatically download and install the library dependencies.
+We also recommend you run ``pip install -r optional_requirements.txt`` to install several additional packages that improve the speed of pandas and allow additional output formats (Excel, HDF5).
+You may need to use ``sudo`` or ``su`` to run pip with administrator privileges, though we recommend using virtualenv.
+If you do not have pip installed, use your package manager (the package is probably called ``python-pip``---at least for users of Fedora, Ubuntu, and openSUSE).
+If you are one of the unfortunate souls who uses Windows, or worse, Mac OS X, then clearly we come from different planets.
+The `pip documentation <http://www.pip-installer.org/en/latest/installing.html>`_ may help you.
+
+During development, you should usually start ``python`` (or ``ipython``, etc.) from within the main "vis" directory to ensure proper importing.
+
+The :class:`~vis.workflow.WorkflowManager` is not required for the framework's operation.
+We recommend you use the :class:`WorkflowManager` directly or as an example to write new applications.
+The vis framework gives you tools to answer a wide variety of musical questions.
+The :class:`WorkflowManager` uses the framework to answer specific questions.
+Please refer to :ref:`use_the_workflowmanager` for more information.
+
+Optional: Test the Framework Before Use
+---------------------------------------
+
+You may wish to run the VIS Framework's automated test suite before you use it, to confirm the installation was successful and is working as expected.
+
+To run the test suite:
+
+#. Change into the main VIS directory (this will depend on your installation, but it will be similar to ``/usr/lib/python3.4/site-packages/vis-framework`` or ``/home/crantila/virtualenvs/vis-virtenv3/lib/python3.4/site-packages/vis-framework``.
+#. If you installed VIS with a virtualenv, activate the virtualenv (e.g., ``source ~/virtualenvs/vis-virtenv3/bin/activate``).
+#. Run the test script: ``python run_tests.py``.
+#. Python prints a ``.`` character for every successful test, and an error or warning for every test that fails.
+
+If Tests Fail
+^^^^^^^^^^^^^
+
+If you are using a deployment release and one or more of the automated tests fail, the most likely reason is that the installation was somehow unsuccessful.
+This may result from using untested versions of a dependency, unexpectedly troublesome environment variables, or other factors.
+If you used distribution packages (i.e., did not use virtualenv) and tests fail, we do recommend in that case installing with virtualenv.
+Both with and without virtualenv, if tests fail, ensure you are using the exact package versions indicated in ``requirements.txt``, since they are the versions used by the development team.
+
+If you are using a development release and automated tests fail, check if those tests also fail on our "Travis CI" test server.
+If so, the VIS development team is already aware of the failure and working toward a fix, since Travis will notify us by email if a test fails.
+You may also try the version-related advice in the previous paragraph, though you may also wish to prepare a fix for the problem and submit it to us as a GitHub pull request.
+
+Optional: Install R and ggplot2 for Graphs
+------------------------------------------
+
+If you wish to produce graphs with the VIS Framework, you must install an R interpreter and the *ggplot2* library.
+We test with 3.0.x versions of R.
+
+If you use a "Windows" computer, download a pre-compiled R interpreter from http://cran.r-project.org.
+If you use an "OS X" computer, you may download a pre-compiled binary or use a package from Homebrew, Fink, or MacPorts.
+If you use a "Linux" computer (or "BSD," etc.), check your package manager for an appropriate version of R.
+You may have difficulty searching for "R," as in ``yum search R``, since it is a common letter, so we recommend you assume the package is called "R" and try to search only if that does not work.
+If your distribution does not provide an R binary, or provides an older version than 3.0.0, install R from source code: http://cran.r-project.org/doc/manuals/r-release/R-admin.html.
+For all operating systems, if you encounter a problem, the R manuals offer extensive help, but require careful attention.
+
+After you install R, you must install the *ggplot2* package.
+If you installed R with your package manager, we recommend you search for and use the "ggplot" package, if one is provided.
+(Distribution packages offer maximum compatibility, and take advantage of centralized software updates).
+
+Use the following instructions, which work on all operating systems, if you do not have a distribution package for *ggplot2*.
+
+#. Start R (with superuser privileges, if not using Windows).
+#. Run the following command to install *ggplot2*::
+
+    install.packages("ggplot2")
+
+#. Run the following program to test R and *ggplot2*::
+
+    huron <- data.frame(year=1875:1972, level=as.vector(LakeHuron))
+    library(plyr)
+    huron$decade <- round_any(huron$year, 10, floor)
+    library(ggplot)
+    h <- ggplot(huron, aes(x=year))
+    h + geom_ribbon(aes(ymin=level-1, ymax=level+1))
+
+Expect to see a chart like this:
+
+.. figure:: geom_ribbon-6.png
+    :alt: Ribbon chart produced by the *ggplot2* package in the R language.
+
+    Image credit: taken from the `"ggplot2" documentation <http://docs.ggplot2.org/current/geom_ribbon.html>`_ on 26 November 2013; reused here under the GNU General Public License, version 2.
+
+Quit R. You do not need to save your workspace::
+
+    q()
+
+Optional: Install LilyPond for Annotated Scores
+-----------------------------------------------
+
+If you wish to produce annotated scores with the VIS Framework, you must install LilyPond.
+
+The *outputlilypond* module, used by VIS, is targeted for 2.18.x versions of LilyPond, though it should also work with 2.16.x versions.
+We do note recommend versions numbered 2.15.x, 2.17.x, and 2.19.x, since these are intended only for LilyPond developers, and they may crash or produce incorrect output.
+If possible, we recommend you install LilyPond with your distribution's package management system, or (on "OS X") with a package manager such as Homebrew.
+"Windows" users, and users who do not have a 2.16.x or 2.18.x version of LilyPond available from their package manager, may download and install a pre-compiled version of LilyPond from their website, `lilypond.org <https://www.lilypond.org>`_.
+
+We very strongly discourage users from compiling LilyPond themselves.
+There is very little chance that the benefits of self-compilation will outweigh the mental distress.