diff --git a/doc/index.rst b/doc/index.rst index 80c0ab2..e137b8a 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -53,6 +53,7 @@ and followed by guides on basic usage, through to the more complicated options a interactive_sizing_options interactive_colour_options selection_and_filtering + offline_mode .. toctree:: :maxdepth: 1 diff --git a/doc/offline_mode.ipynb b/doc/offline_mode.ipynb new file mode 100644 index 0000000..2326e37 --- /dev/null +++ b/doc/offline_mode.ipynb @@ -0,0 +1,798 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "05cfe0de-5e0a-4074-9f13-af43502a632d", + "metadata": {}, + "source": [ + "# Offline Mode Interactive Plots\n", + "\n", + "Interactive plots from datamapplot can be saved as HTML files for easy sharing. One catch is that the files refer to javascript dependencies and fonts that are provided by CDNs. This means that the generated HTML files will not work when offline, or behind strict firewalls. This can be problematic for users stuck in those situations. To remedy this datamapplot has support for an \"offline mode\" that embeds the relevant javascript and fonts directly into the HTML so the resulting output is fully independent and can be viewed offline or behind firewalls.\n", + "\n", + "Of course to do this the relevant javascript and fonts need to be collected to be embedded. For this you will need an internet connection. To make this slightly easier a command-line tool is provided: ``dmp_offline_cache``. This will allow you to download and store the relevant data while connected to the internet, and then make use of the cached files for offline mode work thereafter. Let's have a look at the help for this tool:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "e2155424-9497-42ef-b1c8-234e567a9620", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "usage: dmp_offline_cache [-h] [--js_urls JS_URLS [JS_URLS ...]]\n", + " [--font_names FONT_NAMES [FONT_NAMES ...]]\n", + " [--refresh] [--js_cache_file JS_CACHE_FILE]\n", + " [--font_cache_file FONT_CACHE_FILE]\n", + "\n", + "Cache JS and font files for offline mode\n", + "\n", + "options:\n", + " -h, --help show this help message and exit\n", + " --js_urls JS_URLS [JS_URLS ...]\n", + " CDN URLs to fetch and cache js from\n", + " --font_names FONT_NAMES [FONT_NAMES ...]\n", + " Names of google font fonts to cache\n", + " --refresh Force refresh cached files\n", + " --js_cache_file JS_CACHE_FILE\n", + " Path to save JS cache file\n", + " --font_cache_file FONT_CACHE_FILE\n", + " Path to save font cache file\n", + "\u001b[0m" + ] + } + ], + "source": [ + "!dmp_offline_cache --help" + ] + }, + { + "cell_type": "markdown", + "id": "32bd0261-d2d3-473d-bba8-b4cd3332c9d5", + "metadata": {}, + "source": [ + "You can specify which javascripty files you wish to collect, which font names you wish to collect, and file paths of where to store the cache files. If these aren't specified default choices will be used that should cover most basic cases. There is also a ``--refresh`` option that will download the latest versions of javascript libraries and fonts to make sure you are up to date.\n", + "\n", + "We can see what javascript and fonts will be collected, and where they will be cached by inspecting variables in the ``offline_mode_caching`` module:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "c25c67c2-aa7c-4bb6-8d05-39936f2a88a9", + "metadata": {}, + "outputs": [], + "source": [ + "import datamapplot.offline_mode_caching" + ] + }, + { + "cell_type": "markdown", + "id": "c3a42883-20a2-4d7e-9a55-f3226482bc00", + "metadata": {}, + "source": [ + "The Javascriupt libraries cached by default are stored in the ``DEFAULT_URLS`` variable:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "3f1235e1-f801-48e6-8d7c-ca38131e41f4", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "['https://unpkg.com/deck.gl@latest/dist.min.js',\n", + " 'https://unpkg.com/apache-arrow@latest/Arrow.es2015.min.js',\n", + " 'https://unpkg.com/d3@latest/dist/d3.min.js',\n", + " 'https://unpkg.com/jquery@3.7.1/dist/jquery.min.js',\n", + " 'https://unpkg.com/d3-cloud@1.2.7/build/d3.layout.cloud.js']" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "datamapplot.offline_mode_caching.DEFAULT_URLS" + ] + }, + { + "cell_type": "markdown", + "id": "d0253f4a-f063-4dc1-9268-c18ba2e9d327", + "metadata": {}, + "source": [ + "A selection of fonts that are good choices for interactive datamapplots are stored in ``BASE_FONTS``:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "240c416b-91b2-45cc-828b-a649294fc13a", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "['Roboto',\n", + " 'Open Sans',\n", + " 'Montserrat',\n", + " 'Oswald',\n", + " 'Merriweather',\n", + " 'Merriweather Sans',\n", + " 'Playfair Display',\n", + " 'Playfair Display SC',\n", + " 'Roboto Condensed',\n", + " 'Ubuntu',\n", + " 'Cinzel',\n", + " 'Cormorant',\n", + " 'Cormorant SC',\n", + " 'Marcellus',\n", + " 'Marcellus SC',\n", + " 'Anton',\n", + " 'Anton SC',\n", + " 'Arsenal',\n", + " 'Arsenal SC',\n", + " 'Baskervville',\n", + " 'Baskervville SC',\n", + " 'Lora',\n", + " 'Quicksand',\n", + " 'Bebas Neue']" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "datamapplot.offline_mode_caching.BASE_FONTS" + ] + }, + { + "cell_type": "markdown", + "id": "79f619f9-13aa-47a4-abe0-16e16bec9cd9", + "metadata": {}, + "source": [ + "Finally you can see where the cache files will be stored by checking ``DEFAULT_CACHE_FILES``. Note that this is operating system dependent, so the files may have different locations if you are running on a Mac or Windows." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "2fbee0b0-cfd0-4f58-bad9-24fb1bfe2ea5", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'javascript': '/home/lmmcinn/.local/share/datamapplot/datamapplot_js_encoded.json',\n", + " 'fonts': '/home/lmmcinn/.local/share/datamapplot/datamapplot_fonts_encoded.json'}" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "datamapplot.offline_mode_caching.DEFAULT_CACHE_FILES" + ] + }, + { + "cell_type": "markdown", + "id": "50017f75-0736-4af4-9ca9-1bdd6e34f3d0", + "metadata": {}, + "source": [ + "Now let's see an example of this in action. We will import datamapplot and load up some data suitable for making an interactive plot." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "d5ac8a54-76f0-41db-b783-5eef61087c3b", + "metadata": {}, + "outputs": [], + "source": [ + "import datamapplot\n", + "import numpy as np\n", + "import requests\n", + "import io\n", + "\n", + "base_url = \"https://github.com/TutteInstitute/datamapplot\"\n", + "data_map_file = requests.get(\n", + " f\"{base_url}/raw/main/examples/arxiv_ml_data_map.npy\"\n", + ")\n", + "arxivml_data_map = np.load(io.BytesIO(data_map_file.content))\n", + "arxivml_label_layers = []\n", + "for layer_num in range(5):\n", + " label_file = requests.get(\n", + " f\"{base_url}/raw/interactive/examples/arxiv_ml_layer{layer_num}_cluster_labels.npy\"\n", + " )\n", + " arxivml_label_layers.append(np.load(io.BytesIO(label_file.content), allow_pickle=True))\n", + "hover_data_file = requests.get(\n", + " f\"{base_url}/raw/interactive/examples/arxiv_ml_hover_data.npy\"\n", + ")\n", + "arxiv_hover_data = np.load(io.BytesIO(hover_data_file.content), allow_pickle=True)" + ] + }, + { + "cell_type": "markdown", + "id": "dbdeb50f-69e6-48a7-a9ff-bb666616e1b0", + "metadata": {}, + "source": [ + "Now we make a plot exactly as we normally would, but include the extra keyword argument ``offline_mode=True``. If you have stored your cache files somewhere other than the default location you can specify these via ``offline_mode_js_data_file`` and ``offline_mode_font_data_file``. It is worth noting that if you haven't run ``dmp_offline_cache`` on the command line, or the relevant files in the default location don't yet exist, they will be created for you." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "548ee419-abec-4c47-8d5b-93e6bae84ebc", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " " + ], + "text/plain": [ + "" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "plot = datamapplot.create_interactive_plot(\n", + " arxivml_data_map,\n", + " arxivml_label_layers[0],\n", + " arxivml_label_layers[2],\n", + " arxivml_label_layers[4],\n", + " hover_text = arxiv_hover_data,\n", + " font_family=\"Cormorant SC\",\n", + " title=\"ArXiv Machine Learning Landscape\",\n", + " sub_title=\"A data map of papers from the Machine Learning section of ArXiv\",\n", + " logo=\"https://upload.wikimedia.org/wikipedia/commons/thumb/b/bc/ArXiv_logo_2022.svg/512px-ArXiv_logo_2022.svg.png\",\n", + " logo_width=180,\n", + " on_click=\"window.open(`http://google.com/search?q=\\\"{hover_text}\\\"`)\",\n", + " enable_search=True,\n", + " darkmode=True,\n", + " offline_mode=True,\n", + ")\n", + "plot" + ] + }, + { + "cell_type": "markdown", + "id": "82ce0a8e-5829-4b1f-a97e-cbe19e25260f", + "metadata": {}, + "source": [ + "You can now save the resulting file and have an interactive plot that works offline and behind firewalls." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "fe0d1c4b-8506-4457-ab19-3fafc925601a", + "metadata": {}, + "outputs": [], + "source": [ + "plot.save(\"arxiv_ml_offline_mode.html\")" + ] + }, + { + "cell_type": "markdown", + "id": "3a1ea4f2-64f1-48cd-971e-9ba4b58d25e8", + "metadata": {}, + "source": [ + "Equally, if you need to work behind a firewall you can use ``dmp_offline_cache`` to create cache files when outside the firewall and get the resulting files transferred inside the firewall and run successfully as long as you use ``offline_mode=True``." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}