Chat functionality and chat UI

CHANGES.rst

@@ -1,3 +1,8 @@
+Version 1.3.0
+-------------
+- Chat feature for text analysis.
+
+
 Version 1.2.0
 -------------
 - Support for all Penn POS tags

README.rst

@@ -89,15 +89,23 @@ of interest where we endeavor to find them in order to shed light into our data.
 Wordview offers several anomaly and outlier detection functions.
 
 
-
 Cluster Analysis
 ****************
 Clustering can be used to identify different groups of documents with similar information, in an unsupervised fashion.
 Despite it's ability to provide valuable insights into your data, you do not need labeled data for clustering.
 
 
+
+Chat with Wordview
+******************
+Worldview integrates GPT-3.5-Turbo to enable the users to interact with their data and get insights from it via Natural Language.
+
+.. image:: sphinx-docs/figs/chat.png
+   :width: 100%
+   :align: center
+
 Utilities
-#########
+*********
 Wordview offers a number of utility functions that you can use for common pre and post processing tasks in NLP.
 
 Contributing

docs/_sources/chat.rst.txt

@@ -0,0 +1,32 @@
+Chat with Wordview
+##################
+
+Worldview integrates GPT-3.5-Turbo to enable the users to interact with their data and get insights from it via Natural Language.
+See examples of chatting with different components of Wordview below.
+
+
+Chat with TextStatsPlots
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+After allowing Wordview to process and analyze your corpus, you can 
+call the `chat` method to interact with the data and get insights from it via Natural Language.
+
+.. code:: python
+
+   import json
+
+   import pandas as pd
+
+   from wordview.text_analysis import TextStatsPlots
+   imdb_df = pd.read_csv("data/IMDB_Dataset_sample_5k.csv")
+   with open("wordview/chat/secrets/openai_api_key.json", "r") as f:
+      credentials = json.load(f)
+
+   tsp = TextStatsPlots(df=imdb_df, text_column="review")
+   tsp.chat(api_key=credentials.get("openai_api_key"))
+
+The chat UI is available under http://127.0.0.1:5000/
+
+|chat|
+
+.. |chat| image:: ../figs/chat.png

docs/api.html

@@ -73,6 +73,8 @@
 <li class="toctree-l2"><a class="reference internal" href="#text-analysis">Text Analysis</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="#wordview.text_analysis.TextStatsPlots"><code class="docutils literal notranslate"><span class="pre">TextStatsPlots</span></code></a><ul>
 <li class="toctree-l4"><a class="reference internal" href="#wordview.text_analysis.TextStatsPlots.__init__"><code class="docutils literal notranslate"><span class="pre">TextStatsPlots.__init__()</span></code></a></li>
+<li class="toctree-l4"><a class="reference internal" href="#wordview.text_analysis.TextStatsPlots.chat"><code class="docutils literal notranslate"><span class="pre">TextStatsPlots.chat()</span></code></a></li>
+<li class="toctree-l4"><a class="reference internal" href="#wordview.text_analysis.TextStatsPlots.return_stats"><code class="docutils literal notranslate"><span class="pre">TextStatsPlots.return_stats()</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="#wordview.text_analysis.TextStatsPlots.show_bar_plots"><code class="docutils literal notranslate"><span class="pre">TextStatsPlots.show_bar_plots()</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="#wordview.text_analysis.TextStatsPlots.show_distplot"><code class="docutils literal notranslate"><span class="pre">TextStatsPlots.show_distplot()</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="#wordview.text_analysis.TextStatsPlots.show_insights"><code class="docutils literal notranslate"><span class="pre">TextStatsPlots.show_insights()</span></code></a></li>
@@ -191,6 +193,35 @@ <h2>Text Analysis<a class="headerlink" href="#text-analysis" title="Permalink to
 </dl>
 </dd></dl>
 
+<dl class="py method">
+<dt class="sig sig-object py" id="wordview.text_analysis.TextStatsPlots.chat">
+<span class="sig-name descname"><span class="pre">chat</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">api_key</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">''</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#wordview.text_analysis.TextStatsPlots.chat" title="Permalink to this definition"></a></dt>
+<dd><p>Chat with OpenAI’s latest model about the results of Wordview’s text analysis.
+Access the chat UI in your localhost under <a class="reference external" href="http://127.0.0.1:5000/">http://127.0.0.1:5000/</a></p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><p><strong>api_key</strong> – OpenAI API key.</p>
+</dd>
+<dt class="field-even">Returns<span class="colon">:</span></dt>
+<dd class="field-even"><p>None</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="wordview.text_analysis.TextStatsPlots.return_stats">
+<span class="sig-name descname"><span class="pre">return_stats</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><span class="pre">dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">Any</span><span class="p"><span class="pre">]</span></span></span></span><a class="headerlink" href="#wordview.text_analysis.TextStatsPlots.return_stats" title="Permalink to this definition"></a></dt>
+<dd><p>Returns dataset statistics, including:
+Language/s
+Number of unique words
+Number of all words
+Number of documents
+Median document length
+Number of nouns
+Number of adjectives
+Number of verbs.</p>
+</dd></dl>
+
 <dl class="py method">
 <dt class="sig sig-object py" id="wordview.text_analysis.TextStatsPlots.show_bar_plots">
 <span class="sig-name descname"><span class="pre">show_bar_plots</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">pos</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">layout_settings</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">Any</span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">{}</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">plot_settings</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><span class="pre">str</span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">{}</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><span class="pre">None</span></span></span><a class="headerlink" href="#wordview.text_analysis.TextStatsPlots.show_bar_plots" title="Permalink to this definition"></a></dt>
@@ -391,7 +422,6 @@ <h2>Extraction &amp; Analysis of Multiword Expressions<a class="headerlink" href
 NP: {&lt;DT&gt;?&lt;JJ&gt;*&lt;NN&gt;} # Noun phrase
 You can use multiple and/or nested patterns, separated by a newline character e.g.:
 custom_pattern = ‘’’
-NP: {&lt;DT&gt;?&lt;JJ&gt;*&lt;NN&gt;} # Noun phrase
 VP: {&lt;MD&gt;?&lt;VB.*&gt;&lt;NP|PP|CLAUSE&gt;+$} # Verb phrase
 PROPN: {&lt;NNP&gt;+} # Proper noun
 ADJP: {&lt;RB|RBR|RBS&gt;*&lt;JJ&gt;} # Adjective phrase

docs/chat.html

@@ -0,0 +1,152 @@
+<!DOCTYPE html>
+<html class="writer-html5" lang="en" >
+<head>
+  <meta charset="utf-8" /><meta name="generator" content="Docutils 0.18.1: http://docutils.sourceforge.net/" />
+
+  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+  <title>Chat with Wordview &mdash; wordview &#34;1.1.2&#34; documentation</title>
+      <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
+      <link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
+      <link rel="stylesheet" href="_static/custom.css" type="text/css" />
+      <link rel="stylesheet" href="_static/fonts.css" type="text/css" />
+  <!--[if lt IE 9]>
+    <script src="_static/js/html5shiv.min.js"></script>
+  <![endif]-->
+
+        <script src="_static/jquery.js"></script>
+        <script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
+        <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
+        <script src="_static/doctools.js"></script>
+        <script src="_static/sphinx_highlight.js"></script>
+    <script src="_static/js/theme.js"></script>
+    <link rel="index" title="Index" href="genindex.html" />
+    <link rel="search" title="Search" href="search.html" /> 
+</head>
+
+<body class="wy-body-for-nav"> 
+  <div class="wy-grid-for-nav">
+    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
+      <div class="wy-side-scroll">
+        <div class="wy-side-nav-search" >
+
+
+
+          <a href="index.html" class="icon icon-home">
+            wordview
+          </a>
+<div role="search">
+  <form id="rtd-search-form" class="wy-form" action="search.html" method="get">
+    <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
+    <input type="hidden" name="check_keywords" value="yes" />
+    <input type="hidden" name="area" value="default" />
+  </form>
+</div>
+        </div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
+              <p class="caption" role="heading"><span class="caption-text">Quick Start</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="start.html">Installation</a></li>
+<li class="toctree-l1"><a class="reference internal" href="start.html#what-is-wordview">What is Wordview</a></li>
+<li class="toctree-l1"><a class="reference internal" href="start.html#tutorials">Tutorials</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Exploratory Data Analysis (EDA)</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="textstats.html">Text Analysis</a></li>
+<li class="toctree-l1"><a class="reference internal" href="labels.html">Label Analysis</a></li>
+<li class="toctree-l1"><a class="reference internal" href="mwes.html">Analysis &amp; Extraction of Multiword Expressions (MWEs)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="bias.html">Bias Analysis</a></li>
+<li class="toctree-l1"><a class="reference internal" href="anomalies.html">Analysis of Anomalies &amp; Outliers</a></li>
+<li class="toctree-l1"><a class="reference internal" href="clustering.html">Cluster Analysis</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Utilities</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="utilities.html">Utilities</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Contributing</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="contributing.html">Contributing to wordview</a></li>
+<li class="toctree-l1"><a class="reference internal" href="codeofconduct.html">Contributor Code of Conduct</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">API Reference</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="api.html">API</a></li>
+</ul>
+
+        </div>
+      </div>
+    </nav>
+
+    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
+          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
+          <a href="index.html">wordview</a>
+      </nav>
+
+      <div class="wy-nav-content">
+        <div class="rst-content">
+          <div role="navigation" aria-label="Page navigation">
+  <ul class="wy-breadcrumbs">
+      <li><a href="index.html" class="icon icon-home" aria-label="Home"></a></li>
+      <li class="breadcrumb-item active">Chat with Wordview</li>
+      <li class="wy-breadcrumbs-aside">
+            <a href="_sources/chat.rst.txt" rel="nofollow"> View page source</a>
+      </li>
+  </ul>
+  <hr/>
+</div>
+          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
+           <div itemprop="articleBody">
+
+  <section id="chat-with-wordview">
+<h1>Chat with Wordview<a class="headerlink" href="#chat-with-wordview" title="Permalink to this heading"></a></h1>
+<p>Worldview integrates GPT-3.5-Turbo to enable the users to interact with their data and get insights from it via Natural Language.
+See examples of chatting with different components of Wordview below.</p>
+<section id="chat-with-textstatsplots">
+<h2>Chat with TextStatsPlots<a class="headerlink" href="#chat-with-textstatsplots" title="Permalink to this heading"></a></h2>
+<p>After allowing Wordview to process and analyze your corpus, you can
+call the <cite>chat</cite> method to interact with the data and get insights from it via Natural Language.</p>
+<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">json</span>
+
+<span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span>
+
+<span class="kn">from</span> <span class="nn">wordview.text_analysis</span> <span class="kn">import</span> <span class="n">TextStatsPlots</span>
+<span class="n">imdb_df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s2">&quot;data/IMDB_Dataset_sample_5k.csv&quot;</span><span class="p">)</span>
+<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">&quot;wordview/chat/secrets/openai_api_key.json&quot;</span><span class="p">,</span> <span class="s2">&quot;r&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
+   <span class="n">credentials</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
+
+<span class="n">tsp</span> <span class="o">=</span> <span class="n">TextStatsPlots</span><span class="p">(</span><span class="n">df</span><span class="o">=</span><span class="n">imdb_df</span><span class="p">,</span> <span class="n">text_column</span><span class="o">=</span><span class="s2">&quot;review&quot;</span><span class="p">)</span>
+<span class="n">tsp</span><span class="o">.</span><span class="n">chat</span><span class="p">(</span><span class="n">api_key</span><span class="o">=</span><span class="n">credentials</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;openai_api_key&quot;</span><span class="p">))</span>
+</pre></div>
+</div>
+<p>The chat UI is available under <a class="reference external" href="http://127.0.0.1:5000/">http://127.0.0.1:5000/</a></p>
+<p><img alt="chat" src="_images/chat.png" /></p>
+</section>
+</section>
+
+
+           </div>
+          </div>
+          <footer>
+
+  <hr/>
+
+  <div role="contentinfo">
+    <p>&#169; Copyright 2023, Meghdad Farahmand.</p>
+  </div>
+
+  Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
+    <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
+    provided by <a href="https://readthedocs.org">Read the Docs</a>.
+
+
+</footer>
+        </div>
+      </div>
+    </section>
+  </div>
+  <script>
+      jQuery(function () {
+          SphinxRtdTheme.Navigation.enable(true);
+      });
+  </script> 
+
+</body>
+</html>

docs/contributing.html

@@ -71,6 +71,7 @@
 <li class="toctree-l2"><a class="reference internal" href="#setup-your-dev-environment">Setup your Dev Environment</a></li>
 <li class="toctree-l2"><a class="reference internal" href="#testing">Testing</a></li>
 <li class="toctree-l2"><a class="reference internal" href="#code-quality">Code Quality</a></li>
+<li class="toctree-l2"><a class="reference internal" href="#building-the-documentation">Building the Documentation</a></li>
 <li class="toctree-l2"><a class="reference internal" href="#pull-request-pr">Pull Request (PR)</a></li>
 </ul>
 </li>
@@ -210,6 +211,16 @@ <h2>Code Quality<a class="headerlink" href="#code-quality" title="Permalink to t
 skip pre-commit checks by running your <code class="docutils literal notranslate"><span class="pre">git</span> <span class="pre">commit</span></code> with the <code class="docutils literal notranslate"><span class="pre">--no-verify</span></code> flag (e.g. <code class="docutils literal notranslate"><span class="pre">git</span> <span class="pre">commit</span> <span class="pre">-m</span> <span class="pre">'dirty</span> <span class="pre">fix'</span> <span class="pre">--no-verify</span></code>), however,
 this is discouraged unless you really have to.</p>
 </section>
+<section id="building-the-documentation">
+<h2>Building the Documentation<a class="headerlink" href="#building-the-documentation" title="Permalink to this heading"></a></h2>
+<p>We use <a class="reference external" href="https://www.sphinx-doc.org/">Sphinx</a> to build the documentation.</p>
+<p>When you create new functions, write new docstring, or change the existing one, or in general change the documentation, you need to build the documentation again. To do so, run the following command:</p>
+<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>sphinx-build<span class="w"> </span>-b<span class="w"> </span>html<span class="w"> </span>sphinx-docs/source<span class="w"> </span>docs/
+</pre></div>
+</div>
+<p>Also, ensure that the file <cite>.nojekyll</cite> always  exists in the docs directory otherwise, the docs will not render correctly on GitHub Pages.</p>
+<p>This will build the documentation in the <a class="reference external" href="docs/">docs</a> directory. You can now commit and push your changes to remote.</p>
+</section>
 <section id="pull-request-pr">
 <h2>Pull Request (PR)<a class="headerlink" href="#pull-request-pr" title="Permalink to this heading"></a></h2>
 <p>Once your work is complete, you can make a pull request. Remember to