Merge pull request #41 from neutrons/numpy_store

Deprecate Pandas, Matplotlib, and Python 2 support

Pipfile

@@ -9,12 +9,10 @@ ipython = "*"
 flake8 = "*"
 autopep8 = "*"
 tox = "*"
+notebook = "*"
 
 [packages]
-matplotlib = "*"
 numpy = "*"
-pandas = "== 0.24.2"
-six = "*"
 
 [requires]
 python_version = "3.6"

README.md

@@ -7,23 +7,28 @@ Total Scattering Function Manipulator:
 | [![codecov](https://codecov.io/gh/neutrons/pystog/branch/master/graph/badge.svg)](https://codecov.io/gh/neutrons/pystog) | [![Anaconda-Server Badge](https://anaconda.org/neutrons/pystog/badges/version.svg)](https://anaconda.org/neutrons/pystog)| [![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)  |
 |[![Documentation Status](https://readthedocs.org/projects/pystog/badge/?version=latest)](https://pystog.readthedocs.io/en/latest/?badge=latest) | [![Project Status: Active – The project has reached a stable, usable state and is being actively developed.](https://www.repostatus.org/badges/latest/active.svg)](https://www.repostatus.org/#active) |  |
 
-From total scattering functions, we have reciprocal-space structure factors and real-space pair distribution functions that are related via a Fourier transform. PyStoG is a package that allows for:
+From total scattering functions, we have reciprocal-space structure factors and real-space pair distribution functions that are related via a Fourier transform.
+PyStoG is a package that allows for:
 1. Converting between the various functions used by different "communities" (ie researchers who study crystalline versus amorphous or glass materials). Conversions are for either real-space or reciprocal-space.
 2. Perform the transform between the different available functions of choice
 3. Fourier filter to remove spurious artificats in the data (ie aphysical, sub-angstrom low-r peaks in G(r) from experiments)
 
-![alt text](https://raw.githubusercontent.com/marshallmcdonnell/mantid_total_scattering/master/images/sofq_to_gofr.png)
+![alt text](https://raw.githubusercontent.com/neutrons/pystog/master/images/sofq_to_gofr.png)
 
 
-The name **PyStoG** comes from the fact that this is a _Pythonized_ version of **StoG**, a ~30 year old Fortran program that is part of the [RMCProfile software suite](http://www.rmcprofile.org/Main_Page). **StoG** means **"S(Q) to G(r)"** for the fact that it takes recirpocal-space S(Q) patterns from files and transforms them into a single G(r) pattern. The original *StoG* program has been developed, in reverse chronological order, by:
+The name **PyStoG** comes from the fact that this is a _Pythonized_ version of **StoG**, a ~30 year old Fortran program that is part of the [RMCProfile software suite](http://www.rmcprofile.org/Main_Page).
+**StoG** means **"S(Q) to G(r)"** for the fact that it takes recirpocal-space S(Q) patterns from files and transforms them into a single G(r) pattern.
+The original *StoG* program has been developed, in reverse chronological order, by:
 
  * Matthew Tucker and Martin Dove (~2009)
  * Spencer Howells (~1989)
  * Jack Carpenter (prior to 1989)
 
  A current state of the **StoG** program is kept in the `fortran` directory of this package.
 
-This project was initially just a "sandbox" for taking the capabilities of **StoG** and migrating them over to the [Mantid Framework](https://github.com/mantidproject/mantid). Yet, with more and more use cases, **PyStoG** was further developed as the stand-alone project it is now. Yet, migration to the Mantid Framework is still a goal since it feeds into the [ADDIE project](https://github.com/neutrons/addie)
+This project was initially just a "sandbox" for taking the capabilities of **StoG** and migrating them over to the [Mantid Framework](https://github.com/mantidproject/mantid).
+Yet, with more and more use cases, **PyStoG** was further developed as the stand-alone project it is now.
+Yet, migration to the Mantid Framework is still a goal since it feeds into the [ADDIE project](https://github.com/neutrons/addie)
 
 ## Installation
 
@@ -45,7 +50,8 @@ source /path/to/ENV/bin/activate
 pip install pystog
 ```
 
-Also, [`direnv`](https://github.com/direnv/direnv) is a useful and recommended way to manage the virtual environment. You can simply use an `.envrc` file which contains:
+Also, [`direnv`](https://github.com/direnv/direnv) is a useful and recommended way to manage the virtual environment.
+You can simply use an `.envrc` file which contains:
 
 `layout python3`
 
@@ -64,19 +70,20 @@ from pystog import StoG
 ```
 ** WARNING: Testing of the CLI is still ongoing**
 
-Also, there is a beta-version of a python script in the package that can be run on JSON input files and operates similarly to the original **StoG** program, only with extra `matplotlib` visualization of the output. This is `python cli` and can be used as follows:
+Also, there is a beta-version of a python script in the package that can be run on JSON input files and operates similarly to the original **StoG** program.
+This is `python cli` and can be used as follows:
 
 ```bash
 pystog_cli --json <input json>
 ```
-An example JSON can be found [here](https://github.com/marshallmcdonnell/pystog/blob/master/data/examples/argon_pystog.json)
+An example JSON can be found [here](https://github.com/neutrons/pystog/blob/master/data/examples/argon_pystog.json)
 
 ### Documentation
 The official documentation is hosted on readthedocs.org: [https://pystog.readthedocs.io/en/latest/](https://pystog.readthedocs.io/en/latest/)
 
 Also, a useful example reference is the [PDFFourierTransform](http://docs.mantidproject.org/nightly/algorithms/PDFFourierTransform-v1.html) algorithm in the Mantid Framework that has similar yet limited capabilities.
 
-Finally, tutorials in the form of Jupyter Notebooks can be launched via Binder by clicking the badge here [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/marshallmcdonnell/pystog/master?filepath=tutorials) or at the top of the page.
+Finally, tutorials in the form of Jupyter Notebooks can be launched via Binder by clicking the badge here [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/neutrons/pystog/master?filepath=tutorials) or at the top of the page.
 
 ## Running the tests
 From the parent directory of the module, run:

conda.recipe/meta.yaml

@@ -14,13 +14,9 @@ build:
 
 requirements:
   build:
-    - matplotlib
     - numpy
-    - pandas
   run:
-    - matplotlib
     - numpy
-    - pandas
 
 test:
   imports:

data/examples/argon_pystog.json

@@ -42,7 +42,6 @@
     "<b_coh>^2" : 3.644,
     "<b_tot^2>" : 5.435,
     "LorchFlag" : false,
-    "PlotFlag" : true,
     "Outputs" : { "StemName" : "merged_argon" }
 }
 

data/examples/input.json

@@ -25,7 +25,6 @@
     "FourierFilter" : { "Cutoff" : 1.5 },
     "<b_coh>^2" : 5.0,
     "LorchFlag" : "True",
-    "PlotFlag" : "True",
     "Outputs" : { "StemName" : "merged" }
 }
 

docs/source/about.rst

@@ -1,26 +1,34 @@
 =====
 About
 =====
-From total scattering functions, we have reciprocal-space structure factors and real-space pair distribution functions that are related via a Fourier transform. PyStoG is a package that allows for:
+From total scattering functions, we have reciprocal-space structure factors and real-space pair distribution functions that are related via a Fourier transform.
+PyStoG is a package that allows for:
 
-1. Converting between the various functions used by different "communities" (ie researchers who study crystalline versus amorphous or glass materials). Conversions are for either real-space or reciprocal-space.
+1. Converting between the various functions used by different "communities" (ie researchers who study crystalline versus amorphous or glass materials).
+   Conversions are for either real-space or reciprocal-space.
 2. Perform the transform between the different available functions of choice
 3. Fourier filter to remove spurious artificats in the data (ie aphysical, sub-angstrom low-r peaks in G(r) from experiments)
 
 .. image:: ../../images/sofq_to_gofr.png
 
-The name **PyStoG** comes from the fact that this is a *Pythonized* version of **StoG**, a ~30 year old Fortran program that is part of the RMCProfile_ software suite. **StoG** means **"S(Q) to G(r)"** for the fact that it takes recirpocal-space S(Q) patterns from files and transforms them into a single G(r) pattern. The original *StoG* program has been developed, in reverse chronological order, by:
+The name **PyStoG** comes from the fact that this is a *Pythonized* version of **StoG**, a ~30 year old Fortran program that is part of the RMCProfile_ software suite.
+**StoG** means **"S(Q) to G(r)"** for the fact that it takes recirpocal-space S(Q) patterns from files and transforms them into a single G(r) pattern.
+The original *StoG* program has been developed, in reverse chronological order, by:
 
  * Matthew Tucker and Martin Dove (~2009)
  * Spencer Howells (~1989)
  * Jack Carpenter (prior to 1989)
 
  A current state of the **StoG** program is kept in the `fortran` directory of this package.
 
-This project was initially just a "sandbox" for taking the capabilities of **StoG** and migrating them over to the Mantid_ Framework. With more and more use cases, **PyStoG** was further developed as the stand-alone project it is now. Yet, migration to the Mantid Framework is still a goal since it feeds into the ADDIE_ project.
+This project was initially just a "sandbox" for taking the capabilities of **StoG** and migrating them over to the Mantid_ Framework.
+With more and more use cases, **PyStoG** was further developed as the stand-alone project it is now.
+Yet, migration to the Mantid Framework is still a goal since it feeds into the ADDIE_ project.
 
 
-PyStoG is not a Python replica of StoG but has the same goal as StoG. Yet, has capabilites that surpass StoG such as multiple input/output real and reciprocal space functions, diagnostic plotting with matplotlib, and modules allowing for re-construction of the workflow for processing total scattering data.
+PyStoG is not a Python replica of StoG but has the same goal as StoG.
+Yet, has capabilites that surpass StoG such as multiple input/output real
+and reciprocal space functions and modules allowing for re-construction of the workflow for processing total scattering data.
 
 .. _RMCProfile: http://www.rmcprofile.org/Main_Page
 .. _Mantid: https://github.com/mantidproject/mantid

docs/source/install.rst

@@ -5,15 +5,11 @@ Installation
 Requirements
 ============
 
-* Python (2.7, 3.5, 3.6 are tested)
-* matplotlib_
+* Python (3.6, 3.7, 3.8, and 3.9 are tested)
 * numpy_
-* pandas_
 
 
-.. _matplotlib: https://matplotlib.org/
 .. _numpy: http://www.numpy.org/
-.. _pandas: http://pandas.pydata.org/
 
 
 

pystog/cli.py

@@ -1,11 +1,17 @@
-from __future__ import (absolute_import, division, print_function)
-import argparse
-import inspect
+"""
+==============
+PyStoG CLI
+==============
+
+This module defines the PyStoG CLI function that performs the workflow of the
+original FORTRAN-based StoG CLI
+"""
+
+
 import json
 
-from pystog import Converter, Transformer, StoG
+from pystog import StoG
 from pystog.io import get_cli_args, parse_cli_args
-from pystog.utils import get_data, create_domain, write_out
 
 
 def pystog_cli(kwargs=None):
@@ -25,28 +31,18 @@ def pystog_cli(kwargs=None):
     stog.merge_data()
     stog.write_out_merged_sq()
 
-    if "PlotFlag" in kwargs:
-        if kwargs["PlotFlag"]:
-            stog.plot_merged_sq()
-
     # Initial S(Q) -> g(r) transform
     stog.transform_merged()
     stog.write_out_merged_gr()
 
-    if "PlotFlag" in kwargs:
-        if kwargs["PlotFlag"]:
-            stog.plot_gr(
-                ylabel=stog.real_space_function,
-                title="Merged %s" % stog.real_space_function)
-
     # TODO: Add the lowR minimizer here
     # print stog.get_lowR_mean_square()
 
     # Set the S(Q) and g(r) if no Fourier Filter
-    r = stog.df_gr_master[stog.gr_title].values
-    q = stog.df_sq_master[stog.sq_title].values
-    sq = stog.df_sq_master[stog.sq_title].values
-    gr_out = stog.df_gr_master[stog.gr_title].values
+    r = stog.gr_master[stog.gr_title].values
+    q = stog.sq_master[stog.sq_title].values
+    sq = stog.sq_master[stog.sq_title].values
+    gr_out = stog.gr_master[stog.gr_title].values
 
     # Apply Fourier Filter
     if "FourierFilter" in kwargs:
@@ -59,136 +55,3 @@ def pystog_cli(kwargs=None):
     # Apply final scale number
     stog._add_keen_fq(q, sq)
     stog._add_keen_gr(r, gr_out)
-
-    if "PlotFlag" in kwargs:
-        if kwargs["PlotFlag"]:
-            stog.plot_summary_sq()
-            stog.plot_summary_gr()
-
-
-def transformer_cli():
-    # -------------------------------------#
-    # Converter / Transform Factory
-    tf = Transformer()
-    transform_functions = inspect.getmembers(tf, predicate=inspect.ismethod)
-    transform_dict = {entry[0]: entry[1] for entry in transform_functions}
-    transform_dict.pop('fourier_transform')
-    transform_dict.pop('_extend_axis_to_low_end')
-    transform_dict.pop('_low_x_correction')
-    transform_dict.pop('__init__')
-
-    cv = Converter()
-    converter_functions = inspect.getmembers(cv, predicate=inspect.ismethod)
-    converter_dict = {entry[0]: entry[1] for entry in converter_functions}
-    converter_dict.pop('__init__')
-
-    choices = transform_dict.copy()
-    choices.update(converter_dict)
-
-    def TransformationFactory(transform_name):
-        if transform_name in transform_dict:
-            return transform_dict[transform_name]
-        elif transform_name in converter_dict:
-            return converter_dict[transform_name]
-
-    # -------------------------------------#
-    # Main function CLI
-
-        parser = argparse.ArgumentParser()
-        parser.add_argument(
-            'transformation',
-            choices=choices.keys(),
-            help="Fourier transform G(r) -> S(Q)")
-        parser.add_argument(
-            '-x',
-            '--domain_range',
-            nargs=3,
-            default=(0.0, 100.0, 0.05),
-            type=float,
-            help="The domain (xmin, xmax, binsize) for transformed function")
-        parser.add_argument('-i', '--input', type=str, help='Input Filename')
-        parser.add_argument('-o', '--output', type=str, help='Output Filename')
-        parser.add_argument(
-            '-s',
-            '--skiprows',
-            type=int,
-            default=2,
-            help='Number of rows to skip in datasets')
-        parser.add_argument(
-            '-t',
-            '--trim',
-            type=int,
-            default=0,
-            help='Number of rows to trim off end in datasets')
-        parser.add_argument(
-            '--xcol', type=int, default=0, help='Set x-col for filename')
-        parser.add_argument(
-            '--ycol', type=int, default=1, help='Set y-col for filename')
-        parser.add_argument(
-            '--bcoh-sqrd',
-            type=float,
-            default=None,
-            dest='bcoh_sqrd',
-            help='Squared mean coherent scattering length (units=fm^2)')
-        parser.add_argument(
-            '--btot-sqrd',
-            type=float,
-            default=None,
-            dest='btot_sqrd',
-            help='Mean squared total scattering length (units=fm^2)')
-        parser.add_argument(
-            '--rho',
-            type=float,
-            default=None,
-            dest='rho',
-            help='Number density (units=atoms/angstroms^3)')
-        parser.add_argument(
-            '--plot',
-            action='store_true',
-            help='Show plot of before and after transformation')
-        parser.add_argument(
-            '--lorch',
-            action='store_true',
-            default=False,
-            help='Apply Lorch Modifcation')
-
-        args = parser.parse_args()
-
-        # Read in data
-        data = get_data(
-            args.input,
-            skiprows=args.skiprows,
-            skipfooter=args.trim,
-            xcol=args.xcol,
-            ycol=args.ycol)
-        # Setup domain
-        xmin, xmax, binsize = args.domain_range
-        xnew = create_domain(xmin, xmax, binsize)
-
-        # Add extra  key-word arguments needed for some conversions
-        kwargs = dict()
-        if args.bcoh_sqrd:
-            kwargs['<b_coh>^2'] = args.bcoh_sqrd
-        if args.rho:
-            kwargs['rho'] = args.rho
-        if args.btot_sqrd:
-            kwargs['<b_tot^2>'] = args.btot_sqrd
-        kwargs['lorch'] = args.lorch
-
-        # Transform data to new form
-        tf = TransformationFactory(args.transformation)
-        if args.transformation in transform_dict:
-            xnew, ynew = tf(data['x'].values, data['y'].values, xnew, **kwargs)
-        elif args.transformation in converter_dict:
-            xnew = data['x'].values
-            ynew = tf(data['x'].values, data['y'].values, **kwargs)
-
-        # Output file
-        write_out(args.output, xnew, ynew, title=args.transformation)
-
-        # Plot old and new data
-        if args.plot:
-            import matplotlib.pyplot as plt
-            plt.plot(data['x'].values, data['y'].values)
-            plt.plot(xnew, ynew)
-            plt.show()

pystog/converter.py

@@ -7,7 +7,6 @@
 that converts functions in the same space
 """
 
-from __future__ import (absolute_import, division, print_function)
 import numpy as np
 
 

pystog/fourier_filter.py

@@ -8,9 +8,6 @@
 given range to exclude.
 """
 
-
-from __future__ import (absolute_import, division, print_function)
-
 import numpy as np
 
 from pystog.converter import Converter

pystog/io.py

@@ -59,8 +59,6 @@ def get_cli_args():
         default=1.0,
         dest="btot_sqrd",
         help="The (sum c*b^2) term needed for DCS(Q) input")
-    parser.add_argument("--plot", action="store_true", default=False,
-                        help="Plots using matplotlib along the way")
     parser.add_argument("--merging", nargs=2, type=float, default=[0.0, 1.0],
                         help="Offset and Scale to apply to the merged S(Q)")
     parser.add_argument("--low-q-correction", action='store_true',
@@ -90,7 +88,6 @@ def parse_cli_args(args):
         "FourierFilter": {
             "Cutoff": args.fourier_filter_cutoff},
         "LorchFlag": args.lorch_flag,
-        "PlotFlag": args.plot,
         "Outputs": {
             "StemName": args.stem_name},
         "Merging": {