#106 Early version of test (required columns, json)

[Ciheim]

This an early version of our testing infrastructure. This version looks for incorrect json modifications and empty values within required columns.

[Ciheim]

The script takes two arguments: (1) The path to the JSON that was generated by the catalog builder and (2) The path to the JSON template that was used for generation.

It is called using this format: test.py custom_json_path json_template_path

The script compares the json and returns any json sections that have changes (besides the catalog path section which we expect to change).

The script also grabs all required columns, as defined by the groupby_attrs section, and checks for any null values within them.

Here's an example I ran: test.py ~/output.json ../cats/gfdl_template.json

And here's the output:

attributes section of JSON does not refect template
source_id contains empty values
member_id contains empty values

[ceblanton]

The test is failing on from jsondiff import diff.

meta.yaml needs an update to install jsondiff.

[Ciheim]

The test is failing on from jsondiff import diff.

meta.yaml needs an update to install jsondiff.

I made the change to environment.yml in #125 but you're right. I forgot to add it to meta.yaml. I'll remove the jsondiff change there and add both changes to this PR. Nice Catch.

[ceblanton]

what is environment.yaml used for? Almost too many yamls in the world (just kidding..)

[Ciheim]

what is environment.yaml used for? Almost too many yamls in the world (just kidding..)

We expect users who want to run the tool directly to create an environment from this yaml

[ceblanton]

Hello, I tested this and think it is good and ready. I rephrased the error messages slightly and rephrased the try/except logic.

One thing I have mixed feelings about is that it gives the bad news in a drip-drip way rather than giving all the bad news at once. I think it's ok though as the problems are likely to be repeated. It would probably be worse to give the same error 1000x times for a 1000 line catalog.

[Ciheim]

Hello, I tested this and think it is good and ready. I rephrased the error messages slightly and rephrased the try/except logic.

One thing I have mixed feelings about is that it gives the bad news in a drip-drip way rather than giving all the bad news at once. I think it's ok though as the problems are likely to be repeated. It would probably be worse to give the same error 1000x times for a 1000 line catalog.

Yes you're right about this. Thanks for the changes. Bennett and I noticed it so I put in a fix but the commit got a little messed up. Git errors bleh... I'll take care of this soon.