Skip to content

Instructions for testers

petermr edited this page Apr 7, 2021 · 16 revisions

Alpha test of pygetpapers

This script is for alpha-testers. Please make sure you record which section/s you tested and the date. If you feel that any tests reveal serious errors (e.g. Stack traces) please raise an issue and report that you have done so.

Always record your actions and results

current environment

Operating system

details including version

Python installation

details including version

Environment variables

details of any relevant to pygetpapers

installation

###download software

URL of source

Proof of installation

simple commandline with help output.

installation smoke tests

@Ayush provide simple tests with known outcomes so everyone runs the same.

@Testers. Always record the date as yyyy-mm-dd as EPMC will change.

This search should give 5 hits - maybe one or two more

pygetpapers -q covid20 -n 

gave the output

*cut and paste output here*

The rest are much too general

query tests

`Make a simple query. Use the exisiting documentation to guide your way.

single word query

use the docs to create and submit a query for "tulsi" without download. Record the date, your input and output

multiword query

repeat for "Ocimum sanctum"

download tests

download 50 papers for "tulsi" record what you did, the terminal output, and the disk output

... and so on, testing each feature of pygetpapers one by one...


Tests for each function.

flags

-k

result of test.

-l

result of test

and all other flags, etc. ...


comparison with getpapers

Please run the same query on getpapers and pygetpapers and see if the results are identical. Probably a good idea to use a query that returns a very small number of hits.

compatibility with ami

test that the output of pygetpapers can be directly read into ami. The name of the ami project should be the same as --outdir. No renaming should be necessary. It should be possible to run:

pygetpapers -q foo -o bar -x -k 10
ami -p bar search --dict country

stress testing

Attempt to break the system. Examples could be:

  • queries which return nothing
  • queries which would dump the whole database
  • queries with complex syntax and quoting
  • non-existing output directory or pre-existing file
  • characters outside ANSI 32-127 (e.g. Hindi)
  • queries with characters needing to be escaped