diff --git a/Project.toml b/Project.toml index 52a6f01..de819b7 100644 --- a/Project.toml +++ b/Project.toml @@ -1,7 +1,7 @@ name = "REDCap" uuid = "ba918724-fbf9-5e4a-a61c-87e95654e718" authors = ["Cory Cothrum", "Dilum Aluthge ", "Ashlin Harris "] -version = "2.1.0" +version = "2.2.0" [deps] Dates = "ade2ca70-3891-5945-98fb-dc099432e06a" diff --git a/README.md b/README.md index 6fb367e..b1eb92d 100644 --- a/README.md +++ b/README.md @@ -7,6 +7,8 @@ A Julia frontend for the REDCap API +[REDCap](https://en.wikipedia.org/wiki/REDCap) is a data capture system for scientific research, especially clinical trials. + ## Example ```julia using REDCap @@ -14,8 +16,8 @@ using REDCap export_version() project_token = create_project( - data=Dict(:project_title => "Test Project",:purpose => 0), - odm="Data_Dictionary.xml") + data = (project_title = "Test Project", purpose = 0), + odm = "Data_Dictionary.xml") import_records(token=project_token, data="example.csv", format=:csv) @@ -26,22 +28,96 @@ export_logging(token=project_token) More examples can be found in the [documentation](https://docs.bcbi.brown.edu/REDCap.jl/latest/examples/). ## Syntax +Each REDCap method accepts a number of parameters that follow a shared naming convention. +Generally, a parameter of a given name shares a similar role in all methods where it can be used. +Parameters can hold various datatypes and might even be composed of multiple named attributes. + +REDCap.jl is designed to closely follow the design and syntax patterns of REDCap. Every REDCap API method is available as a function that supplies certain required parameters and checks user inputs for validity. -Type and coherency checks are quite strict, which prevents certain user errors that can be difficult to diagnose with the REDCap's error messages. +Return values and REDCap messages are returned as Strings directly, but the documentation shows how these can be parsed in useful ways. -Function arguments are named after RECap method parameters. +Function arguments are named after REDCap method parameters. These are passed as named arguments and take values with intuitive types, with a few exceptions to note: ### Token and URL +Almost all REDCap methods accept a token that is unique to the project and user. +A super token can be used to generate a project and project-level token. +The URL must exactly match this example: +```https://example.example/redcap/api/``` + Your REDCap token and your institution's REDCap API URL can be read by default from Julia's environment variables. You can make them avaiable to REDCap.jl by putting the following lines in [your local Julia startup file](https://docs.julialang.org/en/v1/manual/command-line-interface/#Startup-file) (probably `~/.julia/config/startup.jl`): ```julia ENV["REDCAP_API_TOKEN"] = "C0FFEEC0AC0AC0DEC0FFEEC0AC0AC0DE" ENV["REDCAP_API_URL"] = "http://example.com/redcap/api/" ``` +They can also be passed as ordinary arguments. +If you have a super token, you might wish to keep that in your startup file, generating and saving project-level tokens as needed. + +### `data` +The `data` parameter accepts a collection (Dict, NamedTuple, etc.) or a String. +If you use a a collection, it will be translated internally into whatever `format` you use (xml by default). +A NamedTuple is the most elegant format: +```julia +import_project_info( + data=( + project_title="New name", + project_notes="New notes" + ), + returnFormat=:csv, +) +``` +But please keep in mind that a NamedTuple must contain at least one comma: +```julia +import_project_info( + data=( + project_title="New name", # this comma is required + ), + returnFormat=:csv, +) +``` +A `Dict` value is fine as well. +```julia +import_project_info(data=Dict(:project_title=>"New name"), returnFormat=:csv) +``` +String values are parsed - if they end with a .csv, .json, or .xml file extension, they are treated as a file name; otherwise, they are assumed to be a formatted string and are sent directly as part of the API request. +```julia +data_string = """ + [{"data_access_group_name":"CA Site","unique_group_name":"ca_site"}, + {"data_access_group_name":"FL Site","unique_group_name":"fl_site"}, + {"data_access_group_name":"New Site","unique_group_name":""}] +""" +out = open("data_file.json","w"); write(out, data_string); close(out) + +import_DAGs(token=t,data=data_string, format=:json) # string is passed to the API + +import_DAGs(token=t, data="data_file.json", format=:json) # string is pattern-matched as a filename + +``` +As for collections, only collections of scalar entries are currently supported. +So, a list of attributes and values is accepted, but a Dict containing multiple rows per column can only be read in from a file. + +In the REDCap API, the presence of a `data` parameter often changes the behavior of a method. +For instance, most import methods are implemented as an export method with an added data parameter. +In REDCap.jl, it would be considered a bug for `import_project_data` to ever act as `export_project_data`, so the data paramater is almost always required where it is present. + +### `format` and `returnFormat` +Supported options are `:csv`, `:json`, `:xml` (the default value), and sometimes `:odm`. +These values can be passed as Strings or Symbols. + +Generally, the `format` parameter designates user input and the `returnFormat` parameter applies to REDCap messages and return values. +However, this is not consistent within REDCap. +REDCap.jl functions are designed to not accept any parameters that have no effect on the result. + +## `content` and `action` +The `content` and `action` parameters are what define each REDCap method, for the most part. +In REDCap.jl, these are passed internally and don't need to be supplied by the user. +Instead, they're fixed for each function. + +### Troubleshooting -### Data -The data parameter accepts either a filename, or a Julia `Dict`. +If a function call doesn't produce the expected results, try making debug messages visible for this package by running `ENV["JULIA_DEBUG"] = REDCap`. +Feel free to create an issue for any unexpected errors, or for feature requests. ## Acknowledgments The contributors are grateful for the support of Mary McGrath, Paul Stey, Fernando Gelin, the Brown Data Science Institute, the Brown Center for Biomedical Informatics, and the Tufts CTSI Informatics core. diff --git a/docs/src/examples.md b/docs/src/examples.md index ae9e0d7..c7a75c3 100644 --- a/docs/src/examples.md +++ b/docs/src/examples.md @@ -11,16 +11,9 @@ export_metadata() export_logging(format=:json) |> JSON.parse |> DataFrame # Add ability to delete records -import_users( - data=Dict( - :username => "userName", - :record_delete => 1 - ) -) +import_users(data=(username = "userName",record_delete = 1)) -delete_records( - records=["TM22-15374","TM22-16931","TM22-21015"] -) +delete_records(records=["TM22-15374","TM22-16931","TM22-21015"]) ``` The following example is a more realistic workflow that creates a new project from an existing project XML (data dictionary) and uploads large CSV data files into it in segments. diff --git a/src/api_methods/arms.jl b/src/api_methods/arms.jl index 05d4064..fe70bbb 100644 --- a/src/api_methods/arms.jl +++ b/src/api_methods/arms.jl @@ -35,6 +35,8 @@ function export_arms(; ) end +#All examples use JSON +#TODO: what is the proper format for multi-item XML? I can't find this anywhere... function import_arms(; url::redcap_url_input=get_url(), token::redcap_token_input=get_token(), @@ -51,7 +53,7 @@ function import_arms(; override=override, action=REDCap_action(:import), format=REDCap_format(format), - data=REDCap_data(data,REDCap_format(format)), + data=REDCap_data(data,REDCap_format(format),xml_tag="arms"), returnFormat=REDCap_format(returnFormat), ) end diff --git a/src/api_methods/data_access_groups.jl b/src/api_methods/data_access_groups.jl index ee75142..7e49c6e 100644 --- a/src/api_methods/data_access_groups.jl +++ b/src/api_methods/data_access_groups.jl @@ -31,7 +31,7 @@ function export_DAGs(; token=REDCap_token(token), content=REDCap_content(:dag), format=REDCap_format(format), - returnFormat=REDCap_format(returnFormat), + #returnFormat=REDCap_format(returnFormat), ) end @@ -63,7 +63,7 @@ function import_DAGs(; content=REDCap_content(:dag), action=REDCap_action(:import), format=REDCap_format(format), - data=REDCap_data(data,REDCap_format(format)), + data=REDCap_data(data,REDCap_format(format),xml_tag="dags"), returnFormat=REDCap_format(returnFormat), ) end @@ -81,7 +81,7 @@ function import_user_DAG_assignment(; token=REDCap_token(token), content=REDCap_content(:userDagMapping), format=REDCap_format(format), - data=REDCap_data(data,REDCap_format(format)), + data=REDCap_data(data,REDCap_format(format),xml_tag="items"), returnFormat=REDCap_format(returnFormat), ) end diff --git a/src/api_methods/events.jl b/src/api_methods/events.jl index 9bbfa4e..66e5818 100644 --- a/src/api_methods/events.jl +++ b/src/api_methods/events.jl @@ -52,7 +52,7 @@ function import_events(; content=REDCap_content(:event), action=REDCap_action(:import), override=override, - data=REDCap_data(data,REDCap_format(format)), + data=REDCap_data(data,REDCap_format(format),xml_tag="events"), ) end diff --git a/src/api_methods/instruments.jl b/src/api_methods/instruments.jl index 04686d7..47d9e9a 100644 --- a/src/api_methods/instruments.jl +++ b/src/api_methods/instruments.jl @@ -75,7 +75,7 @@ function import_instrument_event_mappings(; token=REDCap_token(token), content=REDCap_content(:formEventMapping), format=REDCap_format(format), - data=REDCap_data(data,REDCap_format(format)), + data=REDCap_data(data,REDCap_format(format),xml_tag="items"), returnFormat=REDCap_format(returnFormat), ) end diff --git a/src/api_methods/metadata.jl b/src/api_methods/metadata.jl index c1431a8..375260e 100644 --- a/src/api_methods/metadata.jl +++ b/src/api_methods/metadata.jl @@ -21,6 +21,7 @@ function export_metadata(; ) end +#TODO: there is no guidance on what the metadata should look like... is itbasically like the odm parameter in create_project? function import_metadata(; data::redcap_data_input, url::redcap_url_input=get_url(), diff --git a/src/api_methods/projects.jl b/src/api_methods/projects.jl index 2d4249f..d639706 100644 --- a/src/api_methods/projects.jl +++ b/src/api_methods/projects.jl @@ -16,7 +16,7 @@ function create_project(; content=REDCap_content(:project), format=REDCap_format(format), returnFormat=REDCap_format(returnFormat), - data=REDCap_data(data,REDCap_format(format)), + data=REDCap_data(data,REDCap_format(format)),#xml_tag="items") url=REDCap_url(url), token=REDCap_super_token(token), odm=odm, @@ -78,14 +78,16 @@ end function export_project_info(; url::redcap_url_input=get_url(), token::redcap_token_input=get_token(), - returnFormat::redcap_returnFormat_input=nothing, + format::redcap_returnFormat_input=nothing, + #returnFormat::redcap_returnFormat_input=nothing, ) REDCap.request(; content=REDCap_content(:project), url=REDCap_url(url), token=REDCap_token(token), - returnFormat=REDCap_format(returnFormat), + format=REDCap_format(format), + #returnFormat=REDCap_format(returnFormat), ) end @@ -132,7 +134,8 @@ function import_project_info(; token=REDCap_token(token), content=REDCap_content(:project_settings), format=REDCap_format(format), - data=REDCap_data(data,REDCap_format(format)), + data=REDCap_data(data,REDCap_format(format),xml_tag="items"), + #project_title, project_language, purpose, purpose_other, project_notes, custom_record_label, secondary_unique_field, is_longitudinal, surveys_enabled, scheduling_enabled, record_autonumbering_enabled, randomization_enabled, project_irb_number, project_grant_number, project_pi_firstname, project_pi_lastname, display_today_now_button, bypass_branching_erase_field_prompt ) end diff --git a/src/api_methods/records.jl b/src/api_methods/records.jl index bac9b0a..1911722 100644 --- a/src/api_methods/records.jl +++ b/src/api_methods/records.jl @@ -28,6 +28,7 @@ function delete_records(; ) end +#TODO: Note, an export function with a data parameter function export_records(; data::redcap_data_input, url::redcap_url_input=get_url(), @@ -52,7 +53,7 @@ function export_records(; exportBlankForGrayFormStatus::redcap_bool_input=nothing ) REDCap.request( - data=REDCap_data(data,REDCap_format(format)), + data=REDCap_data(data,REDCap_format(format),xml_tag="records"), url=REDCap_url(url), token=REDCap_token(token), content=REDCap_content(:record), @@ -76,6 +77,8 @@ function export_records(; ) end +#TODO: this was that functino that took a data parameter with no value in data, right? +# did that change in REDCap 14? #if data == nothing, this is an export request function generate_next_record_name( url::redcap_url_input=get_url(), @@ -83,7 +86,7 @@ function generate_next_record_name( ) REDCap.request( - data=REDCap_data(data,REDCap_format(format)), + #data=REDCap_data(data,REDCap_format(format),xml_tag="I have no clue"), url=REDCap_url(url), token=REDCap_token(token), content=REDCap_content(:generateNextRecordName), @@ -106,7 +109,7 @@ function import_records(; ) REDCap.request( - data=REDCap_data(data,REDCap_format(format)), + data=REDCap_data(data,REDCap_format(format),xml_tag="records"), url=REDCap_url(url), token=REDCap_token(token), content=REDCap_content(:record), @@ -121,6 +124,7 @@ function import_records(; ) end +#TODO: syntax may have changed between REDCap versions function rename_record(; url::redcap_url_input=get_url(), token::redcap_token_input=get_token(), @@ -130,7 +134,7 @@ function rename_record(; ) REDCap.request( - data=REDCap_data(data,REDCap_format(format)), + #data=REDCap_data(data,REDCap_format(format),xml_tag="deprecated?"), url=REDCap_url(url), token=REDCap_token(token), content=REDCap_content(:record), diff --git a/src/api_methods/user_roles.jl b/src/api_methods/user_roles.jl index e3baee8..0a9c618 100644 --- a/src/api_methods/user_roles.jl +++ b/src/api_methods/user_roles.jl @@ -49,7 +49,7 @@ function import_user_roles(; token=REDCap_token(token), content=REDCap_content(:userRole), format=REDCap_format(format), - data=REDCap_data(data,REDCap_format(format)), + data=REDCap_data(data,REDCap_format(format),xml_tag="users"), returnFormat=REDCap_format(returnFormat), ) end @@ -84,7 +84,7 @@ function import_user_role_assignments(; token=REDCap_token(token), content=REDCap_content(:userRoleMapping), format=REDCap_format(format), - data=REDCap_data(data,REDCap_format(format)), + data=REDCap_data(data,REDCap_format(format),xml_tag="items"), returnFormat=REDCap_format(returnFormat), ) end diff --git a/src/api_methods/users.jl b/src/api_methods/users.jl index 2c0961a..cebe36b 100644 --- a/src/api_methods/users.jl +++ b/src/api_methods/users.jl @@ -40,6 +40,11 @@ function import_users(; data::redcap_data_input, ) +#= +Data Export: 0=No Access, 2=De-Identified, 3=Remove Identifier Fields, 1=Full Data Set +Form Rights: 0=No Access, 2=Read Only, 1=View records/responses and edit records (survey responses are read-only), 3=Edit survey responses +Other attribute values: 0=No Access, 1=Access. +=# #= if isa(data,Dict) @assert Symbol.(keys(data)) ⊆ [:username, :expiration, :data_access_group, :design, :alerts, :user_rights, :data_access_groups, :data_export, :reports, :stats_and_charts, :manage_survey_participants, :calendar, :data_import_tool, :data_comparison_tool, :logging, :file_repository, :data_quality_create, :data_quality_execute, :api_export, :api_import, :mobile_app, :mobile_app_download_data, :record_create, :record_rename, :record_delete, :lock_records_customization, :lock_records, :lock_records_all_forms, :forms, :forms_export] @@ -53,7 +58,7 @@ function import_users(; token=REDCap_token(token), content=REDCap_content(:user), format=REDCap_format(format), - data=REDCap_data(data,REDCap_format(format)), + data=REDCap_data(data,REDCap_format(format),xml_tag="users"), returnFormat=REDCap_format(returnFormat), ) end diff --git a/src/types.jl b/src/types.jl index c9e872e..411ec37 100644 --- a/src/types.jl +++ b/src/types.jl @@ -1,10 +1,11 @@ -#TODO: Should some of these functions be moved to src/utils.jl? -#TODO: separate into modules +#TODO: recommend users try different format parameters with Dict, etc. for degbugging +#TODO: recommend turning on debug messages for debugging +#TODO: Should some of these functions be moved to src/utils.jl? DEFINITELY # Any types that users may pass to a function named after a REDCap method const redcap_token_input = String const redcap_super_token_input = redcap_token_input -const redcap_data_input = Union{String, Dict} #TODO: there might be 1 REDCap method where Dict can be nothing, but passing it as an argument has an effect +const redcap_data_input = Union{String, Tuple, NamedTuple, Dict} #TODO: there might be 1 REDCap method where Dict can be nothing, but passing it as an argument has an effect const redcap_filterLogic_input = Union{String, Nothing} const redcap_odm_input = Union{String, Nothing} const redcap_array_input = Union{Array, Nothing} @@ -63,19 +64,37 @@ Base.display(x::REDCap_format) = Base.display(x.id) Base.string(x::REDCap_format) = Base.string(x.id) Base.convert(String,x::REDCap_format) = string(x) -function REDCap_data(x::Dict, format::Union{REDCap_format, Nothing}) +#TODO: handle more complicated examples +##TODO: There's no need to translate the data - I can leave it in its Julia type +#TODO: if format is nothing, just pass the data unchanged +##TODO: account for capitalization - maybe make the internal id a string? +function REDCap_data(x::Dict, format::Union{Nothing,REDCap_format}; xml_tag=nothing) return if format == REDCap_format(:json) "[$(JSON.json(x))]" elseif format == REDCap_format(:csv) join(keys(x),',') * "\n" * join(values(x),',') - else - "" + else # default, assume XML + #TODO: this looks ugly and isn't tested. test every data paramater that take an xml_tag + #elseif format == REDCap_format(:xml) + attribute = isnothing(xml_tag) ? "" : "<$xml_tag>" + close_attribute = isnothing(xml_tag) ? "" : "" + + "" * + attribute * "" * join(["<$k>$v" for (k,v) in x]) * - "" + "" * + close_attribute end end -REDCap_data(x::String, format::REDCap_format) = read(x, String) +#TODO: someway to throw an error if there's no comma to make the args a NamedTupe? I can't think of anything +#TODO: THese methods are a mess - fix the XML bug, then loop around to here... +#REDCap_data(x::Dict, format::Nothing) = x #If there's no format tag, pass the data parameter unchanged +#REDCap_data(x::NamedTuple, format::Nothing) = x |> pairs |> Dict |> x -> "$x" +#TODO: ONly pairs is needed +REDCap_data(x::NamedTuple, format::Union{REDCap_format,Nothing}; xml_tag=nothing) = REDCap_data(x |> pairs |> Dict, format, xml_tag=xml_tag) +#TODO: Add file checking +REDCap_data(x::String, format::Union{REDCap_format, Nothing}; xml_tag=nothing) = x struct REDCap_token id::redcap_token_input @@ -94,9 +113,12 @@ Base.string(x::REDCap_super_token) = string(x.id) Base.convert(String,x::REDCap_super_token) = string(x) redcap_generic_parameter = Union{ + REDCap_action, REDCap_format, DateTime, String, + Integer, + Vector, Nothing, } diff --git a/src/utils.jl b/src/utils.jl index f04430b..d910ffa 100644 --- a/src/utils.jl +++ b/src/utils.jl @@ -5,6 +5,7 @@ function request(; token::Union{REDCap_token, REDCap_super_token}, content::REDCap_content, data=nothing, + odm=nothing, kwargs...) #TODO: replace interpolation with string()? @@ -12,11 +13,22 @@ function request(; html_request_body["token"] = token html_request_body["content"] = "$content" #TODO: Can't the data parameter be nothing and still hav an effect in at least 1 function? - if !isnothing(data); html_request_body["data"] = "$data" end + + if !isnothing(odm) + html_request_body["odm"] = read(odm,String) + end + #TODO: Add chunking (but what is bakgroundProcess=true, new to REDCap 14?) + #Also, different formats have to be chunked differently. + #Maybe stick to creating an iterator outside REDCap... + if !isnothing(data) + if endswith.(data,[".csv",".json",".xml"]) |> any + html_request_body["data"] = read(data,String) + else + html_request_body["data"] = "$data" + end + end - #TODO: make this clear to users - #ENV["JULIA_DEBUG"] = REDCap @debug(filter(x->(first(x)!="token"), html_request_body)) response = HTTP.post( diff --git a/test/api_methods/arms.jl b/test/api_methods/arms.jl deleted file mode 100644 index 8b13789..0000000 --- a/test/api_methods/arms.jl +++ /dev/null @@ -1 +0,0 @@ - diff --git a/test/api_methods/data_access_groups.jl b/test/api_methods/data_access_groups.jl deleted file mode 100644 index 8b13789..0000000 --- a/test/api_methods/data_access_groups.jl +++ /dev/null @@ -1 +0,0 @@ - diff --git a/test/api_methods/logging.jl b/test/api_methods/logging.jl deleted file mode 100644 index 8b13789..0000000 --- a/test/api_methods/logging.jl +++ /dev/null @@ -1 +0,0 @@ - diff --git a/test/api_methods/projects.jl b/test/api_methods/projects.jl deleted file mode 100644 index 86dcdad..0000000 --- a/test/api_methods/projects.jl +++ /dev/null @@ -1,15 +0,0 @@ -@test_throws UndefKeywordError create_project() - -@test create_project(data = Dict(:project_title => "AAA", "purpose" => 1),format=:csv) == -"ERROR: You must provide some text for 'purpose_other' since you specified 'purpose' as '1' (Other)." - -@test create_project(data = Dict(:project_title => "AAA", "purpose" => 1),format=:json) == -"{\"error\":\"You must provide some text for 'purpose_other' since you specified 'purpose' as '1' (Other).\"}" - -@test create_project(data = Dict(:project_title => "AAA", "purpose" => 1),format=:xml) == -"You must provide some text for 'purpose_other' since you specified 'purpose' as '1' (Other)." - -@test create_project(data = Dict(:project_title => "AAA", "purpose" => 1)) == -"You must provide some text for 'purpose_other' since you specified 'purpose' as '1' (Other)." - - diff --git a/test/api_methods/redcap.jl b/test/api_methods/redcap.jl deleted file mode 100644 index 8b13789..0000000 --- a/test/api_methods/redcap.jl +++ /dev/null @@ -1 +0,0 @@ - diff --git a/test/api_methods/users.jl b/test/api_methods/users.jl deleted file mode 100644 index 8b13789..0000000 --- a/test/api_methods/users.jl +++ /dev/null @@ -1 +0,0 @@ - diff --git a/test/runtests.jl b/test/runtests.jl index 0cc3f71..7894b67 100644 --- a/test/runtests.jl +++ b/test/runtests.jl @@ -4,35 +4,37 @@ using Test using Dates using DataFrames -if (get(ENV, "REDCAP_API_URL", "")) |> isempty && (get(ENV, "REDCAP_API_TOKEN", "")) |> isempty - @info("To run these tests, add values for REDCAP_API_URL and REDCAP_API_TOKEN to your Julia startup file") - @test true -else +#TODO: I have to rewrite these tests... +# I can't just use a tempname, since the file handle gets parsed... +# Maybe just add a method definition for file handles? +# Is it bad practice to parse strings that way? +#TODO: ensure tests are run from a certain directory? + +function run_all_tests() + +list_of_test_files = [] +function all_subfiles(dir) + contents = readdir(dir, join=true) + for file in contents[@. !isdir(contents)] + if endswith(file, ".jl"); push!(list_of_test_files, file) end + end + for subdirectory in contents[@. isdir(contents)] + all_subfiles(subdirectory) + end +end +all_subfiles("test_files") + +@testset verbose=true "Full testset" begin + for test_file in list_of_test_files + @testset "$test_file" begin + begin + global file_name, file_handle + include(test_file) + end + end + end +end -include("api_methods/arms.jl") -include("api_methods/data_access_groups.jl") -include("api_methods/events.jl") -include("api_methods/field_names.jl") -include("api_methods/file_repository.jl") -include("api_methods/files.jl") -include("api_methods/instruments.jl") -include("api_methods/logging.jl") -include("api_methods/metadata.jl") -include("api_methods/projects.jl") -include("api_methods/records.jl") -include("api_methods/redcap.jl") -include("api_methods/repeating_instruments_and_events.jl") -include("api_methods/reports.jl") -include("api_methods/surveys.jl") -include("api_methods/user_roles.jl") -include("api_methods/users.jl") - - - -@test export_version() == "14.5.8" -#TODO: account for running test without token in E - -#TODO: more tests like this, checking the API's return value #= write("users.csv", export_users(format=:csv)) import_users(data=read("small.json",String), format=:json, returnFormat=:xml) @@ -41,19 +43,13 @@ import_users(data="""[{"username":"userName"}]""", format=:json) import_users(data="""username\naharris""", format=:csv) =# -begin - - #TODO: for now, put a :json format tag when using a Dict - project_token = create_project(format=:json,data=Dict(:project_title=>"$(now())",:purpose=>0)) - export_project_XML(token=project_token) - export_project_info(token=project_token) - export_metadata(token=project_token) + #export_project_XML(token=project_token) + #export_project_info(token=project_token) + #export_metadata(token=project_token) - @assert "1" == import_project_info(format=:json,token=project_token,data=Dict(:project_title=>"$(now())",:purpose=>0)) - export_logging(token=project_token, format=:json) |>JSON.parse |> DataFrame - @test export_logging(token=project_token,format=:json, endTime="1999-01-01") |> JSON.parse |> DataFrame == DataFrame() + #@assert "1" == import_project_info(token=project_token,data=(project_title="$(now())",purpose=0)) #TODO: for CSV inputs, use triple quotes, and add a comma at the end if the last inner character is also a quote # Can this always be done, or only when the last column is blank? @@ -86,117 +82,16 @@ begin #CSV.read(export_users(format=:csv) |> IOBuffer, DataFrame ) - @test true end -#TODO: Add sensible arguments and expected return values - -#= -test_sets = Dict( - "Arms" => [ - :(delete_arms()), - :(export_arms()), - :(import_arms()), - ], - "Data Access Groups" => [ - :(delete_DAGs()), - :(export_DAGs()), - :(export_user_DAG_assignment()), - :(import_DAGs()), - :(import_user_DAG_assignment()), - :(switch_DAG()), - ], - "Events" => [ - :(delete_events()), - :(export_events()), - :(import_events()), - ], - "Field Names" => [ - :(export_list_of_export_field_names), - ], - "Files" => [ - :(delete_file()), - :(export_file()), - :(import_file()), - ], - "File Repository" => [ - :(create_folder()), - :(delete_file_from_file_repository()), - :(export_file_from_file_repository()), - :(export_list_of_folders()), - :(import_file_from_file_repository()), - ], - "Instruments" => [ - :(export_instrument_event_mappings()), - :(export_instruments()), - :(export_PDF()), - :(import_instrument_event_mappings()), - ], - "logging" => [ - :(export_logging()), - ], - "Metadata" => [ - :(import_metadata()), - :(export_metadata()), - ], - "Projects" => [ - :(create_project()), - :(export_project_info()), - :(export_project_XML()), - #:(import_project_info()), - ], - "REDCap" => [ - :(export_version()), - ], - "Records" => [ - :(delete_records()), - :(export_records()), - :(generate_next_record_name()), - :(import_records()), - :(rename_record()), - ], - "Repeating Instruments and Events" => [ - :(export_repeating_instruments_and_events()), - :(import_repeating_instruments_and_events()), - ], - "Reports" => [ - :(export_reports()), - ], - "Surveys" => [ - :(export_survey_link()), - :(export_survey_participants()), - :(export_survey_queue_link()), - :(export_survey_return_code()), - ], - "Users" => [ - :(export_users()), - :(import_users()), - :(delete_users()), - ], - "User Roles" => [ - :(export_user_roles()), - :(import_user_roles()), - :(delete_user_roles()), - :(export_user_role_assignment()), - :(import_user_role_assignment()), - ], -) - -@testset "API Methods" begin - name_width = maximum(length.(keys(test_sets))) - for method_type in keys(test_sets) - for function_call in test_sets[method_type] - @testset "$(rpad(method_type, name_width, '.')): $function_call" begin - @test begin - eval(function_call) - true - end - end - end - end +if (get(ENV, "REDCAP_API_URL", "")) |> isempty && (get(ENV, "REDCAP_API_TOKEN", "")) |> isempty + @info("To run these tests, add values for REDCAP_API_URL and REDCAP_API_TOKEN to your Julia startup file") + @test true +else + const project_token = create_project(data=(project_title="$(now())",purpose=0)) + const file_name = tempname() |> touch + const file_handle = open(file_name, "w+") + run_all_tests() end -nothing -=# -end diff --git a/test/test_files/api_methods/arms.jl b/test/test_files/api_methods/arms.jl new file mode 100644 index 0000000..c2ad174 --- /dev/null +++ b/test/test_files/api_methods/arms.jl @@ -0,0 +1,94 @@ +write(file_handle, """ + + + 1 + Arm 01 + + + 2 + Arm 02 + + + 3 + Arm 03 + + + 4 + Arm 04 + + + 5 + Arm 05 + + + 6 + Arm 06 + + + 7 + Arm 07 + + + 8 + Arm 08 + +""") +seekstart(file_handle) +@test import_arms(override=1,data=file_name,token=project_token,format=:xml) == "8" +seekstart(file_handle) +@test import_arms(override=1,data=file_name,token=project_token) == "8" + +truncate(file_handle,0) +write(file_handle, """ +[ + { + "arm_num": 1, + "name": "Arm 01" + }, + { + "arm_num": 2, + "name": "Arm 02" + }, + { + "arm_num": 3, + "name": "Arm 03" + }, + { + "arm_num": 4, + "name": "Arm 04" + }, + { + "arm_num": 5, + "name": "Arm 05" + }, + { + "arm_num": 6, + "name": "Arm 06" + }, + { + "arm_num": 7, + "name": "Arm 07" + }, + { + "arm_num": 8, + "name": "Arm 08" + } +] +""") +seekstart(file_handle) +@test import_arms(override=1,data=file_name,token=project_token,format=:json) == "8" + +truncate(file_handle,0) +write(file_handle, """ +arm_num,name +1,Drug A +2,Drug B +3,Drug C +4,Drug D +5,Drug E +6,Drug F +7,Drug G +8,Drug H +""") +seekstart(file_handle) +@test import_arms(override=1,data=file_name,token=project_token,format=:csv) == "8" diff --git a/test/test_files/api_methods/data_access_groups.jl b/test/test_files/api_methods/data_access_groups.jl new file mode 100644 index 0000000..8eafcb2 --- /dev/null +++ b/test/test_files/api_methods/data_access_groups.jl @@ -0,0 +1,6 @@ +q="""data_access_group_name,unique_group_name + Subset 1, + Subset 2, + Subset 3,""" +@test import_DAGs(token=project_token,data=q,format=:csv) == "3" + diff --git a/test/api_methods/events.jl b/test/test_files/api_methods/events.jl similarity index 100% rename from test/api_methods/events.jl rename to test/test_files/api_methods/events.jl diff --git a/test/api_methods/field_names.jl b/test/test_files/api_methods/field_names.jl similarity index 100% rename from test/api_methods/field_names.jl rename to test/test_files/api_methods/field_names.jl diff --git a/test/api_methods/file_repository.jl b/test/test_files/api_methods/file_repository.jl similarity index 100% rename from test/api_methods/file_repository.jl rename to test/test_files/api_methods/file_repository.jl diff --git a/test/api_methods/files.jl b/test/test_files/api_methods/files.jl similarity index 100% rename from test/api_methods/files.jl rename to test/test_files/api_methods/files.jl diff --git a/test/api_methods/instruments.jl b/test/test_files/api_methods/instruments.jl similarity index 100% rename from test/api_methods/instruments.jl rename to test/test_files/api_methods/instruments.jl diff --git a/test/test_files/api_methods/logging.jl b/test/test_files/api_methods/logging.jl new file mode 100644 index 0000000..47bc158 --- /dev/null +++ b/test/test_files/api_methods/logging.jl @@ -0,0 +1,4 @@ +export_logging(token=project_token, format=:json) |>JSON.parse |> DataFrame +@test export_logging(token=project_token,format=:json, endTime="1999-01-01") |> JSON.parse |> DataFrame == DataFrame() + +#TODO: Users might try to use returnFormat, but it has to be format... diff --git a/test/api_methods/metadata.jl b/test/test_files/api_methods/metadata.jl similarity index 100% rename from test/api_methods/metadata.jl rename to test/test_files/api_methods/metadata.jl diff --git a/test/test_files/api_methods/projects.jl b/test/test_files/api_methods/projects.jl new file mode 100644 index 0000000..471e7a0 --- /dev/null +++ b/test/test_files/api_methods/projects.jl @@ -0,0 +1,112 @@ +@test create_project(data = (project_title = "AAA", purpose = 1),format=:csv) == +"ERROR: You must provide some text for 'purpose_other' since you specified 'purpose' as '1' (Other)." + +@test create_project(data = (project_title = "AAA", purpose = 1),format=:json) == +"{\"error\":\"You must provide some text for 'purpose_other' since you specified 'purpose' as '1' (Other).\"}" + +@test create_project(data = (project_title = "AAA", purpose = 1),format=:xml) == +"You must provide some text for 'purpose_other' since you specified 'purpose' as '1' (Other)." + +@test create_project(data = (project_title = "AAA", purpose = 1)) == +"You must provide some text for 'purpose_other' since you specified 'purpose' as '1' (Other)." + +#Why do I get different values for different formats? +@test "18" == +import_project_info(token=project_token,data=( +project_title="ABCDEFG", +project_language="English", +purpose="Testing", +purpose_other="Other", +project_notes="NONE", +custom_record_label="Q", +secondary_unique_field="R", +is_longitudinal=true, +surveys_enabled="1", +scheduling_enabled=true, +record_autonumbering_enabled=true, +randomization_enabled=true, +project_irb_number=123456, +project_grant_number=123456, +project_pi_firstname="First", +project_pi_lastname="Last", +display_today_now_button=1, +bypass_branching_erase_field_prompt=1 +),) + +@test "18" == +import_project_info(token=project_token,data=( +project_title="ABCDEFG", +project_language="English", +purpose="Testing", +purpose_other="Other", +project_notes="NONE", +custom_record_label="Q", +secondary_unique_field="R", +is_longitudinal=true, +surveys_enabled="1", +scheduling_enabled=true, +record_autonumbering_enabled=true, +randomization_enabled=true, +project_irb_number=123456, +project_grant_number=123456, +project_pi_firstname="First", +project_pi_lastname="Last", +display_today_now_button=1, +bypass_branching_erase_field_prompt=1 +),format=:csv) + +@test "18" == +import_project_info(token=project_token,data=( +project_title="ABCDEFG", +project_language="English", +purpose="Testing", +purpose_other="Other", +project_notes="NONE", +custom_record_label="Q", +secondary_unique_field="R", +is_longitudinal=true, +surveys_enabled="1", +scheduling_enabled=true, +record_autonumbering_enabled=true, +randomization_enabled=true, +project_irb_number=123456, +project_grant_number=123456, +project_pi_firstname="First", +project_pi_lastname="Last", +display_today_now_button=1, +bypass_branching_erase_field_prompt=1 +),format=:json) + +@test "18" == +import_project_info(token=project_token,data=( +project_title="ABCDEFG", +project_language="English", +purpose="Testing", +purpose_other="Other", +project_notes="NONE", +custom_record_label="Q", +secondary_unique_field="R", +is_longitudinal=true, +surveys_enabled="1", +scheduling_enabled=true, +record_autonumbering_enabled=true, +randomization_enabled=true, +project_irb_number=123456, +project_grant_number=123456, +project_pi_firstname="First", +project_pi_lastname="Last", +display_today_now_button=1, +bypass_branching_erase_field_prompt=1 +),format=:xml) + +@test export_project_info(token=project_token) |> JSON.parse |> x -> x["custom_record_label"] == "Q" +@test export_project_info(token=project_token,format=:csv) |> JSON.parse |> x -> x["custom_record_label"] == "Q" +@test export_project_info(token=project_token,format=:json) |> JSON.parse |> x -> x["custom_record_label"] == "Q" +@test export_project_info(token=project_token,format=:xml) |> JSON.parse |> x -> x["custom_record_label"] == "Q" + +export_project_info(token=project_token) + + +import_project_info(token=project_token,data=(project_title="NEWNAME"),format=:csv) +@test export_project_info(token=project_token,format=:json) |> JSON.parse |> x -> x["project_title"] == "NEWNAME" + diff --git a/test/api_methods/records.jl b/test/test_files/api_methods/records.jl similarity index 100% rename from test/api_methods/records.jl rename to test/test_files/api_methods/records.jl diff --git a/test/test_files/api_methods/redcap.jl b/test/test_files/api_methods/redcap.jl new file mode 100644 index 0000000..3e1a7ac --- /dev/null +++ b/test/test_files/api_methods/redcap.jl @@ -0,0 +1 @@ +@test export_version() |> VersionNumber ≥ v"13" diff --git a/test/api_methods/repeating_instruments_and_events.jl b/test/test_files/api_methods/repeating_instruments_and_events.jl similarity index 100% rename from test/api_methods/repeating_instruments_and_events.jl rename to test/test_files/api_methods/repeating_instruments_and_events.jl diff --git a/test/api_methods/reports.jl b/test/test_files/api_methods/reports.jl similarity index 100% rename from test/api_methods/reports.jl rename to test/test_files/api_methods/reports.jl diff --git a/test/api_methods/surveys.jl b/test/test_files/api_methods/surveys.jl similarity index 100% rename from test/api_methods/surveys.jl rename to test/test_files/api_methods/surveys.jl diff --git a/test/api_methods/user_roles.jl b/test/test_files/api_methods/user_roles.jl similarity index 100% rename from test/api_methods/user_roles.jl rename to test/test_files/api_methods/user_roles.jl diff --git a/test/test_files/api_methods/users.jl b/test/test_files/api_methods/users.jl new file mode 100644 index 0000000..e2d2bd7 --- /dev/null +++ b/test/test_files/api_methods/users.jl @@ -0,0 +1 @@ +@test export_users(token=project_token,format=:json) |> JSON.parse |> first |> x -> x["api_import"] == 1 diff --git a/test/test_files/errors.jl b/test/test_files/errors.jl new file mode 100644 index 0000000..eca2217 --- /dev/null +++ b/test/test_files/errors.jl @@ -0,0 +1,2 @@ + +@test_throws UndefKeywordError create_project() diff --git a/test/test_files/xml.jl b/test/test_files/xml.jl new file mode 100644 index 0000000..78c9443 --- /dev/null +++ b/test/test_files/xml.jl @@ -0,0 +1,178 @@ +#TODO: Translating the data parameter to XML is the more difficult +# However, that's the default format parameter. +# What's the best approach? +#= + + + + + 1 + Drug A + + + 2 + Drug B + + + 3 + Drug C + + + + + + + CA Site + ca_site + + + FL Site + fl_site + + + New Site + + + + +This one is used for import user-DAG assignment + + + + ca_dt_person + ca_site + + + fl_dt_person + fl_site + + + global_user + + + + + + + + Baseline + 1 + + + Visit 1 + 1 + + + Visit 2 + 1 + + + +Import instrument event mappings + + + + 1 + baseline_arm_1 +
demographics
+ + + 1 + visit_1_arm_1 +
day_3
+ + + 1 + visit_1_arm_1 +
other
+ + + 1 + visit_2_arm_1 +
other
+ + + +Import records is more complicated +EAV XML: + + + + + + + + + + + +Flat XML: + + + + + each data point as an element + ... + + + +This is used for import_users, where the content parameter is user, not users... + + + + harrispa + 2015-12-07 + 1 + 0 + + 1 + 2 + 0 + + + 1 + 0 + 2 + + + + +Import user roles + + + + U-527D39JXAC + Data Entry Person + 1 + 0 + + 1 + 2 + 0 + + + 1 + 0 + 2 + + + + +Import user rolw assignments + + + + ca_dt_person + U-2119C4Y87T + + + fl_dt_person + U-2119C4Y87T + + + global_user + + + + +=#