Skip to content

Commit

Permalink
ICEWS data end of life; bug fix
Browse files Browse the repository at this point in the history
  • Loading branch information
andybega committed Jun 6, 2023
1 parent 9d79b1d commit 3746c72
Show file tree
Hide file tree
Showing 5 changed files with 24 additions and 26 deletions.
2 changes: 1 addition & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Type: Package
Package: icews
Title: Get the ICEWS Event Data
Version: 1.0.2
Version: 1.0.3
Authors@R:
person(given = "Andreas",
family = "Beger",
Expand Down
8 changes: 6 additions & 2 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
# icews 1.0.3

The ICEWS data were discontinued on 11 April 2023. This final version of the package has a fix to allow it to continue working (the weekly dataverse repo was removed and all data are now in what used to be the yearly file repo).

# icews 1.0

This is the first version released to CRAN.
This is the first version released to CRAN. Well, several failed attempts at least.

## Resumption of ICEWS updates with weekly data drops

Expand All @@ -16,7 +19,8 @@ The ICEWS project ceased updates in late 2019, but resumed again in May 2020. Ho

## Minor fixes, etc.

- Fix bugs in the non-standard 2017 events file; non-standard file name ("Events.2017.20200602.tab.zip") and dates formatted like "12/1/2017" (#57).
- Fix a July 2022 update error due to a non-standard file for the January 2022 data. (#80)
- Fix bugs in the non-standard 2017 events file; non-standard file name ("Events.2017.20200602.tab.zip") and dates formatted like "12/1/2017". (#57)
- Move to testthat 3rd edition. This required replacing the deprecated `testthat::with_mock()` with solutions from the {mockr} and {mockery} packages, which have as a result been added to Suggests. (#76)

## Fixes, etc. for the old daily repo structure
Expand Down
4 changes: 3 additions & 1 deletion R/state_dataverse.R
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,9 @@ get_dvn_manifest <- function(icews_doi = get_doi(), server = Sys.getenv("DATAVER
tibble(repo = c("historic", "weekly"),
content = list(
dataverse::get_dataset(icews_doi$historic, server = server),
dataverse::get_dataset(icews_doi$weekly, server = server)
# Fix for ICEWS end of life (no more weekly repo)
#dataverse::get_dataset(icews_doi$weekly, server = server)
list(files = data.frame())
)),
error = function(e) {
stop("Something went wrong in 'dataverse' or the Dataverse API. Check that the API token is valid (?dataverse_api_token) and try again. Original error message:\n", e$message)
Expand Down
6 changes: 2 additions & 4 deletions README.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ knitr::opts_chunk$set(
[![R-CMD-check](https://github.com/andybega/icews/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/andybega/icews/actions/workflows/R-CMD-check.yaml)
[![Codecov test coverage](https://codecov.io/gh/andybega/icews/branch/master/graph/badge.svg)](https://codecov.io/gh/andybega/icews?branch=master)

_Note: the ICEWS data were discontinued on 11 April 2023. You can still use this package to download the data from dataverse however._

Get the ICEWS event data from the Harvard Dataverse repos at [https://doi.org/10.7910/DVN/28075](https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/28075) (historic data) and [https://doi.org/10.7910/DVN/QI2T9A](https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/QI2T9A) (weekly updates).

The icews package provides these major features:
Expand All @@ -29,10 +31,6 @@ The icews package provides these major features:
- set options so that in future R sessions icews knows where your data lives
- icews keeps the local data in sync with the latest versions on Dataverse

*Note about stability:*

*This package is young. The core functionality of the package involves downloading and keeping track of potentially 10+GB of files and data. I have not found a good way to test that thoroughly. There are probably bugs and errors that pop up on a fresh run without any downloaded data already in place. Please file bug reports or email me (adbeger@gmail.com)!*

## Installation

Not on CRAN, so you will have to install via remotes or devtools:
Expand Down
30 changes: 12 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ status](https://www.r-pkg.org/badges/version/icews)](https://cran.r-project.org/
[![Codecov test
coverage](https://codecov.io/gh/andybega/icews/branch/master/graph/badge.svg)](https://codecov.io/gh/andybega/icews?branch=master)

*Note: the ICEWS data were discontinued on 11 April 2023. You can still
use this package to download the data from dataverse however.*

Get the ICEWS event data from the Harvard Dataverse repos at
[https://doi.org/10.7910/DVN/28075](https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/28075)
(historic data) and
Expand All @@ -16,22 +19,13 @@ Get the ICEWS event data from the Harvard Dataverse repos at

The icews package provides these major features:

- get the ICEWS event data without having to deal with Dataverse
- use raw data files (tab-separated variables, .tsv) or a database
(SQLite3) or both as the storage backend
- set options so that in future R sessions icews knows where your data
lives
- icews keeps the local data in sync with the latest versions on
Dataverse

*Note about stability:*

*This package is young. The core functionality of the package involves
downloading and keeping track of potentially 10+GB of files and data. I
have not found a good way to test that thoroughly. There are probably
bugs and errors that pop up on a fresh run without any downloaded data
already in place. Please file bug reports or email me
(<adbeger@gmail.com>)!*
- get the ICEWS event data without having to deal with Dataverse
- use raw data files (tab-separated variables, .tsv) or a database
(SQLite3) or both as the storage backend
- set options so that in future R sessions icews knows where your data
lives
- icews keeps the local data in sync with the latest versions on
Dataverse

## Installation

Expand Down Expand Up @@ -125,7 +119,7 @@ Remove 'events.1996.20140313082528.tab'

The events come in (zipped) tab-separated files. To load all of these
into memory in a big combined data frame with about 16 million rows
(\~2.5Gb):
(~2.5Gb):

``` r
events <- read_icews("~/Downloads/icews")
Expand Down Expand Up @@ -209,7 +203,7 @@ str(cameo_codes)
#> $ lvl1 : int NA 10 11 12 13 14 15 16 17 18 ...
#> $ description : chr NA "All public statements expressed verbally or in action not otherwise specified." "Explicitly decline or refuse to comment on a situation." "Express pessimism, negative outlook." ...
#> $ usage_notes : chr NA "This residual category is not coded except when distinctions among 011 to 017 cannot be made. Note that statements are typicall "This event form is a verbal act. The target could be who the source actor declines to make a comment to or about." "This event form is a verbal act. Only statements with explicit pessimistic components should be coded as 012; otherwise, defaul ...
#> $ example : chr NA "U.S. military chief General Colin Powell said on Wednesday NATO would need to remain strong." "NATO on Monday declined to comment on an estimate that Yugoslav army and special police troops in Kosovo were losing 90 to 100 "Former West Germany Chancellor Willy Brandt said in a radio interview broadcast today he was skeptical over Moscow\u0082\xc4\xf ...
#> $ example : chr NA "U.S. military chief General Colin Powell said on Wednesday NATO would need to remain strong." "NATO on Monday declined to comment on an estimate that Yugoslav army and special police troops in Kosovo were losing 90 to 100 "Former West Germany Chancellor Willy Brandt said in a radio interview broadcast today he was skeptical over Moscow\x82\xc4\xf4s ...
#> $ order : int 1 2 3 4 5 6 7 8 9 10 ...
#> $ quad_category : chr "verbal cooperation" "verbal cooperation" "verbal cooperation" "verbal cooperation" ...
#> $ penta_category: chr "statement" "statement" "statement" "statement" ...
Expand Down

0 comments on commit 3746c72

Please sign in to comment.