Skip to content

Commit

Permalink
set the splitter to 20
Browse files Browse the repository at this point in the history
  • Loading branch information
paul-butcher committed Jul 23, 2024
1 parent b79f75d commit 35d6444
Showing 1 changed file with 9 additions and 14 deletions.
23 changes: 9 additions & 14 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -33,18 +33,13 @@ shoots/clean:

# Slice a given input file into manageable chunks, so that you can run them through the
# transfer process separately without overwhelming the target system.
# The number here is based on the original input file containing 907 shoots,
# so this creates 9 files of about 100.
# The right number is yet to be discovered.
# The right number is yet to be discovered.
# The right number for archivematica is probably about 20.

%.sliced: %
split -l 101 $< $<.

# Touch the sho
%.touched: %
cat % | xargs aws s3 cp \
--metadata 'touched=touched' \
--recursive --exclude="*" \
--include="$2" \
"${@:3}" \
"$1" "$1"
split -l 20 $< $<.

%.touched.staging: %
cat % | AWS_PROFILE=digitisation-developer python src/touch.py staging

%.touched.production: %
cat % | AWS_PROFILE=digitisation-developer python src/touch.py production

0 comments on commit 35d6444

Please sign in to comment.