Import process from PDC Describe is crashing

[hectorcorrea]

It looks like the import process from PDC Describe is crashing on one of the new and rather large records. The record in question is https://pdc-describe-prod.princeton.edu/describe/works/470. If you load this record in PDC Describe you'll notice that it takes a rather long time to load and loading the file list (60,005 files) also takes a very long time.

This is what Honeybadger reports:

Error importing record from https://pdc-describe-prod.princeton.edu/describe/works/470.json. 
Exception: Net::ReadTimeout 

The record in Discovery still looks like a DataSpace record because the import from Describe is failing: https://datacommons.princeton.edu/discovery/catalog/165364

[hectorcorrea]

Loading the file list on this record takes 30 seconds in production:

pdc-describe(prod)> Rails.logger.level = Logger::INFO
=> 1
pdc-describe(prod)> w = Work.find(470)
=> #<Work:0x00007f8782575420 id: 470>
pdc-describe(prod)> w.file_list.count
Loading S3 objects. Bucket: pdc-describe-prod-postcuration. Prefix: 10.34770/n42z-hb72/470/. Elapsed: 30.917009837 seconds
=> 60005
pdc-describe(prod)> 

[hectorcorrea]

Fetching the data via curl also takes 30+ seconds, notice that the payload is 13M

~/src $ curl https://datacommons.princeton.edu/describe/works/470?format=json > 470.json
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 13.2M    0 13.2M    0     0   407k      0 --:--:--  0:00:33 --:--:-- 3300k