Skip to content

Commit

Permalink
Add HTTPError handling when Reimbursement URL doesn't exist
Browse files Browse the repository at this point in the history
The issue happened when running Rosie on 2020-01-01. By the way the adapter.py works, it tries to fetch reimbursement files up to the year in which the application is running .e.g. 2020. Since there is no such URL (at the time of this commit), the whole process fails due to an HTTPError 404. 

By adding this handling, the reimbursements that cannot be found are skipped from the download process and the application runs without any issues.
  • Loading branch information
luizfzs committed Jan 1, 2020
1 parent 12d23e8 commit 58c442c
Showing 1 changed file with 5 additions and 1 deletion.
6 changes: 5 additions & 1 deletion rosie/rosie/chamber_of_deputies/adapter.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@

import numpy as np
import pandas as pd
from urllib.error import HTTPError

from serenata_toolbox.chamber_of_deputies.reimbursements import Reimbursements
from serenata_toolbox.datasets import fetch
Expand Down Expand Up @@ -85,7 +86,10 @@ def update_reimbursements(self, years=None):

for year in years:
self.log.info(f'Updating reimbursements from {year}')
Reimbursements(year, self.path)()
try:
Reimbursements(year, self.path)()
except HTTPError as e:
self.log.error(f'Could not update Reimbursement from year {year}: {e} - {e.filename}')

def prepare_dataset(self, df):
self.rename_categories(df)
Expand Down

0 comments on commit 58c442c

Please sign in to comment.