Feature/add raw line on error tuple option #95
Conversation
|
Thanks for putting this together I'll hopefully get a look later this week |
| escape_sequence: escape_sequence, | ||
| line: index + 1, | ||
| escape_max_lines: escape_max_lines | ||
| ).message, raw_line} |
There was a problem hiding this comment.
This is breaking backwards compatibility if I interpret it correctly. If someone has existing code that depends on the length of the tuple being returned, they will get a compilation error. We should either try to avoid it by adding the raw line to the message, or reserve this for a major version upgrade.
There was a problem hiding this comment.
I included the :raw_line_on_error option (defauls to false) so that only when set to true the extra raw_line will return on the error tuple, so I'd say it won't break backwards compatibility.
|
Hey @y86 this is looking good, thanks! I think adding more information on errors is going to help users of this library get to the bottom of problems more quickly. There are some issues with backwards compatibility in the pr as it is now. Also, there is some missing coverage of how errors get pulled through from the decoder to the top level module ( Is the idea that this should help with error messaging and human intervention, or do you use this to intervene programatically in case of errors? Depending on what the goal is, we could then ideate on possible backwards compatible ways of achieving the same thing. |
|
Hey @beatrichartz , I am not too familiar with coveralls, so I wouldn't know where to start on the tests to bring it back from where it was. Our use case for this was to easily return the problematic lines to the user in the interface so he could either fix them there and resubmit or download them into separate smaller file for review - the original csv files are quite large and sometimes they can't even edit them on Excel. |
|
I think this functionality could still make it into a future release but would need updating. It's become a bit more challenging due to the parser accepting byte streams additionally to line by line input. Effectively we'll need to collect the full line, which the parser is already doing unless the byte size of the input is smaller than the line length, in which case it collects only the field. Will have a look at adapting this hopefully soon. |
Includes an option to add the raw csv line to the the error tuple and to the exceptions.