Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix bug where csv strings were triple quoted #1970

Merged
merged 1 commit into from
Jun 7, 2024
Merged

Conversation

nnecla
Copy link
Contributor

@nnecla nnecla commented Jun 4, 2024

This pull request is to remove triple quotes from the csv export of statements.

Before, exported csv:

n.types,n.point,n.cuteness,n.duration2,n.birthDate,n.cool,n.time,n.name,n
"[""Ground"", ""Psychic""]","point({srid:7203, x:3, y:0})",10.3,"""P0M0DT91.123456789S""","""2023-04-28""",true,"""12:50:35.556000000+01:00""","""Baltoy""","(:Query_graph {duration: ""P0M14DT58320S"",types: [""Ground"", ""Psychic""],level: 1,cool: true,cuteness: 10.3,name: ""Baltoy"",duration2: ""P0M0DT91.123456789S"",time: ""12:50:35.556000000+01:00"",birthDate: ""2023-04-28"",point: point({srid:7203, x:3, y:0})})"

After with this pr changes, exported csv:

n.types,n.point,n.cuteness,n.duration2,n.birthDate,n.cool,n.time,n.name,n
"[Ground, Psychic]","point({srid:7203, x:3, y:0})",10.3,P0M0DT91.123456789S,2023-04-28,true,12:50:35.556000000+01:00,Baltoy,"(:Query_graph {duration: P0M14DT58320S,types: [Ground, Psychic],level: 1,cool: true,cuteness: 10.3,name: Baltoy,duration2: P0M0DT91.123456789S,time: 12:50:35.556000000+01:00,birthDate: 2023-04-28,point: point({srid:7203, x:3, y:0})})"

Please see the following doc for csv quote requirements: https://www.ietf.org/rfc/rfc4180.txt

@nnecla nnecla added the bug label Jun 4, 2024
@nnecla nnecla self-assigned this Jun 4, 2024
@@ -648,6 +648,31 @@ describe('helpers', () => {
// Then
expect(res).toEqual([['"P1M2DT3.000000004S"']])
})

Copy link
Contributor

@OskarDamkjaer OskarDamkjaer Jun 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we test the records that come from:
return "hello " "
or
return '"'
or even
return '
"
'

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

=> hello " ,
=> ",

=> "
""
"

Maybe it should become something like those?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried these ones and the results end up looking like:

  1. return "hello \" ":
"""hello \"" """
"hello \"" "
  1. return '"':
"'""'"
"\"""
  1. return ' " ' (with new lines):
"'
""
'"
"
\""
"

Seems like they are all valid csv exports but let me check why in the first case the column name got triple quotes once again instead of just one.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also tried the below query to see different types as we've talked before.

CREATE (n {string: "string",boolean: true,integer: 1,float: 2.0,point: point({x:1,y:2}),date: date(),time: time(),datetime: datetime(),localtime: localtime(),localdatetime:localdatetime(),duration: duration("P5M1.5D"),numberList: [1,2,3],stringList: ["a"],emptyList: []}) RETURN n, n.string, n.boolean, n.integer, n.float, n.point, n.date, n.time, n.datetime, n.localtime, n.localdatetime, n.duration, n.numberList, n.stringList, n.emptyList

The exported CSV looks as follows:

n,n.string,n.boolean,n.integer,n.float,n.point,n.date,n.time,n.datetime,n.localtime,n.localdatetime,n.duration,n.numberList,n.stringList,n.emptyList
"({localtime: 10:21:41.776000000,date: 2024-06-06,string: string,integer: 1,float: 2.0,point: point({srid:7203, x:1, y:2}),duration: P5M1DT43200S,emptyList: [],datetime: 2024-06-06T10:21:41.776000000Z,boolean: true,stringList: [a],numberList: [1, 2, 3],time: 10:21:41.776000000Z,localdatetime: 2024-06-06T10:21:41.776000000})",string,true,1,2.0,"point({srid:7203, x:1, y:2})",2024-06-06,10:21:41.776000000Z,2024-06-06T10:21:41.776000000Z,10:21:41.776000000,2024-06-06T10:21:41.776000000,P5M1DT43200S,"[1, 2, 3]",[a],[]

which is a valid CSV without triple quotes 💃.

Thinking back on the triple quotes in the column name in the top comment, I believe it is expected as indeed the column name is "hello \" " rather than just hello \" . We can try to remove the quotes in that case as well but for that, we may need to add an if check which removes the quotes from the beginning and the end of the result record.

Will be testing the jar file and merge the pr if all is good.

@nnecla nnecla merged commit efffa1e into master Jun 7, 2024
16 checks passed
@nnecla nnecla deleted the triple-quotes-bug-fix branch June 7, 2024 08:54
@OskarDamkjaer OskarDamkjaer changed the title removing triple quotes from csv exports Fix bug where csv strings were triple quoted Jun 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants