Source to Field name mapping generation?

[Falcon2k1]

Notes and Context

Hello! I'm using this to help turn human-generated CSV data into something I can work with more easily as as script in Python. This means that I'm often working with CSVs with column titles that don't match Python standards, and it's not in scope for me to change that. For example:

How many Cats?,What are their Colors?,Are they cute?
12,Black,Yes!
5,Grey,No.

If I run datamodel-codegen --input .\cat_file.csv --output model.py --output-model-type dataclasses.dataclass --input-file-type csv --snake-case-field, I get the following output:

# generated by datamodel-codegen:
#   filename:  cat_file.csv
#   timestamp: 2023-08-04T16:57:58+00:00

from __future__ import annotations

from dataclasses import dataclass


@dataclass
class Model:
    how_many_cats_: str
    what_are_their_colors_: str
    are_they_cute_: str

The problem now is that I don't have an easy way to actually put my csv into this data model, so I have to write a mapping by hand:

field_name_mapping = {
    "How many Cats?": "how_many_cats_",
    "What are their Colors?": "what_are_their_colors_",
    "Are they cute?": "are_they_cute_",
}

data_list = [Model(**{field_name_mapping[k] : v for k,v in row.items()) for row in imported_csv_data]

This is also required if I wanted to re-generate the CSV later (by reversing the field mappings) and serialize these dataclasses back to the original data format.

The Request

Is there a feature to generate that field name mapping automatically? If not, would you accept a pull request adding it? It might only be useful for dataclasses.dataclass, but it's nice as this accelerates how quickly I can work with these without just managing a big dictionary.

[koxudaxi]

@Falcon2k1
Can you use the pydnatic model instead of dataclasses.dataclass?
If you need the last output as dataclasses.dataclass, I recommend you parse data with pydantic model and dump to dict or dataclass.

[tom-a-horrocks]

I think my respository might do this for you: https://github.com/tom-a-horrocks/datamodel-inflator, but it assumes that the dataclass names match the dictionary keys exactly.

[denisart]

This idea of course has some problems with dataclasses. This idea of course has some problems with dataclasses. But we need for this in render of graphql. I'll give a typical example:

schema.graphql

interface ICondition {
  id: ID!
}

type EqualityIntCondition implements ICondition {
  id: ID!
  value: Int!
}

type EqualityStringCondition implements ICondition {
  id: ID!
  value: String!
}

type OtherType {
  id: ID!
  value: String!
}

type Query {
  conditions: [ICondition]!
}

the following query is incorrect because we have the same field names with different types:

query getConditions{
  conditions {
    ... on EqualityIntCondition {
      __typename
      id
      value
    }
    ... on EqualityStringCondition {
      __typename
      id
      value
    }
  }
}

Correct query it is query with graphql aliases (https://graphql.org/learn/queries/#aliases)

query getConditions{
  conditions {
    ... on EqualityIntCondition {
      __typename
      id
      intValue: value
    }
    ... on EqualityStringCondition {
      __typename
      id
      stringValue: value
    }
  }
}

and the result of this query it is

{
  "conditions": [
    {
      "__typename": "EqualityIntCondition",
      "id": "1",
      "intValue": 1
    },
    {
      "__typename": "EqualityStringCondition",
      "id": "2",
      "intValue": "2"
    }
  ]
}

If we using aliases arg we have:

datamodel-codegen --input schema.graphql --input-file-type graphql --output model.py --aliases aliases.json

where aliases.json it is {"value": "intValue"} we have problem for EqualityStringCondition type and OtherType type

model.py

...

class ICondition(BaseModel):
    id: ID
    typename__: Optional[Literal['ICondition']] = Field(
        'ICondition', alias='__typename'
    )


class EqualityIntCondition(ICondition):
    id: ID
    intValue: Int = Field(..., alias='value')
    typename__: Optional[Literal['EqualityIntCondition']] = Field(
        'EqualityIntCondition', alias='__typename'
    )


class EqualityStringCondition(ICondition):
    id: ID
    intValue: String = Field(..., alias='value')
    typename__: Optional[Literal['EqualityStringCondition']] = Field(
        'EqualityStringCondition', alias='__typename'
    )


class OtherType(BaseModel):
    id: ID
    intValue: String = Field(..., alias='value')
    typename__: Optional[Literal['OtherType']] = Field('OtherType', alias='__typename')

we need field aliases and field names special for a type model.

For example, I can to create beta version with this functional for graphql parser .

[Falcon2k1]

@Falcon2k1 Can you use the pydnatic model instead of dataclasses.dataclass? If you need the last output as dataclasses.dataclass, I recommend you parse data with pydantic model and dump to dict or dataclass.

I generally would prefer to work with Dataclasses instead of Pydantic; and it shouldn't be necessary to install a third party library anyway. As I pointed out in the discussion, this is just an exposure of data that you're already processing, as you're performing this shift from the external csv file's headers to the python-safe snake_case: this would just be a way to store and include that in the schema provided.

I'll just write something on my own to solve for this. I just wanted to offer to contribute back, but I suspect this might not be a standard use case for this project and thus might not be worth including in your package.

I think my respository might do this for you: https://github.com/tom-a-horrocks/datamodel-inflator, but it assumes that the dataclass names match the dictionary keys exactly.

Unfortunately my whole concern is from situations where the dataclass names cannot match the dictionary; ergo a CSV or other file that has column names that include things that don't match python's snake_case.