Initial models

[dchiller]

As of 14 Sept 2023

Models

NB: All models will include information on revisions (who revised and when) from django-reversion (see django-reversion docs)

User – Attributes of a specific VIM user

• email (required): primary log-in ID and contact method
• full_name (required): user’s full display name
• country (optional): user’s country, Wikidata ID
• language (required): user's preferred site language, language code for use with translator
• city (optional): user’s city, Wikidata ID
• institution (optional): institution the user is associated with, for display on the user’s detail page
• about (optional): text to display on user’s detail page

Instrument – Instruments hosted by VIM

• wikidata_id (required): Wikidata id for the instrument
• hornbostel_sachs_classification (optional): classification in Hornbostel-Sachs classification system
• mimo_classification (optional): classification in MIMO classification system
• default_image (required): foreign key to AVResource to use as the default image for the instrument

Language - Languages supported by VIM

• wikidata_code (required): Wikidata language code
• wikidata_id (required): Item ID in Wikidata
• en_label (required): Language label on wikidata in English
• autonym (required): Language autonym on wikidata in script

InstrumentName - Details and provenance of the name of an instrument in a particular language

Source may be broken out into an additional model.

• instrument (required): foreign key to Instrument
• language (required): foreign key to Language
• name (required): the name of the instrument in the language
• avresource_added (optional): foreign key to AVResource if any AV resources were provided with this name
• source_name (required): who (person) or what (eg. written source) called the instrument this
• source_type (required): what type of source is this?
• source_date (required): date of source
• source_location (required): wikidata ID, geography
• source_description (optional): narrative details on the source of the name provided
• description_language (optional; required if source_description not empty): foreign key to Language; the language in which the description is entered

AVResource – Records AV resources and maps to an instrument

NB: AVResources will have source provenance data, but I need to do more research into how we can harmonize what's on Wikimedia Commons already and new AV resources.

• instrument (required): foreign key to Instrument
• type (required): audio/video/photo/?
• format (required): resource format
• uri (required): ID for the resource on internet archive
• instrument_date (optional): date instrument was made
• instrument_maker (optional): maker of instrument
• instrument_location (optional): place instrument was made; wikidata ID, geography
• instrument_description (optional): additional narrative about the instrument in the resource
• instrument_description_language (optional; required if instrument_description is not empty): foreign key to Language

URLS

NB: Nouns in URLs (eg. users, languages, instruments) are consistently in the plural!

User

“/users/<user_id>” -- User detail page (later: ranking)

Language

“/languages/” -- summary information about languages – list of languages, number of instruments needing names (later: top contributor)

“/languages/<language_id>” -- language detail page

Instrument

“/instruments/” -- summary information about instruments

"/instruments/languages/<language_id>" -- page with lists of instruments that have labels in the given language and that need labels in the given language

“/instruments/<instrument_id>” -- list of an instrument’s names in various languages

“/instruments/<instrument_id>/languages/<language_id> -- details/history/provenance of an instrument’s name in a given language

[lucasmarchd01]

Here is a start, feel free to add/suggest changes:

BaseModel (Model): A base model that contains fields and functionality used in all models.

• date_created (DateTimeField): the date the entry was created
• date_updated (DateTimeField): The date the entry was updated
• created_by (ForeignKey): ForeignKey to the User model of who created the object
• last_updated_by (ForeignKey): ForeignKey to the User model of who last updated the object

Family (Model): An instrument can belong to a family of instruments (Wind, String, Percussion, etc.).

• name (CharField): the name of the family
• description (TextField): description of the family of instruments

Instrument (Model): A particular instrument that may have multiple alternate names in different languages

• name (CharField): The name of the instrument.
• family (ForeignKey): ForignKey to the Family model that describes what family the instrument belongs to
• origin (CharField): The place of origin of the instrument.
• description (TextField): A detailed description of the instrument.
• image (ImageField): An image field to store the image of the instrument.

AlternativeName (Model): To store multiple alternative names for each instrument, along with the corresponding language.

• name (CharField): The alternative name of an instrument
• language (ForignKey): ForignKey to the Language model
• Instrument (ForeignKey): A ForeignKey to the Instrument model linking the alternative name to a specific instrument.

Language (Model):

• name (CharField): name of the language

User (AbstractUser):

• institution
• city
• country
• website
• full_name
• email (EmailField)

Maybe we could also have something like Audio/video clips?

[dchiller]

Maybe we could also have something like Audio/video clips?

Good question whether we anticipate ever supporting these -- idk!

[fujinaga]

Yes, we definitely want to provide options for uploading images, audio/video clips.

[lucasmarchd01]

Wikidata also has audio/video clips in their instrument data entries

[dchiller]

I think we need to assume that there will be multiple images per instrument...

It's also unclear to me, now that I am thinking about it, how this is "moderated" -- if you think you have a new instrument, do you submit it and then that goes in a moderation queue? Or is it more like a wiki that you submit but then there is opportunity for moderation post facto?

[ahankinson]

Just a quick look:

BaseModel (Model): A base model that contains fields and functionality used in all models.

date_created -> created
date_updated -> updated

The "date" prefix is redundant, and anyway it's a "date time" field, not just a date.

created_by, updated_by -> If you use a module like django-reversion for doing model history and recovery, it will store this information anyway, so no need to store it in each model.

Language (Model)

You'll probably want to store the language code somehow, and decide on a standard to follow. (ISO 639, BCP 47). You'll also probably want a way to differentiate between the "language" and the "script", since many languages can be expressed in multiple different scripts (roman alphabet vs. their own alphabet). Some will also have different romanization schemes. (https://en.wikipedia.org/wiki/Romanization_of_Japanese)

Instrument (Model): A particular instrument that may have multiple alternate names in different languages

• You'll probably want multiple images for the instrument.
• If you make origin just a CharField, you can't control the origin so you won't be able to say "show me all instruments from Japan"
• You can't have different descriptions in different languages
• You can't actually say what language the "main" name of the instrument is...
• There are lots of other thesaurus' of instruments that you probably want to link to somehow...

Family (Model): An instrument can belong to a family of instruments (Wind, String, Percussion, etc.).

• You can't have multiple family names in different languages

[dchiller]

If you make origin just a CharField, you can't control the origin so you won't be able to say "show me all instruments from Japan"

Yeah, I think we need to link to an external geographic reference. Origin could be at different geographic levels.

[lucasmarchd01]

To handle cases where an instrument may have many sub-categories (Example: "guitar" could be electric-guitar, acoustic-guitar, etc.) we could have a field in the Instrument model:
related_instruments (ManyToMany): Allows an instrument to point to other instruments, forming relationships between them. With symmetric=False, the relationship doesn't have to be mutual. An instrument can point to another instrument, but the other instrument does not have to point back to the first one.

Perhaps the Family model should just be renamed to "Category" to handle the more abstract instrument categories (idiophones, membranophones, chordophones, aerophones).

[lucasmarchd01]

Additionally, categories such as idiophone (Q12639), string instrument (Q1798603), etc. all belong to the type of musical instrument class.

[dchiller]

Yeah, seems like wikidata has extensive and complex categorization of instruments (for one examples, a instrument or instrument category can be part of multiple non-overlapping or somewhat overlapping categories.

I think we can just point to wikidata here.

[fujinaga]

You can use SPARQL queries to find the class hierarchy.
Go to: https://query.wikidata.org/
Put, for example, to find subclasses of "Plucked String Instrument" (Q230262):

SELECT ?subclass ?subclassLabel
WHERE {
?subclass wdt:P279* wd:Q230262 .
SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
}

Ask ChatGPT for formatting SPARQL queries.

[lucasmarchd01]

Good to know, thanks! I also found this query builder https://query.wikidata.org/querybuilder/?uselang=en

[fujinaga]

Yes, they had that before ChatGPT, which I think is easier to use.

[lucasmarchd01]

Location (Model)

• name (CharField): name of location (could be a country, city, any level of geographical location)
• parent (ForeignKey): ForeignKey to self (Location) of a parent location if the location is a sub-level

Instrument (Model)

• origins (ManyToMany): Instrument model should have an updated field for "origins" that can point to multiple Location objects.

[fujinaga]

Location: Again follow Wikidata or some international standard for locations. These can get quite complex due to historical changes and how different people call the same location different names.

[dchiller]

Note: I'm trying to update the model in the main post to keep track of the discussion, so we can reference it for the current "senseof the group"

[dchiller]

I'm now thinking that instrument names should also come from wikidata. Instrument descriptions we might want more of.

[fujinaga]

Yes, it would be great to have names from multiple sources. I would like to emphasize that data provenances should also be documented, e.g., who, where, and when the data was created.

[dchiller]

Noted, re: data provenance

[dchiller]

@zhannaklimanova @lucasmarchd01 I've tried to capture the discussion here in the initial post in the discussion...since there hasn't been discussion here yet this week wondering if I can go ahead and start on a PR. Any thoughts//reactions to that first post before I do that?

[lucasmarchd01]

Looks good to me!

[dchiller]

A few relevant notes from VIM's 31 July meeting:

• provenance of data is important: not just who added the data, but the who, what, when, and where that that person's data came from. Eg. from today's meeting: "My grandmother called this instrument X." We want to know time and place that the instrument existed, but also the time and place of when the contributor added this name.
• VIM is focused on names and images (and their provenance) but other information (eg. a description) is outside of scope and can be linked elsewhere (wikipedia).
• Language will not be a controlled vocabulary

[dchiller]

To outline in writing some of my thinking, I started with separate Instrument and InstrumentName models, but couldn't figure out what our model of an Instrument was independent of a particular name or a particular image. In other words, what goes in the Instrument model?

As I understand it, there are two "things" that could be associated with an instrument (an entity on a curated, for "garbage" removal, list of wikidata ID's): a name and an image (let's say image for simplicity, but could be a video, could be an audio file). VIM supports an accordion having many names (in different languages and in the same language) and VIM supports an accordion having many images, so these are many-to-one relationships. Are there any one-to-one relationships from an accordion to some property or some other abstraction that VIM supports?

I can think of two, one from before this morning's meeting and one as a result of this morning's meeting.

The first is whether or not VIM added this instrument to Wikidata or if the instrument came to wikidata from some other source. This could be stored in the InstrumentName model as in the proposal above, because an added instrument will always be initially added with at least one name, but it conceptually makes more sense probably to store this as a property in an Instrument model. So, that may be a change we should make.

The second is that this meeting reminded me that there will be curation (some bit of it manual) of what entities in wikidata are going to be in VIM. So there is a one-to-one relationship between an instrument and whether or not it is being included in VIM. A problem with this approach, that is tangential here, is what happens when someone comes to VIM and enters something that is not in VIM but is in wikidata and that we had already decided shouldn't be in VIM. My thought is that we would know if an instrument passed curation to be in VIM because it had names and images (ie. an InstrumentName object was associated with that instrument, etc.). But perhaps again it makes more conceptual sense to say that it passed curation (or has been added by a user) if we have an Instrument object.

Both of these are only issues if we are storing anything about names on VIM itself or all this is being stored directly on wikidata, so that's a pretty big question to resolve...

What I am confused by is why an Instrument object would have name and/or image attributes, which is what I thought was one thing that was being said in the discussion. So, any clarification of this and/or comments about the above would be wonderful!

[jacobdgm]

What I was trying to communicate in today's meeting turns out to be fairly similar to the models outlined in Lucas's first comment - the main suggestion, I think, would be to rename AlternateName to Name, so we don't prioritize one name over all the others in all other languages.

Here is how I imagine models could work (thanks to @annamorphism, @kyrieb-ekat and @lucasmarchd01 for helping refine/clarify this):

User

as outlined above by @dchiller

Instrument

• id - primary key
• wikidata_id
• hornbostel_sachs_classification (if we want)
• mimo_classification (if we want)
• family - foreign key or one-to-many field, pointing to Family table
• default_image (if we want - could use for thumbnails) - foreign key, pointing to AVResource table
• name_shorthand - common name of the instrument in English, just for internal use
• name_set (django gives us this for free, see Name model) - a list of all names this instrument has been labelled as in all languages
• av_resource_set (django gives us this for free, see AVResource model) - list of images/video clips/audio clips associated with the instrument

Name (or NameOfInstrumentInLanguage)

• id - primary key
• instrument - foreign key pointing to Instrument table
• language - wikidata ID, or foreign key pointing to Language table
• name - the name of the instrument translated into the language
• created_by, created, etc. handled by django-reversion

AVResource

• id - primary key
• instrument - foreign key pointing to Instrument table
• type, format, uri, etc., as outlined by @dchiller in initial comment

Family

• id - primary key
• wikidata_id
• we could also include superfamily or something, to account for "flute" family being a subclass of "aerophone" family, etc.

Language

• this model is probably not necessary, but we could include it for ease of querying etc.
• e.g. if we made this model, we'd get name_set for free

[jacobdgm]

What is in the Language model? I hear that it is not necessary, but I'm not sure I'm following what the benefit would be without knowing what model attributes there are.

So in our Name model, I expect we'll have to have a language field, and I imagine we'll want a controlled vocabulary for this field. One level of controlled vocabulary could be a user simply having to select a language by its wikidata ID. But having Language objects in the database might be useful for a couple of reasons - first, it would allow us to ensure at the database level that all added Names have languages, and second, we can use the related objects manager. The model itself would be really simple - the wikidata ID for the language, and the name of the language.

I think we need to figure out what elements of a taxonomy or other relationships among instruments we want to include in VIM. Are these all in Wikidata? If so, why are we storing them in our database? Are we curating these relationships too? What happens when they change in Wikidata? It's not immediately clear to me that we benefit from having this relationship in our database. Knowing instrument relationships could eventually help us help users find other instruments whose names they know (eg. "You knew the name of instrument A in language 1, but we know that instrument A is in the same family as instrument B, so maybe you also know the name of instrument B in language 1"), but even for this we don't need this in our database.

If I summarized this as "do we need models for InstrumentFamily and similar concepts", would this get at the core of your question?

Is the omission of my proposed data provenance fields intentional (ie. can't tell if what's been outlined here means you think we shouldn't have them or if you just didn't include them)?

I.e. source_location, source_name, source_description, and so on for AVResource and Instrument? I was mostly drafting new models from scratch and referring to previous ideas where they existed, so the omission of any particular field wasn't meant to be a statement! If I'm asking about the fields you're talking about, though, I'm wondering whether every instrument or name will have no more than one source.

It's possible I'm not understanding exactly what you mean by source here. If source is some historical or current reference to an instrument, might it make sense to create a Source model as well, allowing instruments/names to be mentioned by multiple sources?

[dchiller]

Re: language field, makes sense. Still struggling to understand a bit about how Wikidata works multi-lingually, but I believe our languages will be some subset of this table

If I summarized this as "do we need models for InstrumentFamily and similar concepts", would this get at the core of your question?

Sure. "Do we need", "do we want", "is it worth the trouble to have"....

I.e. source_location, source_name, source_description, and so on for AVResource and Instrument?

Yes, plus instrument_location, instrument_maker etc for AVResource. If I am trying to record the provenance of an instrument name in a given language, I want to know both who is the person who is providing this (ie. who is the VIM user entering this information? We get this from django-reversion) and where the information is coming from ("I read it in this book published at this time" or "This person called this instrument this name at this time"). For an AVResource, I probably want to know who the person providing the information to VIM is (we get this from django-reversion), where and when the audio visual file is from ("Person A took this picture of this timpani in Vancouver in 2010"), and information about the instrument in the av resource ("The timpani was built by my grandfather in my town in 1974").

[dchiller]

Overall, I think some version of this with additional provenance fields is fine. I remain ambivalent about what you have in the Instrument model, given my uncertainty about whether or not we want to be modeling instrument taxonomy. As I write about in more detail above, at this point I think I'm convinced of an Instrument model as a way of tracking provenance (VIM user or wikidata) and the domain of instruments that we've curated, but not other things.

[jacobdgm]

In my thinking, source_location, source_name, etc. should be attached to Name/AVResource and not to Instrument. I'm thinking of Name.name and .language as the main bits of data we're storing, while the Instrument model is mainly used to associate different names with the same instrument.

[dchiller]

source_location, source_name, etc. should be attached to Name/AVResource and not to Instrument

Exactly!

[dchiller]

I've updated the main discussion post to reflect the direction we've agreed to in today's (14 Sept) meeting. I'm going to close this discussion in recognition that what's above is the direction for our first attempt; substantive changes and additions can be addressed in new discussion, issues, etc.

If you see something in the main post that is not what we agreed to in the meeting, though, that's unintentional, so please edit or comment!

[jacobdgm]

InstrumentName - Details and provenance of the name of an instrument in a particular language

...

• avresource_added (optional): foreign key to AVResource if any AV resources were provided with this name
  ...

I don't understand what avresource_added is meant to indicate - can you clarify?

[dchiller]

This links a name to an avresource. If someone says "This instrument is called X in my language" and the demonstrative refers to the instrument in a picture they are providing (as opposed to a picture that already exists on VIM), this records that. We can connect an AVResource object to an Instrument object through the instrument attribute of an AVResource object; this allows us to connect an AVResource to an InstrumentName.

Maybe this should be a pointer from AVResource to InstrumentName instead of the current direction. IE:

AVResource
...

• instrument_name (optional): foreign key to InstrumentName if the resource was added with an instrument name in a particular language
  ...

[jacobdgm]

This helps - thanks!

I guess we need to decide whether we want people to be labelling pictures or to be labelling instruments that have descriptions/pictures/etc. I'm trying to imagine a situation where the distinction would be relevant: Imagine there are two pictures of, say, a clavinet. A user looks at one picture and says "my best guess is that it's an electric organ". They look at another picture, of the mechanism inside, and say "ah! that's clearly a clavinet!".

If Names are attached to Instruments, then users can look at multiple instruments, as well as potentially a description, in deciding how to label their instrument. But if Names are attached to AVResources, then if we need to point AVResources at different instruments (like if we decide to split "guitar" into "acoustic guitar" and "electric guitar", or if someone mistakenly uploads a picture of a violin to the "viola" page), then the names can come along with the AVResource.

[jacobdgm]

...but if Names are attached to AVResources, they should probably not be directly connected to Instruments, but only connected via the AVResource...

Not sure if I'm making sense - probably getting too deep in the weeds here...

[dchiller]

Ah, good points! Maybe in the weeds...we can move forward either way...

So the ultimate goal of VIM (as I think about it) is to make sure that good InstrumentNames are connected to Instruments, because this is eventually what ends up in Wikidata. The pictures are tools to that end (if I only know instrument names in my language, then a photo of a clarinet is much more useful to me than the word "clarinet" in helping me identify the instrument). Ultimately, an InstrumentName has a relationship to a Instrument, not to a particular instantiation of an instrument or part of an instrument that might be visible/audible in an AVResource. So, I think our instinct to have Instrument and InstrumentName be attached is the right one.

if we need to point AVResources at different instruments (like if we decide to split "guitar" into "acoustic guitar" and "electric guitar", or if someone mistakenly uploads a picture of a violin to the "viola" page), then the names can come along with the AVResource.

True. I think if (when, really) we have to do this kind of clean-up and modification, it is going to be inevitably tricky and I don't know that attaching InstrumentName to AVResource is going to solve that. It seems just as likely to me that someone uploads a picture of a violin as a viola and names it the "right" thing (ie. a "viola") as them uploading a picture of a violin as a viola and calling it the "wrong" think (ie. a "violin"). In the former case, it doesn't do us any more good in cleaning that up to have the AVResource attached to the InstrumentName.

So my thought in having this (optional) connection between an AVResource and an InstrumentName is really about provenance and sourcing: is it useful or interesting to track in some way that a name for an instrument was provided with a particular instance of that instrument in the form of, let's say, a photo. A situation: someone sees some photos of a viola and says "In Greek, we call this X" and someone else comes along and sees those same photos and says "No no, in Greek, we call this Y". Fine...maybe they have different interpretations of the picture, maybe they are using different aliases, etc. We can't and shouldn't (generally) adjudicate that. But if person 2 above have said, "No no, in Greek, we call this Y and also here's a photo of Y," it might be interesting to record that that AVResource was specifically introduced as "Y", even if that AVResource and the InstrumentName "Y" are connected to the viola Instrument.

Again, I don't think any of this is crucial, and I think rather than over-complicate we just leave it out for now.