Merge branch 'master' into angular19

sign · Dec 9, 2024 · aa1e770 · aa1e770
2 parents 6f0abac + 42d5ac4
commit aa1e770
Show file tree

Hide file tree

Showing 8 changed files with 198 additions and 75 deletions.
diff --git a/docs/.vitepress/config.mts b/docs/.vitepress/config.mts
@@ -83,6 +83,20 @@ export default withMermaid(
               },
             ],
           },
+          {
+            text: 'Companies',
+            collapsed: false,
+            items: [
+              {
+                text: 'State of the Art',
+                link: '/docs/companies/state-of-the-art',
+                items: [
+                  {text: 'Spoken to Signed', link: '/docs/companies/state-of-the-art#spoken-to-signed-translation'},
+                  {text: 'Signed to Spoken', link: '/docs/companies/state-of-the-art#signed-to-spoken-translation'},
+                ],
+              },
+            ],
+          },
         ],
       },
 

diff --git a/docs/README.md b/docs/README.md
@@ -0,0 +1,14 @@
+# `sign`/docs
+
+## Contribute
+
+Contributions are welcome.
+You can edit files directly on GitHub ([Tutorial](https://docs.github.com/en/repositories/working-with-files/managing-files/editing-files)),
+or clone the repository to get an instant preview using:
+
+```bash
+git clone https://github.com/sign/translate.git
+cd translate/docs
+npm install
+npm run docs:dev
+```
diff --git a/docs/docs/companies/examples/dall-e-3.webp b/docs/docs/companies/examples/dall-e-3.webp
diff --git a/docs/docs/companies/examples/stable-diffusion-3-5.jpeg b/docs/docs/companies/examples/stable-diffusion-3-5.jpeg
diff --git a/docs/docs/companies/state-of-the-art.md b/docs/docs/companies/state-of-the-art.md
@@ -0,0 +1,69 @@
+# State of the Art
+
+We are often asked to explain why we are specifically building sign language technology,
+given that large language models are currently improving rapidly.
+Therefore, this document aims to provide a high-level overview of the current state of the art in popular vision-language models.
+
+## Spoken to Signed Translation
+
+While video generation models are still in the horizon, image generation models are already mature.
+Thus, we start by testing image generation models to generate an image of a sign.
+
+### OpenAI: [DALL-E 3](https://openai.com/index/dall-e-3/) via ChatGPT 4o
+
+:::tip ChatGPT 4o Prompt
+Sign language interpreter, green screen background, signing the American Sign Language sign for "House".
+:::
+
+To trigger DALL-E 3, ChatGPT 4o generates a new prompt:
+:::info DALL-E 3 Prompt
+A professional sign language interpreter signing the American Sign Language (ASL) sign for 'House' with clear and accurate hand positioning. The background is a plain green screen, suitable for chroma keying. The interpreter is dressed in neutral, professional attire, and the scene is well-lit to ensure clarity of the sign and facial expressions.
+:::
+
+![DALL-E 3 Image Generated using ChatGPT 4o Prompt](./examples/dall-e-3.webp)
+
+The generated image is not a valid representation of the sign for "House".
+The hands are with the wrong hand shapes, and they are not in the correct positions.
+
+### StabilityAI: [Stable Diffusion 3.5 Large](https://huggingface.co/stabilityai/stable-diffusion-3.5-large)
+
+:::info Prompt
+Sign language interpreter, green screen background, signing the American Sign Language sign for "House".
+:::
+
+![Stable Diffusion 3.5 Large Image Generated using Prompt](./examples/stable-diffusion-3-5.jpeg)
+
+The generated image is not a valid representation of the sign for "House".
+The hands are with the wrong hand shapes, and they are not in the correct positions.
+
+## Signed to Spoken Translation
+
+### OpenAI
+
+OpenAI does not currently support video inputs.
+
+### Meta
+
+Meta is conducting research on signed language translation, but their models are not publicly available.[^privacy-aware]
+[^privacy-aware]: Rust et al. 2024. [Towards Privacy-Aware Sign Language Translation at Scale](https://arxiv.org/abs/2402.09611).
+
+### Google
+
+Google is conducting research on signed language translation, but their models are not publicly available.[^scaling-slt]
+[^scaling-slt]: Zhang et al. 2024. [Scaling Sign Language Translation](https://arxiv.org/abs/2407.11855).
+
+What is available, is [Gemini 1.5 Pro](https://console.cloud.google.com/vertex-ai/studio/freeform?model=gemini-1.5-pro-002), which is a general-purpose model which supports video inputs.
+
+We attach a video of a woman signing "What is your name?"
+
+:::info Prompt + Attached video "example-sentence.mp4"
+Translate the attached American Sign Language video "example-sentence.mp4" into English.
+:::
+
+Responses:
+
+1. The woman in the video signs the following: "I'm sorry, I don't understand."
+2. The signer is saying: "Excuse me. I have a question."
+3. The woman in the video signs the following: "Hello, my name is `[name]`. Nice to meet you."
+4. The signer is saying: "Excuse me. Do you mind if I sit here?"
+5. The signer is saying: "Excuse me. Do you have a second? Do you mind if I ask you a question?"
diff --git a/docs/docs/facts/market.md b/docs/docs/facts/market.md
diff --git a/docs/docs/facts/population.md b/docs/docs/facts/population.md
@@ -17,7 +17,7 @@ sankey-beta
   World Population, Hearing, 6500000000
   World Population, Hearing Loss, 1500000000
   Hearing Loss, Disabling Hearing Loss, 430000000
-  Disabling Hearing Loss, Deaf, 72000000
+  Disabling Hearing Loss, Deaf, 70000000
 ```
 
 ### Hearing Loss[^who]
@@ -27,7 +27,7 @@ Of these, **430 million people** experience disabling hearing loss, requiring re
 By 2050, this number is projected to grow to **700 million**.
 
 ::: info Quote: Relevance ([learn more](/facts/market#economic-impact))
-Unaddressed hearing loss poses an annual global cost of US $980 billion.
+Unaddressed hearing loss poses an annual global cost of US\$ 980 billion.
 This includes health sector costs (excluding the cost of hearing devices), costs of educational support, loss of productivity and societal costs.
 :::
 
@@ -71,24 +71,25 @@ In Israel, a survey conducted among the organization's members found that 67% of
 
 We list the number of interpreters in various countries below. For all countries in Europe, visit https://www.eud.eu/member-countries/.
 
-| Country            | Population            | HoH Population                                              | Deaf Population                            | Interpreters                                                                        | Deaf Interpreters                           |
-| ------------------ | --------------------- | ----------------------------------------------------------- | ------------------------------------------ | ----------------------------------------------------------------------------------- | ------------------------------------------- |
-| United States      | 331 million (2023)    | 11[^disabilitystatistics]-12[^researchondisability] million | 1 million[^mitchell]                       | 10,385[^rid]                                                                        |                                             |
-| India              | 1.4 billion (2023)    |                                                             | 2 million[^wfdeaf-sig]                     | 300[^wfdeaf-sig]                                                                    |                                             |
-| Germany            | 83 million[^aiic-ch]  |                                                             | 200,000[^ddl]                              | 300[^aiic-ch]                                                                       | 26-30[^ddl]                                 |
-| Switzerland        | 8.7 million[^aiic-ch] | 800,000[^swissinfo]                                         | 10,000[^swissinfo][^ddl][^eud-switzerland] | 270[^aiic-ch] (63 Swiss-German, 32 Swiss-French, 8 Swiss-Italian)[^eud-switzerland] | (1-5 Swiss-German, 1-5 Swiss-Italian)[^ddl] |
-| Israel             | 9.7 million (2023)    | 55,700[^cbs-isl]                                            | 10,000[^isl-deaf]                          | 147[^isl-interpreters]-200[^isl-shortage]                                           |                                             |
-| The United Kingdom |                       |                                                             | 77,000[^ddl]                               |                                                                                     | 16-20[^ddl]                                 |
-| Slovenia           |                       |                                                             | 1016[^eud-slovenia]                        | 61[^eud-slovenia]                                                                   | 2-5[^ddl]                                   |
-| Spain              |                       |                                                             | 100,000[^eud-spain]                        | 667[^eud-spain]                                                                     | <15[^ddl]                                   |
-| Austria            |                       |                                                             | 8,000 – 10,000[^ddl][^eud-austria]         | 150[^eud-austria]                                                                   | 5[^ddl]                                     |
-| Belgium            |                       |                                                             | 6,500[^eud-belgium] - 9,000[^ddl]          | 165[^eud-belgium]                                                                   | 16-26[^ddl] (5-10 Flanders, 16-26 Wallonia) |
+| Country            | Population            | HoH Population                                              | Deaf Population                            | Interpreters                                                        | Deaf Interpreters                           |
+| ------------------ | --------------------- | ----------------------------------------------------------- | ------------------------------------------ | ------------------------------------------------------------------- | ------------------------------------------- |
+| United States      | 331 million (2023)    | 11[^disabilitystatistics]-12[^researchondisability] million | 1 million[^mitchell]                       | 10,385[^rid]                                                        |                                             |
+| India              | 1.4 billion (2023)    |                                                             | 2 million[^wfdeaf-sig]                     | 300[^wfdeaf-sig]                                                    |                                             |
+| Germany            | 83 million[^aiic-ch]  |                                                             | 200,000[^eud-germany][^ddl]                | 750[^eud-germany]                                                   | 26-30[^ddl]                                 |
+| Switzerland        | 8.7 million[^aiic-ch] | 800,000[^swissinfo]                                         | 10,000[^swissinfo][^ddl][^eud-switzerland] | 63 Swiss-German, 32 Swiss-French, 8 Swiss-Italian[^eud-switzerland] | (1-5 Swiss-German, 1-5 Swiss-Italian)[^ddl] |
+| Israel             | 9.7 million (2023)    | 55,700[^cbs-isl]                                            | 10,000[^isl-deaf]                          | 147[^isl-interpreters]-200[^isl-shortage]                           |                                             |
+| The United Kingdom |                       |                                                             | 77,000[^ddl]                               |                                                                     | 16-20[^ddl]                                 |
+| Slovenia           |                       |                                                             | 1016[^eud-slovenia]                        | 61[^eud-slovenia]                                                   | 2-5[^ddl]                                   |
+| Spain              |                       |                                                             | 100,000[^eud-spain]                        | 667[^eud-spain]                                                     | <15[^ddl]                                   |
+| Austria            |                       |                                                             | 8,000 – 10,000[^ddl][^eud-austria]         | 150[^eud-austria]                                                   | 5[^ddl]                                     |
+| Belgium            |                       |                                                             | 6,500[^eud-belgium] - 9,000[^ddl]          | 165[^eud-belgium]                                                   | (5-10 Flanders, 16-26 Wallonia)[^ddl]       |
 
 [^eud-slovenia]: European Union of the Deaf. 2024. [Member Countries: Slovenia](https://www.eud.eu/member-countries/slovenia/).
 [^eud-spain]: European Union of the Deaf. 2024. [Member Countries: Spain](https://www.eud.eu/member-countries/spain/).
 [^eud-switzerland]: European Union of the Deaf. 2024. [Member Countries: Switzerland](https://www.eud.eu/member-countries/switzerland/).
 [^eud-austria]: European Union of the Deaf. 2024. [Member Countries: Austria](https://www.eud.eu/member-countries/austria/).
 [^eud-belgium]: European Union of the Deaf. 2024. [Member Countries: Belgium](https://www.eud.eu/member-countries/belgium/).
+[^eud-germany]: European Union of the Deaf. 2024. [Member Countries: Germany](https://www.eud.eu/member-countries/germany/).
 [^ddl]: Danish Deaf Association. 2016. [Deaf Interpreters in Europe](https://ddl.dk/wp-content/uploads/Deaf-interpreters-in-Europe-2016.pdf).
 [^isl-shortage]: Omer Sharvit. 2022. [מחסור חמור במתורגמנים לשפת הסימנים](https://www.zman.co.il/285475/popup).
 [^cbs-isl]: Central Bureau of Statistics, 2024. [Data on People with Hearing Impairments in Israel on the Occasion of World Hearing Day 2024](https://www.cbs.gov.il/he/mediarelease/DocLib/2024/065/11_24_065b.pdf).

diff --git a/docs/package.json b/docs/package.json
@@ -8,7 +8,7 @@
     "docs:build": "vitepress build",
     "docs:preview": "vitepress preview"
   },
-  "license": "MIT",
+  "license": "CC BY 4.0",
   "dependencies": {
     "mermaid": "11.4.1"
   },