Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UTF-8 Normalization #351

Open
SamuelYvon opened this issue Jan 12, 2024 · 1 comment
Open

UTF-8 Normalization #351

SamuelYvon opened this issue Jan 12, 2024 · 1 comment

Comments

@SamuelYvon
Copy link
Contributor

Hey!

Can we assume the input is UTF-8 normalized? Many SIMD-powered implementation, or implementations that do not rely on building Strings right away will assume the input is UTF-8 normalized. Is this a valid assumption?

@sharpobject
Copy link

Yes, you can assume that byte-wise different strings are different.

Because normalization is about equivalence of sequences of code-points and because the reference solution does not contain any code to normalize the city names, I think that the reference solution treats equivalent names represented by different sequences of code-points as unequal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants