-
Notifications
You must be signed in to change notification settings - Fork 208
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added 2 features: word segment and number conversion #264
base: master
Are you sure you want to change the base?
Conversation
import java.util.regex.Matcher; | ||
import java.util.regex.Pattern; | ||
|
||
public class NumberTextConverter { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add small doc, what it does exactly.
public class NumberTextConverter { | ||
|
||
private TurkishMorphology morphology; | ||
private String patternToFind = "(\\s\\d+\\s(buçuk)\\s)|(\\s(bir|iki|üç|dört|beş|altı|yedi|sekiz|dokuz|on|yirmi|otuz|kırk|elli|atmış|altmış|yetmiş|seksen|doksan|yüz|bin|milyon|milyar)\\s(buçuk)\\s)"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can be final.
|
||
public NumberTextConverter(TurkishMorphology morphology) { | ||
this.morphology = morphology; | ||
this.halfNumberPatternT2N = Pattern.compile(patternToFind); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can be static final defined as a class parameter. No need to initialize it here.
|
||
public class NumberTextConverter { | ||
|
||
private TurkishMorphology morphology; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can be final
this.halfNumberPatternT2N = Pattern.compile(patternToFind); | ||
} | ||
|
||
public String replaceTextualNumberWithNumerically(String sentence) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Document.
String[] split = matchedPart.split(" "); | ||
String newString = ""; | ||
if(split.length > 1) { | ||
Double bucukNumber = 0.0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"halfNumber"?
return resultText; | ||
} | ||
|
||
private String convertNumberListToNumericalString(String sentence, List<List<String>> numbers){ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is rather complex, I will check later. If you add some tests, it is fine.
import zemberek.morphology.TurkishMorphology; | ||
import zemberek.normalization.NumberTextConverter; | ||
|
||
public class NumberConversion { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In examples, prefer command like names. Like "ConvertNumbers"
import java.io.IOException; | ||
import java.util.List; | ||
|
||
public class WordSegment { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"SegmentConnectedWords" or such.
import zemberek.morphology.TurkishMorphology; | ||
import zemberek.morphology.analysis.WordAnalysis; | ||
|
||
public class WordSegmenter { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add small doc. To class or public methods. If you add some tests it is fine, I will check algortihm when I have time.
Word segmenter: it uses dfs algorithm to find parts of string. It depends on Lexicon and word generation.
Number conversion: it uses morphology and disambiguation to find POS and detect numeric values for conversion.