Skip to content
This repository has been archived by the owner on May 3, 2024. It is now read-only.

Commit

Permalink
Resolve todo
Browse files Browse the repository at this point in the history
  • Loading branch information
MBkkt committed Sep 20, 2023
1 parent 2350a18 commit 0c49247
Showing 1 changed file with 2 additions and 3 deletions.
5 changes: 2 additions & 3 deletions core/analysis/segmentation_token_stream.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -354,15 +354,14 @@ bool segmentation_token_stream::next() {
term.value = {reinterpret_cast<const byte_type*>(&(*begin.base())),
length};
break;
// TODO(MBkkt) do we need to call as_graphemes? Feels like no
case options_t::case_convert_t::LOWER:
term_buf_.clear();
to_lower(as_graphemes(begin, end), from_utf32_back_inserter(term_buf_));
to_lower(begin, begin, end, from_utf32_back_inserter(term_buf_));
term.value = irs::ViewCast<byte_type>(std::string_view{term_buf_});
break;
case options_t::case_convert_t::UPPER:
term_buf_.clear();
to_upper(as_graphemes(begin, end), from_utf32_back_inserter(term_buf_));
to_upper(begin, begin, end, from_utf32_back_inserter(term_buf_));
term.value = irs::ViewCast<byte_type>(std::string_view{term_buf_});
break;
}
Expand Down

0 comments on commit 0c49247

Please sign in to comment.