Is it possible to get the offsets of sentences in the text?

[hehuan2112]

I'm trying to get the sentences with detailed offsets for downstream task.
For example, I'd like to get the sentences from the below text:

It is a fox. It is a fox. Is that a fox?

With the current code, I can get the three sentences:

const text = 'It is a fox. It is a fox. Is that a fox?';
const doc = nlp.readDoc( text );
const sentences = doc.sentences().out();
console.log( sentences );

(3) ['It is a fox.', 'It is a fox.', 'Is that a fox?']

And in my case, I need to get the offsets of each sentence in original text.
For example:

0: {text: 'It is a fox.', start:0, end: 12}
1: {text: 'It is a fox.', start: 13, end: 25}
2: {text: 'Is that a fox?', start: 26, end: 40}

I'm wondering if there is any parameter or setting I could use to get those offsets?
Thank you :)

[rachnachakraborty]

Hi @hehuan2112

Sure, you can use its.span for sentence offsets.

const sentences = doc.sentences().out( its.span );

returns the following

[ [ 0, 4 ], [ 5, 9 ], [ 10, 14 ] ]

Thanks

[rachnachakraborty]

Hi @hehuan2112

The earlier solution was for extracting span of tokens. Sharing a sample code with a slight change in the text to extract the span of sentences in a document.

const text = '      It is a fox.   It    is a fox.Is that a fox? Is it?';
const doc = nlp.readDoc( text );
curIndex = 0;
const spans = [];
doc.sentences().each( ( s, k ) => {
  const sk = s.out();
  curIndex += s.tokens().itemAt(0).out(its.precedingSpaces).length;
  const start = curIndex;
  const end = start + sk.length - 1;
  curIndex = end + 1;
  spans.push( { text: sk, start: start, end: end } );
} );
console.log( spans )

This will give the following output:

These are the text spans:

Hope this helps!

Cheers,
Rachna

Note: If you intend to use web model with NodeJS, please use Node Ver. 16 and above.

[hehuan2112]

Wow! Thank you so much for the detailed reply and example!
It works so good! I will try to integrate the solution into my code.

Thanks again! Have a great weekend! :)