runtime error: integer divide by zero #209
Comments
|
We just received another similar crash, again caused by |
|
Would you retest this with the latest v2.3.x release of bleve (v2.3.10)? In situations such as these, it'd greatly help our team if you could give us a test case or some sample data with which we can deterministically reproduce the error. |
|
We received another crash, same as before, on v2.3.10: |
|
@moshaad7 would you look into this, see if this covered or separate from what you've been looking into. |
|
This is unrelated to the GetCardinality panic issue that we are working on at #259 This is essentially PostingsList.chunkSize being 0 and that is leading to divide by 0 error func (i *PostingsIterator) nextDocNumAtOrAfter(atOrAfter uint64) (uint64, bool, error) {
...
nChunk := n / uint32(i.postings.chunkSize)
...
}I see previous attempts to avoid setting chunkSize to 0 in this PR raised by @abhinavdangeti https://github.com/blevesearch/zapx/pull/151/files |
|
Here is my proposal to solve this problem: So we have a method named getChunkSize which return chunkSize and an error if any. Now in the previous PR that I mentioned , we are handling this case of chunkSize being 0 at only one place. So, we should update the implementation of getChunkSize() method to return error in case chunkSize is 0. |
The existing implementation of the method getChunkSize(...) (uint64, error), in some cases, can return chunkSize==0 and err==nil. The onus is on the caller to check for such possibility and handle it properly. Callers often use the returned chunkSize as a divisor and a zero chunkSize lead to panic. see #209 This PR intends to update the method implementation to always return an error in case the returned chunkSize value is 0. That way caller need to only worry about error being non-nil. Callers which are ok with 0 chunkSize can check the returned error against ErrChunkSizeZero
The existing implementation of the method getChunkSize(...) (uint64, error), in some cases, can return chunkSize==0 and err==nil. The onus is on the caller to check for such possibility and handle it properly. Callers often use the returned chunkSize as a divisor and a zero chunkSize lead to panic. see #209 This PR intends to update the method implementation to always return an error in case the returned chunkSize value is 0. That way caller need to only worry about error being non-nil. Callers which are ok with 0 chunkSize can check the returned error against ErrChunkSizeZero
The existing implementation of the method getChunkSize(...) (uint64, error), in some cases, can return chunkSize==0 and err==nil. The onus is on the caller to check for such possibility and handle it properly. Callers often use the returned chunkSize as a divisor and a zero chunkSize lead to panic. see #209 This PR intends to update the method implementation to always return an error in case the returned chunkSize value is 0. That way caller need to only worry about error being non-nil. Callers which are ok with 0 chunkSize can check the returned error against ErrChunkSizeZero
The existing implementation of the method getChunkSize(...) (uint64, error), in some cases, can return chunkSize==0 and err==nil. The onus is on the caller to check for such possibility and handle it properly. Callers often use the returned chunkSize as a divisor and a zero chunkSize lead to panic. see #209 This PR intends to update the method implementation to always return an error in case the returned chunkSize value is 0. That way caller need to only worry about error being non-nil. Callers which are ok with 0 chunkSize can check the returned error against ErrChunkSizeZero
The existing implementation of the method getChunkSize(...) (uint64, error), in some cases, can return chunkSize==0 and err==nil. The onus is on the caller to check for such possibility and handle it properly. Callers often use the returned chunkSize as a divisor and a zero chunkSize lead to panic. see #209 This PR intends to update the method implementation to always return an error in case the returned chunkSize value is 0. That way caller need to only worry about error being non-nil. Callers which are ok with 0 chunkSize can check the returned error against ErrChunkSizeZero
|
mattermost folks (@cpoile , @agarciamontoro , @streamer45 ), while we are adding checks to avoid "divide by zero" errors. There are several possibilities, like
And also we ( couchbase) aren't able to reproduce this issue in-house. It would be helpful, if you can try to reproduce it on staging/dummy data and share the index files to probe if there is an actual issue with segment headers etc ... Thanks. |
I am facing the same issue. Here are the files. Download Link You wont be able to see the issue if you make 1 or 2 requests. I'm using blevesearch in a webapp, i found this issue when i was doing a testing. I tried to make request to blevesearch from 100 concurrent connections. each client was making 10,000 requests. after few thousand requests, i encountered this panic. |
|
@mjm918 Thanks for your contribution Can you also share index mapping related information and your query json ? And are you sure your bleve index only have one segment ? For the given zap file, header looks fine I was exploring Dictionaries corresponding to each field. You can check if this aligns with your data :) link: |
|
Dear @mjm918, It would be greatly appreciated by all of us using Bleve and Zap if you could kindly address the questions mentioned in the previous comment. The insights you've shared so far have been incredibly helpful, and any additional information you can provide will be invaluable in tracking down this unusual bug. Thank you for your time and support! |
|
Hi Sorry for late reply.
Index Mapping - Query -
There was no ingestion while querying
No ForceMerge was done. I indexed the data, tried to do a stress test and noticed the error.
Yes it's correct. I was using a sample data database for testing. |
The existing implementation of the method getChunkSize(...) (uint64, error), in some cases, can return chunkSize==0 and err==nil. The onus is on the caller to check for such possibility and handle it properly. Callers often use the returned chunkSize as a divisor and a zero chunkSize lead to panic. see #209 This PR intends to update the method implementation to always return an error in case the returned chunkSize value is 0. That way caller need to only worry about error being non-nil. Callers which are ok with 0 chunkSize can check the returned error against ErrChunkSizeZero
This is a backport of #265. The existing implementation of the method getChunkSize(...) (uint64, error), in some cases, can return chunkSize==0 and err==nil. The onus is on the caller to check for such possibility and handle it properly. Callers often use the returned chunkSize as a divisor and a zero chunkSize lead to panic. see #209 This PR intends to update the method implementation to always return an error in case the returned chunkSize value is 0. That way caller need to only worry about error being non-nil. Callers which are ok with 0 chunkSize can check the returned error against ErrChunkSizeZero
The existing implementation of the method getChunkSize(...) (uint64, error), in some cases, can return chunkSize==0 and err==nil. The onus is on the caller to check for such possibility and handle it properly. Callers often use the returned chunkSize as a divisor and a zero chunkSize lead to panic. see #209 This PR intends to update the method implementation to always return an error in case the returned chunkSize value is 0. That way caller need to only worry about error being non-nil. Callers which are ok with 0 chunkSize can check the returned error against ErrChunkSizeZero
|
Targeting some fixes here for bleve's v2.4.3 release. |
We received a stack trace for a crash on this library:
It looks like
i.postings.chunkSizemay need some checking as it can be zero.The text was updated successfully, but these errors were encountered: