-
Notifications
You must be signed in to change notification settings - Fork 72
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: run deal filters in parallel #1746
Conversation
storagemarket/provider_loop.go
Outdated
// set up deal handler so that clients can subscribe to deal update events | ||
dh, err := p.mkAndInsertDealHandler(deal.DealUuid) | ||
if err != nil { | ||
sendErrorResp(&acceptError{error: err, isSevereError: true, reason: "server error: starting deal thread"}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sendErrorResp(&acceptError{error: err, isSevereError: true, reason: "server error: starting deal thread"}) | |
sendErrorResp(&acceptError{error: err, isSevereError: true, reason: "server error: setting up deal handler"}) |
storagemarket/provider_loop.go
Outdated
sendErrorResp(&acceptError{error: err, isSevereError: true, reason: "server error: starting deal thread"}) | ||
continue | ||
// send an accept response | ||
dealReq.rsp <- acceptDealResp{&api.ProviderDealRejectionInfo{Accepted: true}, nil} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Specify keys in struct and avoid setting zero value fields (i.e. err
)?
dealReq.rsp <- acceptDealResp{&api.ProviderDealRejectionInfo{Accepted: true}, nil} | |
dealReq.rsp <- acceptDealResp{ri: &api.ProviderDealRejectionInfo{Accepted: true}} |
storagemarket/provider_loop.go
Outdated
} | ||
|
||
// send an accept response | ||
processedDeal.rsp <- acceptDealResp{&api.ProviderDealRejectionInfo{Accepted: true}, nil} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Ditto; see previous comment.
- Can we do something about duplicate code?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have refactored to reduce the duplicate code and fix the RSP channel responses with specific keys.
@LexLuthr I see you posted numbers on how this perform with the new code (this PR). Could you also post numbers on how the code performs on current |
@nonsense I don't see much difference when there is no external filter running. Main Current branch I have asked @cryptowhizzard to test it on his miner and check if there are any real world improvement in timeout rate. We probably need to work with CIDGravity to move to a different execution mode than os.Exec to speed up the filter. Preferably an RPC call to a local process. |
@cryptowhizzard Can you confirm if you are seeing some improvement with this PR? |
Fixes #1733
This PR moves processesOnlineDeals and processOfflineDeals to their own goroutines.
Possible side effects:
DO NOT MERGE