Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JCR-4369: Avoid S3 Incomplete Read Warning by elegant aborting #61

Open
wants to merge 7,994 commits into
base: trunk
Choose a base branch
from
Open

JCR-4369: Avoid S3 Incomplete Read Warning by elegant aborting #61

wants to merge 7,994 commits into from

Conversation

woonsan
Copy link
Contributor

@woonsan woonsan commented Sep 5, 2018

AWS S3 SDK recommends to abort the S3ObjectInputStream if not intended to consume the data because the http connection pool behavior (by consuming data on close to reuse the connection) of HttpClient under the hood could cause a big performance issue when reading big amount of data. By aborting, it's better to simply abort the underlying HttpRequestBase and kick out the connection from the pool from AWS S3 SDK perspective.

In multi-threaded working environment (due to multiple requests and/or proactiveCaching mode of CachingDataStore), the reading and storing actions in o.a.j.c.data.CachingDataStore.getStream(DataIdentifier) results in falling in the else block of o.a.j.core.data.LocalCache.store(String, InputStream) while the file by the name could already exist when executing the else block. In that case, the S3ObjectInputStream is never read and aborted. As a result, com.amazonaws.services.s3.internal.S3AbortableInputStream#close() ends up complaining about non-aborted/non-read-fully input stream.

Therefore, my fix includes the following:

reschke and others added 30 commits January 7, 2017 10:52
…and minin type based inheritance. Currently it appears that only primaryType based inheritance is working

git-svn-id: https://svn.apache.org/repos/asf/jackrabbit/trunk@1780335 13f79535-47bb-0310-9956-ffa450edef68
…and minin type based inheritance. Currently it appears that only primaryType based inheritance is working

git-svn-id: https://svn.apache.org/repos/asf/jackrabbit/trunk@1780336 13f79535-47bb-0310-9956-ffa450edef68
git-svn-id: https://svn.apache.org/repos/asf/jackrabbit/trunk@1780384 13f79535-47bb-0310-9956-ffa450edef68
- Using SHA-256 instead of SHA-1 to create blob record ids


git-svn-id: https://svn.apache.org/repos/asf/jackrabbit/trunk@1785225 13f79535-47bb-0310-9956-ffa450edef68
Candidate Release Notes

git-svn-id: https://svn.apache.org/repos/asf/jackrabbit/trunk@1785264 13f79535-47bb-0310-9956-ffa450edef68
create valid names / do sanity check in test case

git-svn-id: https://svn.apache.org/repos/asf/jackrabbit/trunk@1786325 13f79535-47bb-0310-9956-ffa450edef68
reschke and others added 29 commits July 24, 2018 19:41
… API

Patch provided by Alexander Klimetschek, with review and contribution from Julian Reschke

git-svn-id: https://svn.apache.org/repos/asf/jackrabbit/trunk@1838616 13f79535-47bb-0310-9956-ffa450edef68
…rsistence Manager

(patch by Woonsan Ko from #60)

git-svn-id: https://svn.apache.org/repos/asf/jackrabbit/trunk@1839660 13f79535-47bb-0310-9956-ffa450edef68
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants