Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Guard malloc in compression filters #1152

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from
Draft

Guard malloc in compression filters #1152

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
504ce58
Guard malloc in compression filters
mkitti May 27, 2024
b709574
Update filters/H5Zblosc/src/H5Zblosc.jl
mkitti May 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions filters/H5Zbitshuffle/src/H5Zbitshuffle.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,7 @@ function H5Z_filter_bitshuffle(
end

size = nbytes_uncomp ÷ elem_size
buf_size_out <= 0 && error("bitshuffle_h5plugin: Non-positive buf_size_out for malloc: $buf_size_out")
out_buf = Libc.malloc(buf_size_out)
if out_buf == C_NULL
error(
Expand Down
2 changes: 2 additions & 0 deletions filters/H5Zblosc/src/H5Zblosc.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,7 @@ function blosc_filter(
# the result is larger, we simply return 0. The filter is flagged
# as optional, so HDF5 marks the chunk as uncompressed and proceeds.
outbuf_size = unsafe_load(buf_size)
outbuf_size <= 0 && return Csize_t(0)
outbuf = Libc.malloc(outbuf_size)
outbuf == C_NULL && return Csize_t(0)

Expand All @@ -121,6 +122,7 @@ function blosc_filter(
# See https://github.com/JuliaLang/julia/issues/43402
# Resolved in https://github.com/JuliaLang/julia/pull/43408
outbuf_size, cbytes, blocksize = Blosc.cbuffer_sizes(in)
outbuf_size <= 0 && return Csize_t(0)
outbuf = Libc.malloc(outbuf_size)
outbuf == C_NULL && return Csize_t(0)
status = Blosc.blosc_decompress(in, outbuf, outbuf_size)
Expand Down
2 changes: 2 additions & 0 deletions filters/H5Zbzip2/src/H5Zbzip2.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ function H5Z_filter_bzip2(
# Decompress

outbuflen = nbytes * 3 + 1
outbuflen <= 0 && error("H5Zbzip2: Non-positive outbuflen for malloc: $outbuflen.")
outbuf = Libc.malloc(outbuflen)
if outbuf == C_NULL
error("H5Zbzip2: memory allocation failed for bzip2 decompression.")
Expand Down Expand Up @@ -106,6 +107,7 @@ function H5Z_filter_bzip2(

# Prepare the output buffer
outbuflen = nbytes + nbytes ÷ 100 + 600 # worse case (bzip2 docs)
outbuflen <= 0 && error("H5Zbzip2: Non-positive outbuflen for malloc: $outbuflen.")
outbuf = Libc.malloc(outbuflen)
@debug "Allocated" outbuflen outbuf
if outbuf == C_NULL
Expand Down
6 changes: 5 additions & 1 deletion filters/H5Zlz4/src/H5Zlz4.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,9 @@ function H5Z_filter_lz4(
# malloc a byte buffer of origSize
# outBuf = Vector{UInt8}(undef, origSize)
@debug "OrigSize" origSize
origSize <= 0 && error("H5Zlz4: Non-positive origSize for malloc: $origSize")
outBuf = Libc.malloc(origSize)
outBuf == C_NULL && error("H5Zlz4: Could not allocate memory via malloc")
# Julia should throw an error if it cannot allocate this
roBuf = Ptr{UInt8}(outBuf)
decompSize = 0
Expand Down Expand Up @@ -126,7 +128,9 @@ function H5Z_filter_lz4(
nBlocks = (nbytes - 1) ÷ blockSize + 1
maxDestSize =
nBlocks * CodecLz4.LZ4_compressBound(blockSize) + 4 + 8 + nBlocks * 4
maxDestSize <= 0 && error("H5Zlz4: Non-positive maxDestSize for malloc: $maxDestSize")
outBuf = Libc.malloc(maxDestSize)
outBuf == C_NULL && error("H5Zlz4: Could not allocate memory via malloc")

rpos = Ptr{UInt8}(unsafe_load(buf))
roBuf = Ptr{UInt8}(outBuf)
Expand Down Expand Up @@ -189,7 +193,7 @@ function H5Z_filter_lz4(
catch err
# "In the case of failure, the return value is 0 (zero) and all pointer arguments are left unchanged."
ret_value = Csize_t(0)
@error "H5Zlz4.jl Non-Fatal ERROR: " err
@async @error "H5Zlz4.jl Non-Fatal ERROR: " err
display(stacktrace(catch_backtrace()))
finally
if outBuf != C_NULL
Expand Down
Loading