Why does using precision: :integer here results in only 0 and 255 color values?

[Nakilon]

mask = Array.new(10){ |i| Array.new(10){ |j| 1 / Math::hypot(i-4.5, j-4.5) } }
hm = Vips::Image.new_from_file("PlayerSpawnsHeatMap.png").extract_band(0)
hm.conv(mask, precision: :integer).to_a.flatten.sort.uniq
=> [0, 255]

I just wanted to run conv faster (willing to use mask 100x100 or even bigger -- need the gradient to go far).
image: https://drive.google.com/file/d/1TlRKhZdWZpbKIEXTj4sOxV5HkdjDou40/view

UPD: oh, I got it. Because it's higher than 255.

hm./(255).cast("uchar").conv(mask, precision: :integer).to_a.flatten.sort.uniq
=> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]
hm.cast(:float).conv(mask, precision: :integer).to_a.flatten.sort.uniq
=> [0.0, 255.0, 510.0, 765.0, 1020.0, 1275.0, 1530.0, 1785.0, 2040.0, 2295.0, 2550.0, 2805.0, 3060.0]
hm.cast(:int).conv(mask, precision: :integer).to_a.flatten.sort.uniq
=> [0, 255, 510, 765, 1020, 1275, 1530, 1785, 2040, 2295, 2550, 2805, 3060]

P.S.: the :integer precision is too low. Have to use default (:float?), takes long. With mask 30x30 changing cluster: from 5 to even 100 does not seem to have any impact.

[jcupitt]

Hi @Nakilon,

Int convolutions should be quite a bit quicker. I see:

$ more sharpf.mat 
3 3 
-0.125 -0.125 -0.125
-0.125  2.0   -0.125
-0.125 -0.125 -0.125

ie. a basic 3x3 sharpening mask, then with a 10k x 10k image:

$ time vips conv wtc.jpg x.jpg sharpf.mat 
real	0m0.776s
user	0m5.021s
sys	0m0.193s
$ time vips conv wtc.jpg x.jpg sharpf.mat  --precision integer
real	0m0.765s
user	0m1.766s
sys	0m0.176s

Down from 5s of CPU to 1.7s. Real time doesn't change on a tiny benchmark like this, you'd need to do something more complex for it to show up. If precision doesn't change runtime for you, perhaps you're missing orc?

The cluster stuff is for approximate convolutions, which don't work very well :(

[jcupitt]

Hmm I seem to have answered a completely different question. I'll get some coffee and try again.

[jcupitt]

Int precision convolution doesn't work for large masks -- it'll fall back to the vanilla C path. You can see what it's doing with eg.:

$ time vips conv wtc.jpg x.jpg sharpf.mat --precision integer --vips-info
VIPS-INFO: 12:22:46.527: convi: using vector path

real	0m0.787s
user	0m1.881s
sys	0m0.130s

For very large masks you might be faster with an FFT. Convolution starts to become extremely slow.

Could you possibly combine the results of two 1D masks? That would help too.

[Nakilon]

For now I decided to resize image by x0.5, convolute and then resize by x2 -- that should emulate increasing mask size by x2 while being faster with no noticeable difference unlike the :integer that seems to shrink the number of unique resulting pixel values by magnitude(s).
Before and after the resize trick:


For another case, when the source 1440x1440 image has only a few (~10) non-zero pixels to emulate convolution with infinite mask I made something like this:

Vips::Image.new_from_array Array.new(1440) do |i|
  Array.new(1440) do |j|
    dots.sum{ |x, y| 1 / [Math.hypot (x-j), (y-i).min, 1].max }
  end
end