Is it possible to hash bytes into a hasher returned by recursion.NewShort?

[aayux]

I previously raised #1249 because I believed that the writer for the recursion.NewShort hasher was broken, but after looking more into it, I was probably just using it wrong in the example I gave.

[TLDR] I want to write some marshalled G1 elements (Groth16 verification key elements) as well as a byte string into a hash.

I am able to do the former as

H, _ := recursion.NewShort(ecc.BW6_761.ScalarField(), ecc.BW6_761.ScalarField())
for _, k := range izkvk.(*groth16_bls12377.VerifyingKey).G1.K {
    marshal := k.Marshal()
    H.Write(marshal)
}
digest := H.Sum(nil)

and in the circuit,

H, err := recursion.NewHash(api, ecc.BW6_761.ScalarField(), true)
if err != nil {
	return fmt.Errorf("new hash: %w", err)
}
curve, err := algebra.GetCurve[FR, G1El](api)
if err != nil {
	return fmt.Errorf("get curve: %w", err)
}

for _, k := range c.InnerVerKey.G1.K {
	marshal := curve.MarshalG1(k)
	H.Write(marshal...)
}

digest := H.Sum()

and this works as expected on its own. The problem is that when I try to write a (even arbitrary) byte string into the hash after the G1 elements (and correspondingly in the circuit), a constraint check fails

ERR error="constraint #7331 is not satisfied: 1755361025 ⋅ 258664426012969094010652733694893533536393512754914660539884262666720468348340822774968888139573360124438566097153 != 0" nbConstraints=43252
panic: proving failed: constraint #7331 is not satisfied: 1755361025 ⋅ 258664426012969094010652733694893533536393512754914660539884262666720468348340822774968888139573360124438566097153 != 0

[AlexandreBelling]

The problem plausibly stems from the fact that the byte string that you are attempting to write has an arbitrary size. Constraint-systems (both for r1cs and scs) do not trivially support this because every different "size" will yield a different circuit.

It would be helpful for us if you could provide the code that you use when hashing the arbitrary byte string. Also, I would like to know how you initialize it.

From what you wrote, I can propose this solution:

• Instead of an arbitrary byte string, you can provide a fixed-size buffer that you only partially fill depending on how much data you have. In that case, you should also hash the effective length of the string and add constraints justifying that the length of the string is consistent with the buffer. If the buffer is part of the witness (i.e. it is a field of a sub-field of the Circuit struct) you should preallocate it before calling frontend.Compile. Otherwise, the gnark framework will assume that the buffer has a size of zero.

[aayux]

Ah, that makes sense.

To answer your question: the value I'm trying to write into the hash is the current time obtained as

t := time.Now().Unix()
tBytes := make([]byte, 8)
binary.BigEndian.PutUint64(tBytes, uint64(t))

which I then (try to) write as

H, _ := recursion.NewShort(ecc.BW6_761.ScalarField(), ecc.BW6_761.ScalarField())
for _, k := range izkvk.(*groth16_bls12377.VerifyingKey).G1.K {
    marshal := k.Marshal()
    H.Write(marshal)
}
H.Write(tBytes)
digest := H.Sum(nil)

[ivokub]

@aayux - we need to take some care when handling different elements from different fields. See the following full example:

package discussion1250

import (
	"fmt"
	"testing"
	"time"

	"github.com/consensys/gnark-crypto/ecc"
	fr_bw6761 "github.com/consensys/gnark-crypto/ecc/bw6-761/fr"
	groth16_bw6761 "github.com/consensys/gnark/backend/groth16/bw6-761"
	"github.com/consensys/gnark/frontend"
	"github.com/consensys/gnark/std/algebra"
	"github.com/consensys/gnark/std/algebra/emulated/sw_bw6761"
	"github.com/consensys/gnark/std/math/emulated"
	"github.com/consensys/gnark/std/recursion"
	"github.com/consensys/gnark/std/recursion/groth16"
	"github.com/consensys/gnark/test"
)

type circuit struct {
	Vk       groth16.VerifyingKey[sw_bw6761.G1Affine, sw_bw6761.G2Affine, sw_bw6761.GTEl]
	Time     emulated.Element[sw_bw6761.ScalarField]
	Expected frontend.Variable `gnark:",public"`
}

func (circuit *circuit) Define(api frontend.API) error {
	h, err := recursion.NewHash(api, ecc.BW6_761.ScalarField(), true)
	if err != nil {
		return err
	}
	curve, err := algebra.GetCurve[sw_bw6761.ScalarField, sw_bw6761.G1Affine](api)
	if err != nil {
		return err
	}
	for _, k := range circuit.Vk.G1.K {
		kbits := curve.MarshalG1(k)
		h.Write(kbits...)
	}
	timeBits := curve.MarshalScalar(circuit.Time)
	h.Write(timeBits...)
	t := h.Sum()
	api.Println(t)

	api.AssertIsEqual(t, circuit.Expected)
	return nil
}

func TestAAA(t *testing.T) {
	assert := test.NewAssert(t)
	var timeElement fr_bw6761.Element
	timeElement.SetInt64(time.Now().Unix())

	vk := groth16_bw6761.VerifyingKey{}

	H, _ := recursion.NewShort(ecc.BW6_761.ScalarField(), ecc.BW6_761.ScalarField())
	for _, k := range vk.G1.K {
		marshal := k.Marshal()
		H.Write(marshal)
	}
	tBytes := timeElement.Bytes()
	H.Write(tBytes[:])
	digest := H.Sum(nil)
	fmt.Printf("%x\n", digest)

	cvk, err := groth16.ValueOfVerifyingKey[sw_bw6761.G1Affine, sw_bw6761.G2Affine, sw_bw6761.GTEl](&vk)
	assert.NoError(err)
	assignment := circuit{
		Vk:       cvk,
		Time:     emulated.ValueOf[sw_bw6761.ScalarField](timeElement),
		Expected: digest,
	}
	err = test.IsSolved(&circuit{}, &assignment, ecc.BW6_761.ScalarField())
	assert.NoError(err)
}

[ivokub]

So, the thing to keep in mind is that in-circuit we always work with field elements, even if in the witness you want to use byte strings. So you have to manually set the field elements from your byte strings. And in case your byte string is longer than what the field element representation can carry, then you have to use multiple elements.