How to Apply Different Quantization Settings Per Layer in ExecuTorch?

[crinex]

Dear @kimishpatel @jerryzh168 @shewu-quic

I want to split a model(eg, Llama-3.2-3B) into multiple layers and apply different quantization settings(qnn_8a8w, qnn_16a4w...) to each layer.
Has such a method been tested in ExecuTorch?
If not, could you suggest how this can be achieved?

Thank you

[kimishpatel]

Is this specific to qnn backend or your question is in general?

[crinex]

@kimishpatel

I meant was about the QNN backend.
I’ve noticed that when I quantize the LLaMA 3.2-3B model using QNN_8A8W, inference doesn’t work properly on the device. It seems that others are reporting similar results in issues as well.
Therefore, I’m considering solving this problem using mixed precision on the QNN backend. Would that be possible?