Convert InteropInterface<IEnumerator> to Array for RpcClient to digest

[joeqian10]

Close #230

Any suggestions and modifications are welcome.

[shargon]

If we change the items in the same array, we should not return an array, because it's the same.

[Tommo-L]

Could add a StorageMap type here, which used in nep11, see neo-project/neo-devpack-dotnet#268

[Tommo-L]

This line is too line to read, it's better to split into multiple lines.

[ProDog]

If the contract contains return Enumerator<byte[]> , an exception will be thrown when call invokefunction.

Contract like:

        public static Enumerator<byte[]> GetEnumerator()
        {
            var a = new byte[] { 0x22, 0x32, 0x24 };
            var b = new byte[] { 0x10, 0x11, 0x22 };
            var c = new byte[] { 0x33, 0x44, 0x55 };
            var enumerator = Enumerator<byte[]>.Create(new byte[3][] { a, b, c });
            
            return enumerator;
        }

invokefunction:

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "invokefunction",
  "params": [
    "0xa02a4bb9dbadb03b41507efe1910ecdfb7d3d146",
    "getEnumerator"
  ]
}

result:

{
    "jsonrpc": "2.0",
    "id": 1,
    "error": {
        "code": -2147024809,
        "message": "StackItemType(Buffer) is not supported to ContractParameter.",
        "data": "   at Neo.VM.Helper.ToParameter(StackItem item, List`1 context) in D:\\Work\\neo_code\\neo\\src\\neo\\VM\\Helper.cs:line 340\r\n   at Neo.VM.Helper.<>c__DisplayClass16_0.<ToParameter>b__0(StackItem p) in D:\\Work\\neo_code\\neo\\src\\neo\\VM\\Helper.cs:line 289\r\n   at System.Linq.Enumerable.SelectEnumerableIterator`2.ToList()\r\n   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)\r\n   at Neo.VM.Helper.ToParameter(StackItem item, List`1 context) in D:\\Work\\neo_code\\neo\\src\\neo\\VM\\Helper.cs:line 289\r\n   at Neo.VM.Helper.ToParameter(StackItem item) in D:\\Work\\neo_code\\neo\\src\\neo\\VM\\Helper.cs:line 271\r\n   at Neo.Plugins.RpcServer.<>c.<GetInvokeResult>b__31_0(StackItem p)\r\n   at System.Linq.Enumerable.SelectArrayIterator`2.MoveNext()\r\n   at System.Collections.Generic.List`1.InsertRange(Int32 index, IEnumerable`1 collection)\r\n   at Neo.IO.Json.JArray..ctor(IEnumerable`1 items) in D:\\Work\\neo_code\\neo\\src\\neo\\IO\\Json\\JArray.cs:line 18\r\n   at Neo.Plugins.RpcServer.GetInvokeResult(Byte[] script, IVerifiable checkWitnessHashes)\r\n   at Neo.Plugins.RpcServer.InvokeFunction(JArray _params)\r\n   at Neo.Plugins.RpcServer.ProcessRequest(HttpContext context, JObject request)"
    }
}

[gsmachado]

just to mention that this is pretty important to us since many of our tests are failing at the moment (we had to ignore them) due to this. 😄

[joeqian10]

This PR also closes #254

Now the GetInvokeResult method directly converts a StackItem to json, so there will no longer be ByteArray type in the response.

[ProDog]

Please review this PR.@erikzhang

[erikzhang]

If the enumerator returns too many items, it may cause out of memory exception. That's why we need enumerator instead of using array directly.

[erikzhang]

Paging does not solve all problems. Because if the user requests the last page, you have to traverse all the items.

[joeqian10]

@erikzhang @shargon @gsmachado @Tommo-L Got any other ideas? 🤔😂

[erikzhang]

Currently the only way I can think of is to modify the API of the contract so that the results returned by the contract support paging natively. But one problem I can't solve this way is that Storage.Find() itself cannot support paging. So the contract should also be powerless.

[erikzhang]

So I don't think RPC should do the conversion anymore.

[shargon]

Maybe we should limit to the X first registers, and allow more by config. Then you can allow to your own node to return whole data

[Tommo-L]

Maybe we should limit to the X first registers, and allow more by config. Then you can allow to your own node to return whole data

I think it's okay.

But one problem I can't solve this way is that Storage.Find() itself cannot support paging.

We can create a StorageOrderedMap, the key of map is prefix + number + key => value, when we want to delete one key:value, we just move the last element to the current position. But the implementation is a little complicated.

[shargon]

maxPage in config?