Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

statemanager: small refactor to get storage objects for snapsync #3033

Merged
merged 1 commit into from
Sep 20, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 70 additions & 58 deletions packages/statemanager/src/stateManager.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@

import type { AccountFields, EVMStateManagerInterface, StorageDump } from '@ethereumjs/common'
import type { StorageRange } from '@ethereumjs/common/src'
import type { PrefixedHexString } from '@ethereumjs/util'
import type { DB, PrefixedHexString } from '@ethereumjs/util'
import type { Debugger } from 'debug'
const { debug: createDebugLogger } = debugDefault

Expand Down Expand Up @@ -342,7 +342,7 @@
}

const key = this._prefixCodeHashes ? concatBytes(CODEHASH_PREFIX, codeHash) : codeHash
await this._trie.database().put(key, value)
await this._getCodeDB().put(key, value)

const keyHex = bytesToUnprefixedHex(key)
this._codeCache[keyHex] = value
Expand Down Expand Up @@ -378,7 +378,7 @@
if (keyHex in this._codeCache) {
return this._codeCache[keyHex]
} else {
const code = (await this._trie.database().get(key)) ?? new Uint8Array(0)
const code = (await this._getCodeDB().get(key)) ?? new Uint8Array(0)
this._codeCache[keyHex] = code
return code
}
Expand All @@ -389,7 +389,7 @@
* cache or does a lookup.
* @private
*/
protected async _getStorageTrie(address: Address, account: Account): Promise<Trie> {
protected _getStorageTrie(address: Address, account: Account): Trie {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this function does no actual async call

// from storage cache
const addressHex = bytesToUnprefixedHex(address.bytes)
const storageTrie = this._storageTries[addressHex]
Expand All @@ -406,6 +406,24 @@
return storageTrie
}

/**
* Gets the storage trie for an account from the storage
* cache or does a lookup.
* @private
*/
protected _getAccountTrie(): Trie {
return this._trie
}

Check warning on line 416 in packages/statemanager/src/stateManager.ts

View check run for this annotation

Codecov / codecov/patch

packages/statemanager/src/stateManager.ts#L415-L416

Added lines #L415 - L416 were not covered by tests

/**
* Gets the storage trie for an account from the storage
* cache or does a lookup.
* @private
*/
protected _getCodeDB(): DB {
return this._trie.database()
}

/**
* Gets the storage value associated with the provided `address` and `key`. This method returns
* the shortest representation of the stored value.
Expand All @@ -431,7 +449,7 @@
if (!account) {
throw new Error('getContractStorage() called on non-existing account')
}
const trie = await this._getStorageTrie(address, account)
const trie = this._getStorageTrie(address, account)
const value = await trie.get(key)
if (!this._storageCacheSettings.deactivate) {
this._storageCache?.put(address, key, value ?? hexToBytes('0x80'))
Expand All @@ -453,7 +471,7 @@
): Promise<void> {
// eslint-disable-next-line no-async-promise-executor
return new Promise(async (resolve) => {
const storageTrie = await this._getStorageTrie(address, account)
const storageTrie = this._getStorageTrie(address, account)

modifyTrie(storageTrie, async () => {
// update storage cache
Expand Down Expand Up @@ -654,7 +672,7 @@
(p) => bytesToHex(p)
)
const storageProof: StorageProof[] = []
const storageTrie = await this._getStorageTrie(address, account)
const storageTrie = this._getStorageTrie(address, account)

for (const storageKey of storageSlots) {
const proof = (await storageTrie.createProof(storageKey)).map((p) => bytesToHex(p))
Expand Down Expand Up @@ -802,23 +820,21 @@
if (!account) {
throw new Error(`dumpStorage f() can only be called for an existing account`)
}
const trie = this._getStorageTrie(address, account)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

most of the diff here is because of pulling this out of the promise below


return new Promise((resolve, reject) => {
this._getStorageTrie(address, account)
.then((trie) => {
const storage: StorageDump = {}
const stream = trie.createReadStream()
const storage: StorageDump = {}
const stream = trie.createReadStream()

stream.on('data', (val: any) => {
storage[bytesToHex(val.key)] = bytesToHex(val.value)
})
stream.on('end', () => {
resolve(storage)
})
})
.catch((e) => {
reject(e)
})
stream.on('data', (val: any) => {
storage[bytesToHex(val.key)] = bytesToHex(val.value)
})
stream.on('end', () => {
resolve(storage)
})
stream.on('error', (e) => {
reject(e)

Check warning on line 836 in packages/statemanager/src/stateManager.ts

View check run for this annotation

Codecov / codecov/patch

packages/statemanager/src/stateManager.ts#L836

Added line #L836 was not covered by tests
})
})
}

Expand All @@ -841,48 +857,44 @@
if (!account) {
throw new Error(`Account does not exist.`)
}
const trie = this._getStorageTrie(address, account)

Check warning on line 860 in packages/statemanager/src/stateManager.ts

View check run for this annotation

Codecov / codecov/patch

packages/statemanager/src/stateManager.ts#L860

Added line #L860 was not covered by tests
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

most of the diff here is because of pulling this out of the promise below


return new Promise((resolve, reject) => {
this._getStorageTrie(address, account)
.then((trie) => {
let inRange = false
let i = 0

/** Object conforming to {@link StorageRange.storage}. */
const storageMap: StorageRange['storage'] = {}
const stream = trie.createReadStream()

stream.on('data', (val: any) => {
if (!inRange) {
// Check if the key is already in the correct range.
if (bytesToBigInt(val.key) >= startKey) {
inRange = true
} else {
return
}
}

if (i < limit) {
storageMap[bytesToHex(val.key)] = { key: null, value: bytesToHex(val.value) }
i++
} else if (i === limit) {
resolve({
storage: storageMap,
nextKey: bytesToHex(val.key),
})
}
})
let inRange = false
let i = 0

/** Object conforming to {@link StorageRange.storage}. */
const storageMap: StorageRange['storage'] = {}
const stream = trie.createReadStream()

stream.on('data', (val: any) => {
if (!inRange) {
// Check if the key is already in the correct range.
if (bytesToBigInt(val.key) >= startKey) {
inRange = true
} else {
return
}
}

Check warning on line 878 in packages/statemanager/src/stateManager.ts

View check run for this annotation

Codecov / codecov/patch

packages/statemanager/src/stateManager.ts#L863-L878

Added lines #L863 - L878 were not covered by tests

stream.on('end', () => {
resolve({
storage: storageMap,
nextKey: null,
})
if (i < limit) {
storageMap[bytesToHex(val.key)] = { key: null, value: bytesToHex(val.value) }
i++
} else if (i === limit) {
resolve({
storage: storageMap,
nextKey: bytesToHex(val.key),

Check warning on line 886 in packages/statemanager/src/stateManager.ts

View check run for this annotation

Codecov / codecov/patch

packages/statemanager/src/stateManager.ts#L880-L886

Added lines #L880 - L886 were not covered by tests
})
}
})

stream.on('end', () => {
resolve({
storage: storageMap,
nextKey: null,

Check warning on line 894 in packages/statemanager/src/stateManager.ts

View check run for this annotation

Codecov / codecov/patch

packages/statemanager/src/stateManager.ts#L888-L894

Added lines #L888 - L894 were not covered by tests
})
.catch((e) => {
reject(e)
})
})
stream.on('error', (e) => reject(e))

Check warning on line 897 in packages/statemanager/src/stateManager.ts

View check run for this annotation

Codecov / codecov/patch

packages/statemanager/src/stateManager.ts#L896-L897

Added lines #L896 - L897 were not covered by tests
})
}

Expand Down
Loading