From 3c8494ba16f9ae85648f9624a8a715f87a19f785 Mon Sep 17 00:00:00 2001 From: Alexander Chepurnoy Date: Fri, 23 Sep 2022 19:10:16 +0300 Subject: [PATCH 1/8] initial draft --- eip-0037.md | 56 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+) create mode 100644 eip-0037.md diff --git a/eip-0037.md b/eip-0037.md new file mode 100644 index 00000000..1b888f9d --- /dev/null +++ b/eip-0037.md @@ -0,0 +1,56 @@ +Tweaking Difficulty Adjustment Algorithm +========================================== + +* Author: kushti +* Status: Proposed +* Created: 23-Sep-2022 +* Last edited: 23-Sep-2022 +* License: CC0 +* Forking: hard-fork needed + +Motivation +---------- + +Difficulty adjustment (also, readjustment) algorithm tries to stabilize averate time to generate a block by changing difficulty of solving a Proof-of-Work puzzle by observing difficulties and timestamps of historical blocks. First Proof-of-Work cryptocurrency, Bitcoin, used simplest lienar difficulty recalculation[1], with some limits for it, such as a difficulty never be changed by more than a factor of 4 either way to prevent large changes. + +Bitcoin's difficulty readjustment works more or less well since only dedicated and so loyal mining hardware working on PoW puzzlez. However, in other environments different issues were observed with it, including coin hopping. Thus different solutions to coin hopping appeared, inluding using a least squares method based predictive algorithm [3] as done in Ergo currently. + +The Ergo's algorithm works well in most cases, inluding huge price drops, 100x initial difficulty misestimation during mainnet launch, and so on. However, current simplified and limitless version of algorithm is bumpy. Big influx of mining hashrate over multiple epochs, especially with super-linear hashrate growth over time may result in huge spike of difficulty. Similarly, few slow epochs may cause huge drop. Also, for dapps and other applications it would be desirable to make difficulty readjustment more reactive (currently, readjustment takes place every 1024 blocks, and 8 epochs, so about two weeks normally, are considered). + +Related Work +------------ + +To prevent disastrous effects of hopping, widespread approach (see e.g. https://read.cash/@jtoomim/bch-upgrade-proposal-use-asert-as-the-new-daa-1d875696 ) is to use weighted averaging functions over past epochs which do prefer last epochs. +Hovewer, in case of natural hashrate migration such functions will likely be lagging (in opposite to proactive nature of predictive least square method). + +Proposed Changes +---------------- + +We propose to make current difficulty readjustment more reactive and smoother by shortening epoch length, amplifying weight of the last epoch and put some limits on difficulty change as follows. + +1. Epoch length to be set to 128 blocks. +2. We calcualate *predictive* difficulty according to 8 epochs 128 blocks each and *classic* difficulty as done in Bitcoin. We took average from them. +3. We limit change so that difficulty never be changed by more than 50% per epoch. + + +Simulations +----------- + +Previous simulations [4,5] are based on observing historical data so ignoring the fact that miners will behave differently in the presence of different difficualty adjustment method. + +Thus we made playground simulating random price walking in uptrend or downtrend, assuming that difficulty per unit of account remains stable, and also simulating spikes and drops in difficulty per unit of account. + +Activation +---------- + + +References +---------- + +1. BitcoinWiki. Difficulty in Mining https://en.bitcoinwiki.org/wiki/Difficulty_in_Mining +2. Bitcoin Wiki. Target https://en.bitcoin.it/wiki/Target#When_does_the_target_change_next.3F +3. Meshkov D., Chepurnoy A., Jansen M. Short paper: Revisiting difficulty control for blockchain systems +4. jtoomim BCH upgrade proposal: Use ASERT as the new DAA https://read.cash/@jtoomim/bch-upgrade-proposal-use-asert-as-the-new-daa-1d875696 +5. Ergo Developers. DifficultyControlSimulator.scala https://github.com/ergoplatform/ergo/blob/0af9dd9d8846d672c1e2a77f8ab29963fa5acd1e/src/test/scala/org/ergoplatform/tools/DifficultyControlSimulator.scala + + From 7f10da753cc7d7292adb112778b6c789f5aa00b1 Mon Sep 17 00:00:00 2001 From: Alexander Chepurnoy Date: Sat, 24 Sep 2022 03:01:15 +0300 Subject: [PATCH 2/8] simulations --- eip-0037.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/eip-0037.md b/eip-0037.md index 1b888f9d..b3ba676b 100644 --- a/eip-0037.md +++ b/eip-0037.md @@ -36,10 +36,12 @@ We propose to make current difficulty readjustment more reactive and smoother by Simulations ----------- -Previous simulations [4,5] are based on observing historical data so ignoring the fact that miners will behave differently in the presence of different difficualty adjustment method. +Previous simulations [4,5] are based on observing historical data so ignoring the fact that miners will behave differently in the presence of different difficulty adjustment method. Thus we made playground simulating random price walking in uptrend or downtrend, assuming that difficulty per unit of account remains stable, and also simulating spikes and drops in difficulty per unit of account. +In simulation a blockchain is being mined by rational hashpower only. Hashrate is looking at current price and diffuculty. We assume that price and difficulty are changed and the same time, and then hashrate is moving in or out, which is affecting average block generation time *t*. We may assume then that *t = d * c / p*, so average block generation time *t* is proportinal to difficulty *d* and inversely proportional to price *p* + Activation ---------- From 6f886699a11bea975a5791f0f5fb8ad18250dc51 Mon Sep 17 00:00:00 2001 From: Alexander Chepurnoy Date: Sat, 24 Sep 2022 03:02:30 +0300 Subject: [PATCH 3/8] results --- eip-0037.md | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/eip-0037.md b/eip-0037.md index b3ba676b..71bcd2e5 100644 --- a/eip-0037.md +++ b/eip-0037.md @@ -40,7 +40,35 @@ Previous simulations [4,5] are based on observing historical data so ignoring th Thus we made playground simulating random price walking in uptrend or downtrend, assuming that difficulty per unit of account remains stable, and also simulating spikes and drops in difficulty per unit of account. -In simulation a blockchain is being mined by rational hashpower only. Hashrate is looking at current price and diffuculty. We assume that price and difficulty are changed and the same time, and then hashrate is moving in or out, which is affecting average block generation time *t*. We may assume then that *t = d * c / p*, so average block generation time *t* is proportinal to difficulty *d* and inversely proportional to price *p* +In simulation a blockchain is being mined by rational hashpower only. Hashrate is looking at current price and diffuculty. We assume that price and difficulty are changed and the same time, and then hashrate is moving in or out, which is affecting average block generation time *t*. We may assume then that *t = d * c / p*, so average block generation time *t* is proportinal to difficulty *d* and inversely proportional to price *p*. Fixing *t* (which is set to target block generation time, so 2 minutes), *d* and *p* at the beginning of the experiment, we can evaluate *c*. Then on each step we randomly changing *p*, and, according to difficulty from the previous epoch, we can get block generation time for the new epoch. To have a trend in price, we are changing *p* by adding (or subtracting) a random value with fixed average, and also adding a random fluctuation. + +Test results: + +* if price growth is up to 5% per epoch, and possible fluctuation (up or down) is up to 10%: + +*Bitcoin DAA*: total error: 158841, max delay: 133 +*Current DAA*: total error: 189403, max delay: 151 +*Proposed DAA: total error: 163893, max delay: 141 + + +* if no price growth, and possible fluctuation (up or down) is up to 25% (so price is jumping up and down like crazy): + +*Bitcoin DAA*: total error: 393770, max delay: 161 +*Current DAA*: total error: 528003, max delay: 224 +*Proposed DAA: total error: 429667, max delay: 193 + +* if average price growth is up 2% per epoch (max delay missed as it is 120s max for all the options): + +*Bitcoin DAA*: total error: 30409 +*Current DAA*: total error: 19111 +*Proposed DAA: total error: 20691 + +* 3 epochs us, 3 down , 25% max change +*Bitcoin DAA*: total error: 380139, max delay: 160 +*Current DAA*: total error: 464081, max delay: 221 +*Proposed DAA*: total error: 387880, max delay: 185 + +As we can see, proposed DAA as well as current one is working better during trends, and proposed DAA softens swings better than current one. Activation ---------- From 0822f03c7ae2aa7db9716490ea88b9b74fa5fae3 Mon Sep 17 00:00:00 2001 From: Alexander Chepurnoy Date: Sat, 24 Sep 2022 03:17:02 +0300 Subject: [PATCH 4/8] polishing text --- eip-0037.md | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/eip-0037.md b/eip-0037.md index 71bcd2e5..606f3afc 100644 --- a/eip-0037.md +++ b/eip-0037.md @@ -38,9 +38,7 @@ Simulations Previous simulations [4,5] are based on observing historical data so ignoring the fact that miners will behave differently in the presence of different difficulty adjustment method. -Thus we made playground simulating random price walking in uptrend or downtrend, assuming that difficulty per unit of account remains stable, and also simulating spikes and drops in difficulty per unit of account. - -In simulation a blockchain is being mined by rational hashpower only. Hashrate is looking at current price and diffuculty. We assume that price and difficulty are changed and the same time, and then hashrate is moving in or out, which is affecting average block generation time *t*. We may assume then that *t = d * c / p*, so average block generation time *t* is proportinal to difficulty *d* and inversely proportional to price *p*. Fixing *t* (which is set to target block generation time, so 2 minutes), *d* and *p* at the beginning of the experiment, we can evaluate *c*. Then on each step we randomly changing *p*, and, according to difficulty from the previous epoch, we can get block generation time for the new epoch. To have a trend in price, we are changing *p* by adding (or subtracting) a random value with fixed average, and also adding a random fluctuation. +Thus we made playground simulating random price walking in uptrend or downtrend. In a simulation, a blockchain is being mined by rational hashpower only. Hashrate is looking at current price and diffuculty. We assume that price and difficulty are changed at the same time, and then hashrate is moving in or out, which is affecting average block generation time *t*. We may assume then that *t = d * c / p*, so average block generation time *t* is proportinal to difficulty *d* and inversely proportional to price *p*. Fixing *t* (which is set to target block generation time, so 2 minutes), *d* and *p* at the beginning of the experiment, we can evaluate *c*. Then on each step we are randomly changing *p* and, according to difficulty from the previous epoch, we can get average block generation time for the new epoch. To have a trend in price, we are changing *p* by adding (or subtracting) a random value with fixed average, and also adding a random fluctuation. Test results: @@ -63,11 +61,20 @@ Test results: *Current DAA*: total error: 19111 *Proposed DAA: total error: 20691 -* 3 epochs us, 3 down , 25% max change +* if average price growth is up 10% per epoch (max delay missed as it is 120s max for all the options): + +*Bitcoin DAA*: total error: 143161 +*Current DAA*: total error: 92861 +*Proposed DAA: total error: 105741 + +* 3 epochs price going up, 3 epochs down , 25% max change *Bitcoin DAA*: total error: 380139, max delay: 160 *Current DAA*: total error: 464081, max delay: 221 *Proposed DAA*: total error: 387880, max delay: 185 + +Total error here is sum of differences between observed block generation time and target (120 s). + As we can see, proposed DAA as well as current one is working better during trends, and proposed DAA softens swings better than current one. Activation From 5c032d98be27693f04356a37f719f4294b205250 Mon Sep 17 00:00:00 2001 From: Alexander Chepurnoy Date: Sat, 24 Sep 2022 15:50:21 +0300 Subject: [PATCH 5/8] simulations update --- eip-0037.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/eip-0037.md b/eip-0037.md index 606f3afc..6fb35719 100644 --- a/eip-0037.md +++ b/eip-0037.md @@ -72,10 +72,14 @@ Test results: *Current DAA*: total error: 464081, max delay: 221 *Proposed DAA*: total error: 387880, max delay: 185 +* Coin hopping - first epoch up to 50% of total hashrate jumping on, next epoch jumping off +*Bitcoin DAA*: total error: 770422, max delay: 192 +*Current DAA*: total error: 586972, max delay: 210 +*Proposed DAA*: total error: 670691, max delay: 182 -Total error here is sum of differences between observed block generation time and target (120 s). +Total error here is sum of differences between observed block generation time and target (120 s). The less total error, the better. -As we can see, proposed DAA as well as current one is working better during trends, and proposed DAA softens swings better than current one. +As we can see, proposed DAA as well as current one is working better during trends, and also in case of 1-epoch coin hopping, and proposed DAA softens swings better than current one. Activation ---------- From dc9a796b01912560eb60207fcbe0121ab522b0aa Mon Sep 17 00:00:00 2001 From: Alexander Chepurnoy Date: Tue, 27 Sep 2022 01:23:39 +0300 Subject: [PATCH 6/8] activation --- eip-0037.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/eip-0037.md b/eip-0037.md index 6fb35719..819bada0 100644 --- a/eip-0037.md +++ b/eip-0037.md @@ -29,7 +29,8 @@ Proposed Changes We propose to make current difficulty readjustment more reactive and smoother by shortening epoch length, amplifying weight of the last epoch and put some limits on difficulty change as follows. 1. Epoch length to be set to 128 blocks. -2. We calcualate *predictive* difficulty according to 8 epochs 128 blocks each and *classic* difficulty as done in Bitcoin. We took average from them. +2. We calcualate *predictive* difficulty according to 8 epochs 128 blocks each and *classic* difficulty as done in Bitcoin. +We limit predictive difficulty change so that it never be changed by more than 50% per epoch. Then we took average from classic and predictive difficulties. 3. We limit change so that difficulty never be changed by more than 50% per epoch. @@ -84,6 +85,8 @@ As we can see, proposed DAA as well as current one is working better during tren Activation ---------- +It is possible to activate EIP-37 after block #843,776 and before block #851,969 . For activation, 460 or more votes for activation required in last 512 blocks, with voting checked every 128 blocks (for blocks which height % 128 == 1), and immediate activation once threshold is met. + References ---------- From 7bd9d309baa0a01bf43209cad4bcfe21436f7f41 Mon Sep 17 00:00:00 2001 From: Alexander Chepurnoy Date: Sun, 2 Oct 2022 23:04:01 +0300 Subject: [PATCH 7/8] activation and impl details --- eip-0037.md | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/eip-0037.md b/eip-0037.md index 819bada0..cb64e3ac 100644 --- a/eip-0037.md +++ b/eip-0037.md @@ -2,9 +2,9 @@ Tweaking Difficulty Adjustment Algorithm ========================================== * Author: kushti -* Status: Proposed +* Status: Implmented * Created: 23-Sep-2022 -* Last edited: 23-Sep-2022 +* Last edited: 02-Oct-2022 * License: CC0 * Forking: hard-fork needed @@ -85,7 +85,12 @@ As we can see, proposed DAA as well as current one is working better during tren Activation ---------- -It is possible to activate EIP-37 after block #843,776 and before block #851,969 . For activation, 460 or more votes for activation required in last 512 blocks, with voting checked every 128 blocks (for blocks which height % 128 == 1), and immediate activation once threshold is met. +It is possible to activate EIP-37 after block #843,776 and before block #851,969 . For activation, 232 or more votes for activation required in the last 256 blocks, with voting checked every 128 blocks (for blocks which height % 128 == 1), and immediate activation once threshold is met. + +Implementation +-------------- + +Proposed difficulty adjustment algorithm and its activation procedure are implemented in the reference protocol client 4.0.100, all the newer versions support them as well [6]. References @@ -96,5 +101,5 @@ References 3. Meshkov D., Chepurnoy A., Jansen M. Short paper: Revisiting difficulty control for blockchain systems 4. jtoomim BCH upgrade proposal: Use ASERT as the new DAA https://read.cash/@jtoomim/bch-upgrade-proposal-use-asert-as-the-new-daa-1d875696 5. Ergo Developers. DifficultyControlSimulator.scala https://github.com/ergoplatform/ergo/blob/0af9dd9d8846d672c1e2a77f8ab29963fa5acd1e/src/test/scala/org/ergoplatform/tools/DifficultyControlSimulator.scala - +6. Ergo reference protocol client releases https://github.com/ergoplatform/ergo/releases From ddbca24fef5e91e0c80c6881fc31d8831ae69768 Mon Sep 17 00:00:00 2001 From: Alexander Chepurnoy Date: Fri, 7 Oct 2022 02:02:56 +0300 Subject: [PATCH 8/8] Update eip-0037.md Co-authored-by: Ricardo M. --- eip-0037.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/eip-0037.md b/eip-0037.md index cb64e3ac..977d7973 100644 --- a/eip-0037.md +++ b/eip-0037.md @@ -29,7 +29,7 @@ Proposed Changes We propose to make current difficulty readjustment more reactive and smoother by shortening epoch length, amplifying weight of the last epoch and put some limits on difficulty change as follows. 1. Epoch length to be set to 128 blocks. -2. We calcualate *predictive* difficulty according to 8 epochs 128 blocks each and *classic* difficulty as done in Bitcoin. +2. We calculate *predictive* difficulty according to 8 epochs 128 blocks each and *classic* difficulty as done in Bitcoin. We limit predictive difficulty change so that it never be changed by more than 50% per epoch. Then we took average from classic and predictive difficulties. 3. We limit change so that difficulty never be changed by more than 50% per epoch.