BIP0322_verification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "750da738",
   "metadata": {},
   "source": [
    "# Bitcoin Improvement Proposal 322 - Generic Signed Message Format\n",
    "\n",
    "## Verification Notebook\n",
    "\n",
    "This notebook verifies a BIP 0322 simple signature on the message \"Hello World\" using address `bc1q9vza2e8x573nczrlzms0wvx3gsqjx7vavgkx0l` which is listed as a test vector with a coresponding signature in the BIP.\n",
    "\n",
    "The notebook attempts to verify both the signature produced in the [BIP0322_signing](http://localhost:8888/notebooks/BIP0322_signing.ipynb) notebook and the signature test vector provided in BIP 0322. It then goes on to contrast this implementation with results from the [P.R.](https://github.com/bitcoin/bitcoin/pull/24058) to add BIP 0322 to Bitcoin core demonstrating that the current test vector in BIP 0322 is incorrect, whilst the signature from the signing notebook is correct (although different from the one initially included in the P.R.'s test suite."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27894f77",
   "metadata": {},
   "source": [
    "### A Simple Signature\n",
    "\n",
    "\n",
    "From the BIP-0322:\n",
    "\n",
    "`A simple signature consists of a witness stack, consensus encoded as a vector of vectors of bytes, and base64-encoded. Validators should construct to_spend and to_sign as defined below, with default values for all fields except that`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac8ef14f",
   "metadata": {},
   "source": [
    "## Notebook Author\n",
    "\n",
    "* Will Abramson\n",
    "* [Legendary Requirements](http://legreq.com/)\n",
    "* Contact: will@legreq.com"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3f84d66",
   "metadata": {},
   "source": [
    "## Acknowledgements\n",
    "\n",
    "This work was funded by Ryan Grant and Digital Contract Design. Thanks also go to Joe Andrieu, Kalle Alm, Pieter Wuille and Jimmy Song for engaging with and supporting various aspects of this work.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b185001",
   "metadata": {},
   "source": [
    "# Notebook Steps\n",
    "\n",
    "1. Define the inputs to the BIP 0322 signature verification\n",
    "2. BIP 0322 Verification Process\n",
    "    1. Create the to_spend tx\n",
    "    2. Create the to_sign tx\n",
    "    3. Verify the signature on the to_sign tx\n",
    "3. Compare with the P.R. that adds BIP 0322 into Bitcoin Core"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c56cf307",
   "metadata": {},
   "source": [
    "## 1. Define Inputs\n",
    "\n",
    "Inputs include\n",
    "\n",
    "- The message being signed\n",
    "- The address that it is being claimed signed the message\n",
    "- The signature to be verified"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6d4a514c",
   "metadata": {},
   "source": [
    "### 1.1. Message Signed\n",
    "\n",
    "The message signed is \"Hello World\" which is then turned into a tagged hash using the BIP 0322 defined tag. The resulting hash is provided as a test vector in BIP 0322 and verified here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "39505d18",
   "metadata": {},
   "outputs": [],
   "source": [
    "from buidl.hash import tagged_hash\n",
    "from buidl.helper import hash256\n",
    "message = b\"Hello World\"\n",
    "tag = b\"BIP0322-signed-message\"\n",
    "\n",
    "tagged_hash = tagged_hash(tag,message)\n",
    "\n",
    "# This test vector is provided in bip0322\n",
    "test_vector = 'f0eb03b1a75ac6d9847f55c624a99169b5dccba2a31f5b23bea77ba270de0a7a'\n",
    "assert(test_vector == tagged_hash.hex())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c75e7ee",
   "metadata": {},
   "source": [
    "### 1.2. Claimed Address that Signed Message\n",
    "\n",
    "This address is typically associated with a private key which it is claimed was used to sign the message"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "e7397d71",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Test vector taken from the bip\n",
    "address = 'bc1q9vza2e8x573nczrlzms0wvx3gsqjx7vavgkx0l'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "740d6374",
   "metadata": {},
   "source": [
    "### 1.3. Test Vector Sig\n",
    "\n",
    "This signature is the test vector taken from [bip0322](https://github.com/bitcoin/bips/blob/master/bip-0322.mediawiki#message-signing).\n",
    "\n",
    "**Note: the buidl-python library produces a different signature. BUT as we will see later, both are valid**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "eae941be",
   "metadata": {},
   "outputs": [],
   "source": [
    "simple_signature = 'AkcwRAIgZRfIY3p7/DoVTty6YZbWS71bc5Vct9p9Fia83eRmw2QCICK/ENGfwLtptFluMGs2KsqoNSk89pO7F29zJLUx9a/sASECx/EgAxlkQpQ9hYjgGu6EBCPMVPwVIVJqO4XCsMvViHI='"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbd4f285",
   "metadata": {},
   "source": [
    "### 1.4. Notebook Produced Sig\n",
    "\n",
    "This signature was produced in the [BIP0322_signing](http://localhost:8888/notebooks/BIP0322_signing.ipynb) notebook. Currently it is this signature that verifies against the message and signing address.\n",
    "\n",
    "**Note: It seems likely I  just replicating the same mistake across both notebooks?**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "5cb63d95",
   "metadata": {},
   "outputs": [],
   "source": [
    "produced_sig = 'AkgwRQIhAOzyynlqt93lOKJr+wmmxIens//zPzl9tqIOua93wO6MAiBi5n5EyAcPScOjf1lAqIUIQtr3zKNeavYabHyR8eGhowEhAsfxIAMZZEKUPYWI4BruhAQjzFT8FSFSajuFwrDL1Yhy'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e34a0148",
   "metadata": {},
   "source": [
    "## 2. BIP 0322 Verification"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cc9f488f",
   "metadata": {},
   "source": [
    "### A. Compute virtual to_spend transaction\n",
    "\n",
    "Note: the message_hash and message_challenge are both calculable by the verifier based on the inputs provided. Namely the message being signed and the address which it is claimed was used to sign this message.\n",
    "\n",
    "```\n",
    "    nVersion = 0\n",
    "    nLockTime = 0\n",
    "    vin[0].prevout.hash = 0000...000\n",
    "    vin[0].prevout.n = 0xFFFFFFFF\n",
    "    vin[0].nSequence = 0\n",
    "    vin[0].scriptSig = OP_0 PUSH32[ message_hash ]\n",
    "    vin[0].scriptWitness = []\n",
    "    vout[0].nValue = 0\n",
    "    vout[0].scriptPubKey = message_challenge\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5260c393",
   "metadata": {},
   "outputs": [],
   "source": [
    "from buidl.tx import Tx, TxIn, TxOut\n",
    "from buidl.helper import little_endian_to_int\n",
    "from buidl.script import Script,ScriptPubKey"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "31cee86f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Not a valid Tx hash. Will never be spendable on any BTC network.\n",
    "prevout_hash = bytes.fromhex('0000000000000000000000000000000000000000000000000000000000000000')\n",
    "# prevout.n\n",
    "prevout_index = little_endian_to_int(bytes.fromhex('FFFFFFFF'))\n",
    "sequence = 0\n",
    "\n",
    "# Byte array of message hash\n",
    "message_hash = tagged_hash\n",
    "\n",
    "# Spotted the error\n",
    "# OP_PUSH32 gets automatically added to the stack when add 32 bytes of data\n",
    "# commands = [0, 32, message_hash]\n",
    "commands = [0, message_hash]\n",
    "scriptSig = Script(commands)\n",
    "\n",
    "# This is equivalent to the one provided\n",
    "print(scriptSig.raw_serialize().hex())\n",
    "\n",
    "assert(\"0020f0eb03b1a75ac6d9847f55c624a99169b5dccba2a31f5b23bea77ba270de0a7a\" == scriptSig.raw_serialize().hex())\n",
    "\n",
    "# Create Tx Input\n",
    "tx_in = TxIn(prevout_hash,prevout_index,scriptSig,sequence)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8ac39edb",
   "metadata": {},
   "outputs": [],
   "source": [
    "from buidl.script import address_to_script_pubkey\n",
    "\n",
    "# Value of tx output\n",
    "value = 0\n",
    "\n",
    "# Convert address to ScriptPubKey\n",
    "script_pubkey = address_to_script_pubkey(address)\n",
    "\n",
    "tx_out = TxOut(value,script_pubkey)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "85bbd21e",
   "metadata": {},
   "outputs": [],
   "source": [
    "tx_out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7d83b406",
   "metadata": {},
   "outputs": [],
   "source": [
    "# create transaction\n",
    "version=0\n",
    "tx_inputs = [tx_in]\n",
    "tx_outputs = [tx_out]\n",
    "locktime=0\n",
    "network=\"mainnet\"\n",
    "\n",
    "# Could be false, but using a segwit address. I think this is the \"Simple Signature\" in BIP-0322\n",
    "segwit=True\n",
    "\n",
    "virtual_to_spend_tx = Tx(version,tx_inputs,tx_outputs,locktime,network,segwit)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b0b90e22",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(virtual_to_spend_tx.is_coinbase())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5707bc26",
   "metadata": {},
   "source": [
    "### B. Create the virtual to_sign transaction\n",
    "\n",
    "**There are two separate to_sign transactions created in this section. One from the BIP 0322 signature test case and one from the signature generated in the signing notebook.**\n",
    "\n",
    "Note: the signature in simple format is just the witness stack\n",
    "\n",
    "```python\n",
    "    nVersion = 0 or (FULL format only) as appropriate (e.g. 2, for time locks)\n",
    "    nLockTime = 0 or (FULL format only) as appropriate (for time locks)\n",
    "    vin[0].prevout.hash = to_spend.txid\n",
    "    vin[0].prevout.n = 0\n",
    "    vin[0].nSequence = 0 or (FULL format only) as appropriate (for time locks)\n",
    "    vin[0].scriptWitness = message_signature\n",
    "    vout[0].nValue = 0\n",
    "    vout[0].scriptPubKey = OP_RETURN\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a4859e1",
   "metadata": {},
   "source": [
    "### B.1. Decode the signature into a witness stack\n",
    "\n",
    "To do this we:\n",
    "\n",
    "- Decode from base64\n",
    "- Use this to instantiate a bytes stream\n",
    "- Instantiate a Witness by parsing this stream"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e828585b",
   "metadata": {},
   "outputs": [],
   "source": [
    "from buidl.helper import base64_decode\n",
    "from buidl.ecc import Signature\n",
    "from buidl.witness import Witness\n",
    "import io"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd87fa1f",
   "metadata": {},
   "source": [
    "#### B.1.1 Using bip322 test vector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6fbf9faa",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "# Note: Using the test vector I am unable to get the code to verify\n",
    "decoded_witness_test = base64_decode(simple_signature)\n",
    "\n",
    "stream = io.BytesIO(decoded_witness_test)\n",
    "witness_test = Witness.parse(stream)\n",
    "print(witness_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e50cbccf",
   "metadata": {},
   "source": [
    "#### B1.1.2 Using the notebook produced signature"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4845b290",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Using the signature produced in the other notebook does work however\n",
    "decoded_witness_produced = base64_decode(produced_sig)\n",
    "\n",
    "stream = io.BytesIO(decoded_witness_produced)\n",
    "witness_produced = Witness.parse(stream)\n",
    "print(witness_produced)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f08f0de",
   "metadata": {},
   "source": [
    "### B.2. Create the to_sign transaction input and populate with the witness"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5dd21d3b",
   "metadata": {},
   "source": [
    "#### B.2.1. Create tx_in using bip322 test vector witness"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ead24a7d",
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "prevout_hash = virtual_to_spend_tx.hash()\n",
    "prevout_index = 0\n",
    "sequence = 0\n",
    "\n",
    "\n",
    "tx_in_test = TxIn(prevout_hash,prevout_index,sequence=sequence)\n",
    "\n",
    "# set the witness\n",
    "tx_in_test.witness = witness_test\n",
    "\n",
    "tx_in_test.witness"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "58192627",
   "metadata": {},
   "source": [
    "#### B.2.2. Create tx_in using notebook produced witness"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8ca73c64",
   "metadata": {},
   "outputs": [],
   "source": [
    "prevout_hash = virtual_to_spend_tx.hash()\n",
    "prevout_index = 0\n",
    "sequence = 0\n",
    "\n",
    "tx_in_produced = TxIn(prevout_hash,prevout_index,sequence=sequence)\n",
    "\n",
    "# set the witness\n",
    "tx_in_produced.witness = witness_produced\n",
    "\n",
    "tx_in_produced.witness"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3772e17",
   "metadata": {},
   "source": [
    "### B.3. Create tx_out (same output for all)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4e2d210e",
   "metadata": {},
   "outputs": [],
   "source": [
    "value = 0\n",
    "# OP Code 106 for OP_RETURN\n",
    "commands = [106]\n",
    "scriptPubKey = Script(commands)\n",
    "\n",
    "tx_output = TxOut(value,scriptPubKey)\n",
    "tx_output"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b66bea2",
   "metadata": {},
   "source": [
    "### B.4. Create the virtual to_sign transaction"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b923d9e3",
   "metadata": {},
   "source": [
    "#### B.4.1. Create virtual_to_sign_tx for test vector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "48fd363f",
   "metadata": {},
   "outputs": [],
   "source": [
    "version = 0\n",
    "locktime = 0\n",
    "virtual_to_sign_tx_test_vector = Tx(version, [tx_in_test], [tx_output],locktime,segwit=False)\n",
    "\n",
    "print(virtual_to_sign_tx_test_vector.id())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dbf13950",
   "metadata": {},
   "source": [
    "#### B.4.2 Create virtual_to_sign_tx for produced signature"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f2bdfad7",
   "metadata": {},
   "outputs": [],
   "source": [
    "version = 0\n",
    "locktime = 0\n",
    "virtual_to_sign_tx_produced = Tx(version, [tx_in_produced], [tx_output],locktime,segwit=False)\n",
    "print(virtual_to_sign_tx_test_vector.id())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb8571fa",
   "metadata": {},
   "source": [
    "## Note: The txid's are the same. The different witness does not change the txid because it is not included in the hash"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e751f37",
   "metadata": {},
   "source": [
    "### A whole bunch of validation that SHOULD happen ... See [verification process](https://github.com/bitcoin/bips/blob/master/bip-0322.mediawiki#verification-process)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d477db7d",
   "metadata": {},
   "source": [
    "### Initially all I am trying to do is validate the signature on the input in the to_sign transaction\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce7914d3",
   "metadata": {},
   "source": [
    "### The input for the virtual to_sign transation IS the output at index 0 from the virtual to_spend transaction\n",
    "\n",
    "Normally the relevant information (The outputs script_pubkey and value) would be populated by querying the network for the transaction and fetching it from there. In this case we set those values on the input manually."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07b34a96",
   "metadata": {},
   "source": [
    "## C. Verify the Signature on the to_sign transcation\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "31635efa",
   "metadata": {},
   "source": [
    "### C.1. Verify BIP 0322 Test Vector to_sign transaction"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a5a415b",
   "metadata": {},
   "source": [
    "#### C.1.1 Maunally Fetch the to_spend transaction and populate the required values for the to_sign tx_input"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cf5d4ba4",
   "metadata": {},
   "outputs": [],
   "source": [
    "virtual_to_spend_tx"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4f210ecb",
   "metadata": {},
   "outputs": [],
   "source": [
    "virtual_to_sign_tx_test_vector.tx_ins[0]._script_pubkey = virtual_to_spend_tx.tx_outs[0].script_pubkey\n",
    "virtual_to_sign_tx_test_vector.tx_ins[0]._value = virtual_to_spend_tx.tx_outs[0].amount"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08301a53",
   "metadata": {},
   "source": [
    "#### C.1.2. Verify the input is a valid unlocking of the to_spend output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1ce8ad1a",
   "metadata": {},
   "outputs": [],
   "source": [
    "virtual_to_sign_tx_test_vector.verify_input(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "162afb2d",
   "metadata": {},
   "source": [
    "### C.2. Verify the Produced Signature from the [Signing Notebook](http://localhost:8888/notebooks/BIP0322_signing.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2173c256",
   "metadata": {},
   "source": [
    "#### C.2.1. Maunally Fetch the to_spend transaction and populate the required values for the to_sign tx_input"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "79bc45e7",
   "metadata": {},
   "outputs": [],
   "source": [
    "virtual_to_sign_tx_produced.tx_ins[0]._script_pubkey = virtual_to_spend_tx.tx_outs[0].script_pubkey\n",
    "virtual_to_sign_tx_produced.tx_ins[0]._value = virtual_to_spend_tx.tx_outs[0].amount"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92b4e735",
   "metadata": {},
   "source": [
    "#### C.2.2. Verify the input is a valid unlocking of the to_spend output\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c2acac80",
   "metadata": {},
   "outputs": [],
   "source": [
    "virtual_to_sign_tx_produced.verify_input(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1067ded",
   "metadata": {},
   "source": [
    "# This gives us confidence that the signature produced in the signing notebook is a correct BIP 0322 signature\n",
    "\n",
    "## However, It was noticed that this was not the same signature as the one used in the BIP 0322 P.R. tests. So I did some more digging\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a73a62f3",
   "metadata": {},
   "source": [
    "# 3. Checking Against the BIP 0322 P.R to Bitcoin Core\n",
    "\n",
    "## [BIP 0322 basic support #24058](https://github.com/bitcoin/bitcoin/pull/24058)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb7bf5bb",
   "metadata": {},
   "source": [
    "### The Test Case added to the P.R. for message \"Hello World\" and addess \"bc1q9vza2e8x573nczrlzms0wvx3gsqjx7vavgkx0l\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dac74fe0",
   "metadata": {},
   "source": [
    "```c\n",
    "BOOST_CHECK_EQUAL(\n",
    "    MessageVerify(\n",
    "        \"bc1q9vza2e8x573nczrlzms0wvx3gsqjx7vavgkx0l\",\n",
    "        \"AkcwRAIgZRfIY3p7/DoVTty6YZbWS71bc5Vct9p9Fia83eRmw2QCICK/ENGfwLtptFluMGs2KsqoNSk89pO7F29zJLUx9a/sASECx/EgAxlkQpQ9hYjgGu6EBCPMVPwVIVJqO4XCsMvViHI=\",\n",
    "        \"Hello World\"),\n",
    "    MessageVerificationResult::OK);\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "edb6ab2f",
   "metadata": {},
   "source": [
    "### Adding Print Statements to the Code run when executing the above test case\n",
    "\n",
    "Output generated by adding the following cout print statements to MessageVerifyBIP322. See https://github.com/bitcoin/bitcoin/blob/843ce398a1d4661b9d93160635b46cc0a70d6153/src/util/message.cpp#L48 for original"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd80c918",
   "metadata": {},
   "source": [
    "```c\n",
    "MessageVerificationResult MessageVerifyBIP322(\n",
    "    CTxDestination& destination,\n",
    "    std::vector<unsigned char>& signature,\n",
    "    const std::string& message,\n",
    "    MessageVerificationResult legacyError)\n",
    "{\n",
    "    auto txs = BIP322Txs::Create(destination, message, legacyError, &signature);\n",
    "    if (!txs) return legacyError;\n",
    "\n",
    "    const CTransaction& to_sign = txs->m_to_sign;\n",
    "    const CTransaction& to_spend = txs->m_to_spend;\n",
    "\n",
    "    std::cout << \"This is the to_spend tx : \\n\" << to_spend.ToString();\n",
    "    std::cout << \"\\nThis is the to_spend tx hash : \\n\" << to_spend.GetHash().ToString();\n",
    "\n",
    "    std::cout << \"\\n\\n This is the to_sign tx : \\n\" << to_sign.ToString();\n",
    "\n",
    "    std::cout << \"\\nThis is the to_sign tx hash : \\n\" << to_sign.GetHash().ToString();\n",
    "\n",
    "    const CScript scriptSig = to_sign.vin[0].scriptSig;\n",
    "    const CScriptWitness& witness = to_sign.vin[0].scriptWitness;\n",
    "\n",
    "    PrecomputedTransactionData txdata;\n",
    "    txdata.Init(to_sign, {to_spend.vout[0]});\n",
    "    TransactionSignatureChecker sigcheck(&to_sign, /* nInIn= */ 0, /* amountIn= */ to_spend.vout[0].nValue, txdata, MissingDataBehavior::ASSERT_FAIL);\n",
    "    sigcheck.RequireSighashAll();\n",
    "\n",
    "    if (!VerifyScript(scriptSig, to_spend.vout[0].scriptPubKey, &witness, BIP322_REQUIRED_FLAGS, sigcheck)) {\n",
    "        return MessageVerificationResult::ERR_INVALID;\n",
    "    }\n",
    "\n",
    "    // inconclusive checks\n",
    "\n",
    "    if (to_sign.nVersion != 0 && to_sign.nVersion != 2) {\n",
    "        return MessageVerificationResult::INCONCLUSIVE;\n",
    "    }\n",
    "\n",
    "    if (!VerifyScript(scriptSig, to_spend.vout[0].scriptPubKey, &witness, BIP322_INCONCLUSIVE_FLAGS, sigcheck)) {\n",
    "        return MessageVerificationResult::INCONCLUSIVE;\n",
    "    }\n",
    "\n",
    "    // timelock check\n",
    "    if (to_sign.nLockTime > 0 || to_sign.vin[0].nSequence > 0) {\n",
    "        return MessageVerificationResult::OK_TIMELOCKED;\n",
    "    }\n",
    "\n",
    "\n",
    "\n",
    "    return MessageVerificationResult::OK;\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23f75047",
   "metadata": {},
   "source": [
    "### The Resulting Terminal Output"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a42bccdb",
   "metadata": {},
   "source": [
    "```bash\n",
    "This is the to_spend tx : \n",
    "CTransaction(hash=b79d196740, ver=0, vin.size=1, vout.size=1, nLockTime=0)\n",
    "    CTxIn(COutPoint(0000000000, 4294967295), coinbase 0020f0eb03b1a75ac6d9847f55c624a99169b5dccba2a31f5b23bea77ba270de0a7a, nSequence=0)\n",
    "    CScriptWitness()\n",
    "    CTxOut(nValue=0.00000000, scriptPubKey=00142b05d564e6a7a33c087f16e0f7)\n",
    "\n",
    "This is the to_spend tx hash : \n",
    "b79d196740ad5217771c1098fc4a4b51e0535c32236c71f1ea4d61a2d603352b\n",
    "\n",
    " This is the to_sign tx : \n",
    "CTransaction(hash=88737ae86f, ver=0, vin.size=1, vout.size=1, nLockTime=0)\n",
    "    CTxIn(COutPoint(b79d196740, 0), scriptSig=, nSequence=0)\n",
    "    CScriptWitness(304402206517c8637a7bfc3a154edcba6196d64bbd5b73955cb7da7d1626bcdde466c364022022bf10d19fc0bb69b4596e306b362acaa835293cf693bb176f7324b531f5afec01, 02c7f12003196442943d8588e01aee840423cc54fc1521526a3b85c2b0cbd58872)\n",
    "    CTxOut(nValue=0.00000000, scriptPubKey=6a)\n",
    "\n",
    "This is the to_sign tx hash : \n",
    "88737ae86f2077145f93cc4b153ae9a1cb8d56afa511988c149c5c8c9d93bddf\n",
    "\n",
    "\n",
    "## LOGS from Bitcoin Core Test Case Reporting which test was run\n",
    "info: check MessageVerify( \"bc1q9vza2e8x573nczrlzms0wvx3gsqjx7vavgkx0l\", \"AkcwRAIgZRfIY3p7/DoVTty6YZbWS71bc5Vct9p9Fia83eRmw2QCICK/ENGfwLtptFluMGs2KsqoNSk89pO7F29zJLUx9a/sASECx/EgAxlkQpQ9hYjgGu6EBCPMVPwVIVJqO4XCsMvViHI=\", \"Hello World\") == MessageVerificationResult::OK has passed\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f50f0b6",
   "metadata": {},
   "source": [
    "### Bitcoin core generated tx hashes extracted from the above output\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "af8067a4",
   "metadata": {},
   "outputs": [],
   "source": [
    "btc_to_spend_tx_hash = \"b79d196740ad5217771c1098fc4a4b51e0535c32236c71f1ea4d61a2d603352b\"\n",
    "btx_to_sign_tx_hash = \"88737ae86f2077145f93cc4b153ae9a1cb8d56afa511988c149c5c8c9d93bddf\"\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1310177d",
   "metadata": {},
   "source": [
    "## Check to_spend tx hash"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "feca01a2",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(virtual_to_spend_tx.id())\n",
    "assert(virtual_to_spend_tx.id() == btc_to_spend_tx_hash)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56dde276",
   "metadata": {},
   "source": [
    "## Check to_sign tx hash"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2bea48a7",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Note both have valid tx ids because the witness does not affect the hash of the tx\n",
    "assert(virtual_to_sign_tx_test_vector.id() == btx_to_sign_tx_hash)\n",
    "assert(virtual_to_sign_tx_produced.id() == btx_to_sign_tx_hash)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "972494a0",
   "metadata": {},
   "source": [
    "## Identified Differences compared to Below Implementation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aa9dc0c5",
   "metadata": {},
   "source": [
    "### The test vector\n",
    "\n",
    "Most strikingly the test vector that Bitcoin core uses to get a `MessageVerificationResult::OK` on the message \"Hello World\" is **NOT** the same as the BIP!\n",
    "\n",
    "See [line 2596 of util_tests.cpp](https://github.com/bitcoin/bitcoin/blob/843ce398a1d4661b9d93160635b46cc0a70d6153/src/test/util_tests.cpp#L2596):\n",
    "\n",
    "```c\n",
    "    BOOST_CHECK_EQUAL(\n",
    "        MessageVerify(\n",
    "            \"bc1q9vza2e8x573nczrlzms0wvx3gsqjx7vavgkx0l\",\n",
    "            \"AkcwRAIgZRfIY3p7/DoVTty6YZbWS71bc5Vct9p9Fia83eRmw2QCICK/ENGfwLtptFluMGs2KsqoNSk89pO7F29zJLUx9a/sASECx/EgAxlkQpQ9hYjgGu6EBCPMVPwVIVJqO4XCsMvViHI=\",\n",
    "            \"Hello World\"),\n",
    "        MessageVerificationResult::OK);\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7cfad941",
   "metadata": {},
   "outputs": [],
   "source": [
    "btc_core_test_vector_sig = \"AkcwRAIgZRfIY3p7/DoVTty6YZbWS71bc5Vct9p9Fia83eRmw2QCICK/ENGfwLtptFluMGs2KsqoNSk89pO7F29zJLUx9a/sASECx/EgAxlkQpQ9hYjgGu6EBCPMVPwVIVJqO4XCsMvViHI=\"\n",
    "\n",
    "assert(simple_signature == btc_core_test_vector_sig)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf5a0efb",
   "metadata": {},
   "source": [
    "### Although neither is it the same as the signature I produced in the signing notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9d8be88f",
   "metadata": {},
   "outputs": [],
   "source": [
    "assert(produced_sig == btc_core_test_vector_sig)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "721362c0",
   "metadata": {},
   "source": [
    "## to_spend transaction\n",
    "\n",
    "Bitcoin Core output when printing the to_spend.ToString() result\n",
    "\n",
    "################################################################################\n",
    "\n",
    "```bash\n",
    "CTransaction(hash=b79d196740, ver=0, vin.size=1, vout.size=1, nLockTime=0)\n",
    "    CTxIn(COutPoint(0000000000, 4294967295), coinbase 0020f0eb03b1a75ac6d9847f55c624a99169b5dccba2a31f5b23bea77ba270de0a7a, nSequence=0)\n",
    "    CScriptWitness()\n",
    "    CTxOut(nValue=0.00000000, scriptPubKey=00142b05d564e6a7a33c087f16e0f7)\n",
    "```\n",
    "################################################################################\n",
    "\n",
    "### Differences\n",
    "\n",
    "**THERE DON'T APPEAR TO BE ANY**\n",
    "\n",
    "- CTxIn appears as a coinbase in the print, but so is the tx I produced (See below)\n",
    "- nValue in CTxOut is in decimal. \n",
    "    - Don't think this should matter"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "291372a8",
   "metadata": {},
   "source": [
    "### to_sign transaction\n",
    "\n",
    "Bitcoin Core output\n",
    "\n",
    "##########################################################################\n",
    "\n",
    "```bash\n",
    "CTransaction(hash=88737ae86f, ver=0, vin.size=1, vout.size=1, nLockTime=0)\n",
    "    CTxIn(COutPoint(b79d196740, 0), scriptSig=, nSequence=0)\n",
    "    CScriptWitness(304402206517c8637a7bfc3a154edcba6196d64bbd5b73955cb7da7d1626bcdde466c364022022bf10d19fc0bb69b4596e306b362acaa835293cf693bb176f7324b531f5afec01, 02c7f12003196442943d8588e01aee840423cc54fc1521526a3b85c2b0cbd58872)\n",
    "    CTxOut(nValue=0.00000000, scriptPubKey=6a)\n",
    "```\n",
    "##########################################################################\n",
    "\n",
    "### Differences\n",
    "\n",
    "- The witness (and hence the actual BIP0322 signature is different)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2399ca1",
   "metadata": {},
   "source": [
    "## Verify the BIP 0322 P.R. Sgnature Value\n",
    "\n",
    "This replicates the verification code from about but uses the signature value found in the P.R. test case for the same message and address"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82c43bcd",
   "metadata": {},
   "source": [
    "#### Decode signature into witness stack using the btc core test vector witness"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cf8006ca",
   "metadata": {},
   "outputs": [],
   "source": [
    "from buidl.helper import base64_decode\n",
    "from buidl.ecc import Signature\n",
    "from buidl.witness import Witness\n",
    "\n",
    "# Using the signature produced in the other notebook does work however\n",
    "decoded_witness_btc = base64_decode(btc_core_test_vector_sig)\n",
    "\n",
    "import io\n",
    "stream = io.BytesIO(decoded_witness_btc)\n",
    "witness_btc = Witness.parse(stream)\n",
    "print(witness_btc)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52dfb57d",
   "metadata": {},
   "source": [
    "#### Create tx_in using notebook btc_test_vector witness"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5f1ce8ce",
   "metadata": {},
   "outputs": [],
   "source": [
    "prevout_hash = virtual_to_spend_tx.hash()\n",
    "prevout_index = 0\n",
    "sequence = 0\n",
    "\n",
    "tx_in_btc_core = TxIn(prevout_hash,prevout_index,sequence=sequence)\n",
    "\n",
    "# set the witness\n",
    "tx_in_btc_core.witness = witness_btc\n",
    "\n",
    "tx_in_btc_core.witness"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ebf97f79",
   "metadata": {},
   "source": [
    "#### Create tx_out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f61aa1d2",
   "metadata": {},
   "outputs": [],
   "source": [
    "value = 0\n",
    "# OP Code 106 for OP_RETURN\n",
    "commands = [106]\n",
    "scriptPubKey = Script(commands)\n",
    "\n",
    "tx_output = TxOut(value,scriptPubKey)\n",
    "tx_output"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f8ea70c",
   "metadata": {},
   "source": [
    "#### Create virtual_to_sign_tx for btc_core_test_vector signature"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b75fc536",
   "metadata": {},
   "outputs": [],
   "source": [
    "version = 0\n",
    "locktime = 0\n",
    "virtual_to_sign_tx_bitcoin_core = Tx(version, [tx_in_btc_core], [tx_output],locktime,segwit=False)\n",
    "print(virtual_to_sign_tx_bitcoin_core.id())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a58967b",
   "metadata": {},
   "source": [
    "## Verification of Notebook Bitcoin Core Signature"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90280746",
   "metadata": {},
   "source": [
    "### Manually populate the value and script_pubkey on the to_sign tx_input from the to_spend tx_output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b45aff86",
   "metadata": {},
   "outputs": [],
   "source": [
    "virtual_to_sign_tx_bitcoin_core.tx_ins[0]._script_pubkey = virtual_to_spend_tx.tx_outs[0].script_pubkey\n",
    "virtual_to_sign_tx_bitcoin_core.tx_ins[0]._value = virtual_to_spend_tx.tx_outs[0].amount"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2cdd00ea",
   "metadata": {},
   "source": [
    "### Verify the input is a valid unlocking of the to_spend output\n",
    "\n",
    "Not sure why this isn't working for the test vector of BIP-0322"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "998f78c4",
   "metadata": {},
   "outputs": [],
   "source": [
    "virtual_to_sign_tx_bitcoin_core.verify_input(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29c6274d",
   "metadata": {},
   "source": [
    "# This also verifies as true. To double check, I added an additional test case to the BIP 0322 P.R. code in Bitcoin Core for the signature I produced in the signing notebook. This test passed and has since been included as part of the P.R."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7a917ad",
   "metadata": {},
   "source": [
    "## Initially, I thought this must be an error. But because the signature scheme includes randomess, it is actually valid for two signatures to be verifiable for the same message and address pair. See response to my [message](https://github.com/bitcoin/bitcoin/pull/24058/files#r872561862) on the code in the P.R. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "352a4076",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}