Fixes to miniscript descriptor

Now we're back to using regular bdk types, which is a big advantage.

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Side Protocol
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,108 @@
+# Side BFT-CRDT PoC
+
+This is a proof of concept implementation of a BFT-CRDT blockchain system.
+
+## Prerequisites
+
+Install a recent version of Rust.
+
+## Running in development
+
+Run the watcher first:
+
+```bash
+cd side-watcher
+cargo watch -x run
+```
+
+To init a Side node:
+
+```bash
+cd side-node
+cargo run -- init node1
+cargo run -- init node2
+cargo run -- init node3
+cargo run -- init node4
+```
+
+To start a node with a cargo watch for development purposes (from the side-node dir), open up a few terminals and run:
+
+```bash
+cargo watch -x "run -- run -- node1"
+cargo watch -x "run -- run -- node2"
+cargo watch -x "run -- run -- node3"
+cargo watch -x "run -- run -- node4"
+```
+
+You can then type directly into each of the Side Node consoles. Messages will be relayed to each Side Node, and the transaction history will end up being the same on all nodes.
+
+## Discussion
+
+What we have here is a very simple system comprised of two key parts: the Side Node, and the Side Watcher.
+
+### Side Node(s)
+
+The Side Nodes make up a system of BFT-CRDT-producing nodes that can make a blockchain. Currently they can reliably send transactions to each other in a secure way, such that all nodes they communicate with can tell whether received transactions are obeying the rules of the system.
+
+The Side Node does not download any chain state, and if one goes off-line it will miss transactions. This is expected at the moment and fairly easy to fix, with a bit of work.
+
+Next dev tasks:
+
+- [ ] we don't need a Watcher, the first node can act as a leader until people decide they don't want to trust it any more
+- [ ] the leader node can have a timer in it for block creation
+- [ ] code up the ability to switch leaders (can be a human decision at first, later an (optional) automated choice)
+- [ ] pick a commit and reveal scheme to remove MEV. One thing to investigate is [single-use seals](https://docs.rgb.info/distributed-computing-concepts/single-use-seals)
+- [ ] enable Side Nodes to download current P2P chain state so that they start - out with a consistent copy of transaction data, and also do catch-up after going off-line
+- [ ] remove the proc macro code from bft-json-crdt
+- [ ] add smart contract execution engine (CosmWasm would be a good first choice)
+- [ ] enable Side Nodes to download contract code for a given contract
+- [ ] enable Side Nodes to download current contract state for a given contract
+- [ ] switch to full P2P messaging instead of websockets
+
+### Side Watcher
+
+The Side Watcher is a simple relayer node that sits between the Side Chain (Cosmos) and the decentralized Side Nodes. At the moment, it simply relays transactions between nodes via a websocket. We aim to eliminate this component from the architecture, but for the moment it simplifies networking and consensus agreement while we experiment with higher-value concepts.
+
+To fulfill the promises in the Lite Paper, the Side Watcher needs to:
+
+- [ ] make a block for the BFT-CRDT chain when the Side Chain creates a block
+- [ ] submit BFT-CRDT chain data to the Side Chain
+
+Later, we will aim to remove the Side Watcher from the architecture, by (a) moving to pure P2P transactions between Side Nodes, and (b) doing leader election of a Side Node to reach agreement on the submitted block.
+
+## Bitcoin integration
+
+There is an Esplora Bitcoin client integrated into the node
+
+### Simple coin transfers
+
+The client can do simple coin transfers using esplora and the Mutinynet server's Signet (30 second blocktime).
+
+The client's demo driver can be run by doing:
+
+```
+cargo run -- init dave
+cargo run -- init sammy
+cargo run -- btc-transfer
+```
+
+You'll need to have funded the "dave" address prior to running the `btc` command - otherwise the transfer will fail gracefully.
+
+I have been using this primarily as a way to experiment with constructing and broadcasting Bitcoin transactions, with the hope that it would be possible to move on to more advanced constructions (e.g. state channels).
+
+### HTLCs (in progress)
+
+An experimental driver for Bitcoin Hash Time Locked Contracts (HTLCs).
+
+
+## Possible uses
+
+### DKG
+
+It strikes me that there are many, many systems which rely on a trusted setup, and which might be able to use Distributed Key Generation (DKG) instead. SNARK systems for instance all have this problem.
+
+It is not necessarily the case that e.g. signer participants and validators are the same entities. Being able to quickly spin up a blockchain and use it to sign (potentially temporary or ephemeral) keyshare data might be pretty useful.
+
+### Cross chain transfers
+
+The ability to be part of multiple consensus groups at once might provide new opportunities for cross-chain transfers.

crates/bft-json-crdt/Cargo.toml

@@ -17,15 +17,20 @@ bft = []
 bft-crdt-derive = { path = "bft-crdt-derive" }
 colored = "2.0.0"
 fastcrypto = "0.1.8"
-itertools = "0.10.5"
+indexmap = { version = "2.2.6", features = ["serde"] }
 rand = "0.8.5"
 random_color = "0.6.1"
 serde = { version = "1.0", features = ["derive"] }
-serde_json = "1.0.85"
+serde_json = { version = "1.0.85", features = ["preserve_order"] }
 serde_with = "3.8.1"
 sha2 = "0.10.6"
 
 [dev-dependencies]
-serde = { version = "1.0", features = ["derive"] }
-serde_json = "1.0.85"
+criterion = { version = "0.4", features = ["html_reports"] }
 time = "0.1"
+serde = { version = "1.0", features = ["derive"] }
+serde_json = { version = "1.0.85", features = ["preserve_order"] }
+
+[[bench]]
+name = "speed"
+harness = false

crates/bft-json-crdt/benches/speed.rs

@@ -1,59 +1,67 @@
-#![feature(test)]
-
-extern crate test;
 use bft_json_crdt::{
     json_crdt::JsonValue, keypair::make_author, list_crdt::ListCrdt, op::Op, op::ROOT_ID,
 };
+use criterion::{criterion_group, criterion_main, Criterion};
 use rand::seq::SliceRandom;
-use test::Bencher;
 
-#[bench]
-fn bench_insert_1_000_root(b: &mut Bencher) {
-    b.iter(|| {
-        let mut list = ListCrdt::<i64>::new(make_author(1), vec![]);
-        for i in 0..1_000 {
-            list.insert(ROOT_ID, i);
-        }
-    })
-}
-
-#[bench]
-fn bench_insert_1_000_linear(b: &mut Bencher) {
-    b.iter(|| {
-        let mut list = ListCrdt::<i64>::new(make_author(1), vec![]);
-        let mut prev = ROOT_ID;
-        for i in 0..1_000 {
-            let op = list.insert(prev, i);
-            prev = op.id;
-        }
-    })
-}
-
-#[bench]
-fn bench_insert_many_agents_conflicts(b: &mut Bencher) {
-    b.iter(|| {
-        const N: u8 = 50;
-        let mut rng = rand::thread_rng();
-        let mut crdts: Vec<ListCrdt<i64>> = Vec::with_capacity(N as usize);
-        let mut logs: Vec<Op<JsonValue>> = Vec::new();
-        for i in 0..N {
-            let list = ListCrdt::new(make_author(i), vec![]);
-            crdts.push(list);
-            for _ in 0..5 {
-                let op = crdts[i as usize].insert(ROOT_ID, i as i32);
-                logs.push(op);
+fn bench_insert_100_root(c: &mut Criterion) {
+    c.bench_function("bench insert 100 root", |b| {
+        b.iter(|| {
+            let mut list = ListCrdt::<i64>::new(make_author(1), vec![]);
+            for i in 0..100 {
+                list.insert(ROOT_ID, i);
             }
-        }
+        })
+    });
+}
 
-        logs.shuffle(&mut rng);
-        for op in logs {
-            for c in &mut crdts {
-                if op.author() != c.our_id {
-                    c.apply(op.clone());
+fn bench_insert_100_linear(c: &mut Criterion) {
+    c.bench_function("bench insert 100 linear", |b| {
+        b.iter(|| {
+            let mut list = ListCrdt::<i64>::new(make_author(1), vec![]);
+            let mut prev = ROOT_ID;
+            for i in 0..100 {
+                let op = list.insert(prev, i);
+                prev = op.id;
+            }
+        })
+    });
+}
+
+fn bench_insert_many_agents_conflicts(c: &mut Criterion) {
+    c.bench_function("bench insert many agents conflicts", |b| {
+        b.iter(|| {
+            const N: u8 = 10;
+            let mut rng = rand::thread_rng();
+            let mut crdts: Vec<ListCrdt<i64>> = Vec::with_capacity(N as usize);
+            let mut logs: Vec<Op<JsonValue>> = Vec::new();
+            for i in 0..N {
+                let list = ListCrdt::new(make_author(i), vec![]);
+                crdts.push(list);
+                for _ in 0..5 {
+                    let op = crdts[i as usize].insert(ROOT_ID, i as i32);
+                    logs.push(op);
                 }
             }
-        }
 
-        assert!(crdts.windows(2).all(|w| w[0].view() == w[1].view()));
-    })
+            logs.shuffle(&mut rng);
+            for op in logs {
+                for c in &mut crdts {
+                    if op.author() != c.our_id {
+                        c.apply(op.clone());
+                    }
+                }
+            }
+
+            assert!(crdts.windows(2).all(|w| w[0].view() == w[1].view()));
+        })
+    });
 }
+
+criterion_group!(
+    benches,
+    bench_insert_100_root,
+    bench_insert_100_linear,
+    bench_insert_many_agents_conflicts
+);
+criterion_main!(benches);

crates/bft-json-crdt/bft-crdt-derive/Cargo.toml

@@ -8,6 +8,7 @@ publish = false
 proc-macro = true
 
 [dependencies]
+indexmap = { version = "2.2.6", features = ["serde"] }
 proc-macro2 = "1.0.47"
 proc-macro-crate = "1.2.1"
 quote = "1.0.21"

crates/bft-json-crdt/bft-crdt-derive/src/lib.rs

@@ -6,13 +6,12 @@ use syn::{
     parse::{self, Parser},
     parse_macro_input,
     spanned::Spanned,
-    Data, DeriveInput, Field, Fields, ItemStruct, LitStr, Type
+    Data, DeriveInput, Field, Fields, ItemStruct, LitStr, Type,
 };
 
 /// Helper to get tokenstream representing the parent crate
 fn get_crate_name() -> TokenStream {
-    let cr8 = crate_name("bft-json-crdt")
-        .unwrap_or(FoundCrate::Itself);
+    let cr8 = crate_name("bft-json-bft-crdt").unwrap_or(FoundCrate::Itself);
     match cr8 {
         FoundCrate::Itself => quote! { ::bft_json_crdt },
         FoundCrate::Name(name) => {
@@ -106,17 +105,23 @@ pub fn derive_json_crdt(input: OgTokenStream) -> OgTokenStream {
                                 })
                             } else {
                                 Err(format!("failed to convert {:?} -> {}<T>", value, #ident_str.to_string()))
-                            }  
+                            }
                         }
                     }
 
-                    impl #impl_generics std::fmt::Debug for #ident #ty_generics #where_clause {
-                        fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
-                            let mut fields = Vec::new();
-                            #(fields.push(format!("{}", #ident_strings.to_string()));)*
-                            write!(f, "{{ {:?} }}", fields.join(", "))
-                        }
-                    } 
+                    // I'm pulling this out so that we can see actual CRD content in debug output.
+                    //
+                    // The plan is to mostly get rid of the macros anyway, so it's a reasonable first step.
+                    // It could (alternately) be just as good to keep the macros and change this function to
+                    // output actual field content instead of just field names.
+                    //
+                    // impl #impl_generics std::fmt::Debug for #ident #ty_generics #where_clause {
+                    //     fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+                    //         let mut fields = Vec::new();
+                    //         #(fields.push(format!("{}", #ident_strings.to_string()));)*
+                    //         write!(f, "{{ {:?} }}", fields.join(", "))
+                    //     }
+                    // }
 
                     impl #impl_generics #crate_name::json_crdt::CrdtNode for #ident #ty_generics #where_clause {
                         fn apply(&mut self, op: #crate_name::op::Op<#crate_name::json_crdt::JsonValue>) -> #crate_name::json_crdt::OpState {
@@ -128,7 +133,7 @@ pub fn derive_json_crdt(input: OgTokenStream) -> OgTokenStream {
                             }
 
                             if self.path.len() == op.path.len() {
-                                return #crate_name::json_crdt::OpState::ErrApplyOnStruct; 
+                                return #crate_name::json_crdt::OpState::ErrApplyOnStruct;
                             } else {
                                 let idx = self.path.len();
                                 if let #crate_name::op::PathSegment::Field(path_seg) = &op.path[idx] {
@@ -139,12 +144,12 @@ pub fn derive_json_crdt(input: OgTokenStream) -> OgTokenStream {
                                         _ => {},
                                     };
                                 };
-                                return #crate_name::json_crdt::OpState::ErrPathMismatch 
+                                return #crate_name::json_crdt::OpState::ErrPathMismatch
                             }
                         }
 
                         fn view(&self) -> #crate_name::json_crdt::JsonValue {
-                            let mut view_map = std::collections::HashMap::new();
+                            let mut view_map = indexmap::IndexMap::new();
                             #(view_map.insert(#ident_strings.to_string(), self.#ident_literals.view().into());)*
                             #crate_name::json_crdt::JsonValue::Object(view_map)
                         }
@@ -173,7 +178,7 @@ pub fn derive_json_crdt(input: OgTokenStream) -> OgTokenStream {
                         fn debug_view(&self, _indent: usize) -> String {
                             "".to_string()
                         }
-                    } 
+                    }
                 };
 
                 // Hand the output tokens back to the compiler

crates/bft-json-crdt/src/json_crdt.rs

@@ -17,6 +17,13 @@ use fastcrypto::{
     traits::{KeyPair, ToFromBytes},
     // Verifier,
 };
+// TODO: serde's json object serialization and deserialization (correctly) do not define anything
+// object field order in JSON objects. However, the hash check impl in bft-json-bft-crdt does take order
+// into account. This is going to cause problems later for non-Rust implementations, BFT hash checking
+// currently depends on JSON serialization/deserialization object order. This shouldn't be the case
+// but I've hacked in an IndexMap for the moment to get the PoC working. To see the problem, replace this with
+// a std HashMap, everything will screw up (annoyingly, only *most* of the time).
+use indexmap::IndexMap;
 use serde::{Deserialize, Serialize};
 use serde_with::{serde_as, Bytes};
 
@@ -212,7 +219,7 @@ impl<T: CrdtNode + DebugView> BaseCrdt<T> {
     /// Apply a signed operation to this BaseCRDT, verifying integrity and routing to the right
     /// nested CRDT
     pub fn apply(&mut self, op: SignedOp) -> OpState {
-        self.log_try_apply(&op);
+        // self.log_try_apply(&op);
 
         #[cfg(feature = "bft")]
         if !op.is_valid_digest() {
@@ -232,9 +239,9 @@ impl<T: CrdtNode + DebugView> BaseCrdt<T> {
         }
 
         // apply
-        self.log_actually_apply(&op);
+        // self.log_actually_apply(&op);
         let status = self.doc.apply(op.inner);
-        self.debug_view();
+        // self.debug_view();
         self.received.insert(op_id);
 
         // apply all of its causal dependents if there are any
@@ -256,7 +263,7 @@ pub enum JsonValue {
     Number(f64),
     String(String),
     Array(Vec<JsonValue>),
-    Object(HashMap<String, JsonValue>),
+    Object(IndexMap<String, JsonValue>),
 }
 
 impl Display for JsonValue {
@@ -542,7 +549,7 @@ mod test {
     #[test]
     fn test_derive_basic() {
         #[add_crdt_fields]
-        #[derive(Clone, CrdtNode)]
+        #[derive(Clone, CrdtNode, Debug)]
         struct Player {
             x: LwwRegisterCrdt<f64>,
             y: LwwRegisterCrdt<f64>,
@@ -557,14 +564,14 @@ mod test {
     #[test]
     fn test_derive_nested() {
         #[add_crdt_fields]
-        #[derive(Clone, CrdtNode)]
+        #[derive(Clone, CrdtNode, Debug)]
         struct Position {
             x: LwwRegisterCrdt<f64>,
             y: LwwRegisterCrdt<f64>,
         }
 
         #[add_crdt_fields]
-        #[derive(Clone, CrdtNode)]
+        #[derive(Clone, CrdtNode, Debug)]
         struct Player {
             pos: Position,
             balance: LwwRegisterCrdt<f64>,
@@ -582,7 +589,7 @@ mod test {
     #[test]
     fn test_lww_ops() {
         #[add_crdt_fields]
-        #[derive(Clone, CrdtNode)]
+        #[derive(Clone, CrdtNode, Debug)]
         struct Test {
             a: LwwRegisterCrdt<f64>,
             b: LwwRegisterCrdt<bool>,
@@ -642,7 +649,7 @@ mod test {
     #[test]
     fn test_vec_and_map_ops() {
         #[add_crdt_fields]
-        #[derive(Clone, CrdtNode)]
+        #[derive(Clone, CrdtNode, Debug)]
         struct Test {
             a: ListCrdt<String>,
         }
@@ -682,14 +689,14 @@ mod test {
     #[test]
     fn test_causal_field_dependency() {
         #[add_crdt_fields]
-        #[derive(Clone, CrdtNode)]
+        #[derive(Clone, CrdtNode, Debug)]
         struct Item {
             name: LwwRegisterCrdt<String>,
             soulbound: LwwRegisterCrdt<bool>,
         }
 
         #[add_crdt_fields]
-        #[derive(Clone, CrdtNode)]
+        #[derive(Clone, CrdtNode, Debug)]
         struct Player {
             inventory: ListCrdt<Item>,
             balance: LwwRegisterCrdt<f64>,
@@ -748,7 +755,7 @@ mod test {
     #[test]
     fn test_2d_grid() {
         #[add_crdt_fields]
-        #[derive(Clone, CrdtNode)]
+        #[derive(Clone, CrdtNode, Debug)]
         struct Game {
             grid: ListCrdt<ListCrdt<LwwRegisterCrdt<bool>>>,
         }
@@ -809,7 +816,7 @@ mod test {
     #[test]
     fn test_arb_json() {
         #[add_crdt_fields]
-        #[derive(Clone, CrdtNode)]
+        #[derive(Clone, CrdtNode, Debug)]
         struct Test {
             reg: LwwRegisterCrdt<JsonValue>,
         }
@@ -845,13 +852,13 @@ mod test {
     #[test]
     fn test_wrong_json_types() {
         #[add_crdt_fields]
-        #[derive(Clone, CrdtNode)]
+        #[derive(Clone, CrdtNode, Debug)]
         struct Nested {
             list: ListCrdt<f64>,
         }
 
         #[add_crdt_fields]
-        #[derive(Clone, CrdtNode)]
+        #[derive(Clone, CrdtNode, Debug)]
         struct Test {
             reg: LwwRegisterCrdt<bool>,
             strct: ListCrdt<Nested>,

crates/bft-json-crdt/tests/byzantine.rs

@@ -20,7 +20,7 @@ use serde_json::json;
 // 5. block actual messages from honest actors (eclipse attack)
 
 #[add_crdt_fields]
-#[derive(Clone, CrdtNode)]
+#[derive(Clone, CrdtNode, Debug)]
 struct ListExample {
     list: ListCrdt<char>,
 }
@@ -91,13 +91,13 @@ fn test_forge_update() {
 }
 
 #[add_crdt_fields]
-#[derive(Clone, CrdtNode)]
+#[derive(Clone, CrdtNode, Debug)]
 struct Nested {
     a: Nested2,
 }
 
 #[add_crdt_fields]
-#[derive(Clone, CrdtNode)]
+#[derive(Clone, CrdtNode, Debug)]
 struct Nested2 {
     b: LwwRegisterCrdt<bool>,
 }

crates/bft-json-crdt/tests/editing-trace.js

@@ -259796,7 +259796,7 @@ function insertAt(idx, elt) {
       const pos = new_log.findIndex(log => log[0] === parent_id)
       new_log.push([ID_COUNTER, pos, 0, elt])
       crdt.splice(raw_i + 1, 0, { deleted: false, content: elt, id: ID_COUNTER })
-      // console.log(`insert at ${idx} translated as op [${ID_COUNTER}, ${pos}, ${0}, ${escape(elt)}] found at ${raw_i + 1}::`, crdt[raw_i + 1])
+      // console.log(`insert at ${idx} translated as op [${ID_COUNTER}, ${pos}, ${0}, ${escape(elt)}] found at ${raw_i + 1}::`, bft-crdt[raw_i + 1])
       return
     }
 
@@ -259816,7 +259816,7 @@ function deleteAt(idx) {
       const pos = new_log.findIndex(log => log[0] === our_id)
       new_log.push([ID_COUNTER, pos, 1]);
       crdt[raw_i].deleted = true
-      // console.log(`delete at ${idx} translated as op [${ID_COUNTER}, ${pos}, ${1}] found at ${raw_i} with our_id ${our_id}::`, crdt[raw_i])
+      // console.log(`delete at ${idx} translated as op [${ID_COUNTER}, ${pos}, ${1}] found at ${raw_i} with our_id ${our_id}::`, bft-crdt[raw_i])
       return
     }
 
@@ -259853,7 +259853,7 @@ function rawJSString(edits) {
 //     deleteAt(edit[0])
 //   }
 // }
-// console.log(crdt)
+// console.log(bft-crdt)
 // rawJSString(mock_edits)
 // console.log(new_log)
 // const subset = edits.slice(0, 50000) 

side-node/Cargo.lock

@@ -1,7 +0,0 @@
-# This file is automatically @generated by Cargo.
-# It is not intended for manual editing.
-version = 3
-
-[[package]]
-name = "side-node"
-version = "0.1.0"

side-node/Cargo.toml

@@ -6,26 +6,33 @@ edition = "2021"
 # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
 
 [dependencies]
+anyhow = "1.0.86"
 async-trait = "0.1.52"
-base64 = "0.21.7"
+bdk = { version = "0.29.0", features = [
+    "compiler",
+    "use-esplora-blocking",
+    "std",
+    "keys-bip39",
+], default-features = false }
 bft-json-crdt = { path = "../crates/bft-json-crdt" }
 bft-crdt-derive = { path = "../crates/bft-json-crdt/bft-crdt-derive" }
 clap = { version = "4.5.4", features = ["derive"] }
 dirs = "5.0.1"
 ezsockets = { version = "*", features = ["client"] }
 fastcrypto = "0.1.8"
-# serde_cbor = "0.11.2" # move to this once we need to pack things in CBOR
+indexmap = { version = "2.2.6", features = ["serde"] }
 serde = { version = "1.0", features = ["derive"] }
 serde_json = "1.0.117"
-serde_with = "3.8.1"
 sha256 = "1.5.0"
 tokio = { version = "1.37.0", features = ["full"] }
-tracing = "0.1.32"
-tracing-subscriber = "0.3.9"
 toml = "0.8.14"
-url = "2.2.2"
+tracing = "0.1"
+tracing-subscriber = { version = "0.3", features = ["std", "env-filter"] }
 
 
+[dev-dependencies]
+uuid = { version = "1.8.0", features = ["v4"] }
+
 [features]
 default = ["bft", "logging-list", "logging-json"]
 logging-list = ["logging-base"]

side-node/src/bft_crdt/keys.rs

@@ -8,7 +8,7 @@ use bft_json_crdt::keypair::{make_keypair, Ed25519KeyPair};
 use fastcrypto::traits::EncodeDecodeBase64;
 
 /// Writes a new Ed25519 keypair to the file at key_path.
-pub(crate) fn write(key_path: PathBuf) -> Result<(), std::io::Error> {
+pub(crate) fn write(key_path: &PathBuf) -> Result<(), std::io::Error> {
     let keys = make_keypair();
 
     let mut file = File::create(key_path)?;
@@ -17,10 +17,10 @@ pub(crate) fn write(key_path: PathBuf) -> Result<(), std::io::Error> {
     Ok(())
 }
 
-pub(crate) fn load_from_file(side_dir: PathBuf) -> Ed25519KeyPair {
+pub(crate) fn load_from_file(side_dir: &PathBuf) -> Ed25519KeyPair {
     let key_path = crate::utils::side_paths(side_dir.clone()).0;
 
-    let data = fs::read_to_string(key_path).expect("couldn't read key file");
+    let data = fs::read_to_string(key_path).expect("couldn't read bft-bft-crdt key file");
     println!("data: {:?}", data);
 
     Ed25519KeyPair::decode_base64(&data).expect("couldn't load keypair from file")

side-node/src/bft_crdt/mod.rs

@@ -0,0 +1,31 @@
+use bft_crdt_derive::add_crdt_fields;
+use bft_json_crdt::{
+    json_crdt::{CrdtNode, IntoCrdtNode},
+    list_crdt::ListCrdt,
+};
+use serde::{Deserialize, Serialize};
+
+pub mod keys;
+pub mod stdin;
+pub mod websocket;
+
+#[add_crdt_fields]
+#[derive(Clone, CrdtNode, Serialize, Deserialize, Debug)]
+pub struct TransactionList {
+    pub list: ListCrdt<Transaction>,
+}
+
+impl TransactionList {
+    pub fn view_sha(&self) -> String {
+        sha256::digest(serde_json::to_string(&self.list.view()).unwrap().as_bytes()).to_string()
+    }
+}
+
+/// A fake Transaction struct we can use as a simulated payload
+#[add_crdt_fields]
+#[derive(Clone, CrdtNode, Serialize, Deserialize, PartialEq, Debug)]
+pub struct Transaction {
+    from: String,
+    to: String,
+    amount: f64,
+}

side-node/src/bft_crdt/stdin.rs

@@ -5,7 +5,6 @@ pub(crate) fn input(stdin_sender: std::sync::mpsc::Sender<String>) {
     let stdin = std::io::stdin();
     let lines = stdin.lock().lines();
     for line in lines {
-        println!("We're in stdin_input");
         let line = line.unwrap();
         stdin_sender.send(line).unwrap();
     }

side-node/src/bft_crdt/websocket.rs

@@ -3,20 +3,19 @@ use bft_json_crdt::json_crdt::SignedOp;
 use ezsockets::ClientConfig;
 use tokio::sync::mpsc;
 
-pub(crate) struct WebSocketClient {
+use crate::utils;
+
+pub struct Client {
     incoming_sender: mpsc::Sender<SignedOp>,
-    handle: ezsockets::Client<WebSocketClient>,
+    handle: ezsockets::Client<Client>,
 }
 
-impl WebSocketClient {
+impl Client {
     /// Start the websocket client
-    pub(crate) async fn new(
-        incoming_sender: mpsc::Sender<SignedOp>,
-    ) -> ezsockets::Client<WebSocketClient> {
-        tracing_subscriber::fmt::init();
+    pub async fn new(incoming_sender: mpsc::Sender<SignedOp>) -> ezsockets::Client<Client> {
         let config = ClientConfig::new("ws://localhost:8080/websocket");
         let (handle, future) = ezsockets::connect(
-            |client| WebSocketClient {
+            |client| Client {
                 incoming_sender,
                 handle: client,
             },
@@ -31,26 +30,36 @@ impl WebSocketClient {
 }
 
 #[async_trait]
-impl ezsockets::ClientExt for WebSocketClient {
+impl ezsockets::ClientExt for Client {
     // Right now we're only using the Call type for sending signed ops
     // change this to an enum if we need to send other types of calls, and
     // match on it.
     type Call = String;
 
+    /// When we receive a text message, apply the bft-crdt operation contained in it to our
+    /// local bft-crdt.
     async fn on_text(&mut self, text: String) -> Result<(), ezsockets::Error> {
-        tracing::info!("received text: {text:?}");
+        let string_sha = utils::sha256(text.clone());
+        println!("received text, sha: {string_sha}");
         let incoming: bft_json_crdt::json_crdt::SignedOp = serde_json::from_str(&text).unwrap();
+        let object_sha = utils::shappy(incoming.clone());
+        println!("deserialized: {}", object_sha);
+        if string_sha != object_sha {
+            panic!("sha mismatch: {string_sha} != {object_sha}, bft-bft-crdt has failed");
+        }
         self.incoming_sender.send(incoming).await?;
         Ok(())
     }
 
+    /// When we receive a binary message, log the bytes. Currently unused.
     async fn on_binary(&mut self, bytes: Vec<u8>) -> Result<(), ezsockets::Error> {
         tracing::info!("received bytes: {bytes:?}");
         Ok(())
     }
 
+    /// Call this with the `Call` type to send application data to the websocket client
+    /// (and from there, to the server).
     async fn on_call(&mut self, call: Self::Call) -> Result<(), ezsockets::Error> {
-        tracing::info!("sending signed op: {call:?}");
         self.handle.text(call)?;
         Ok(())
     }

side-node/src/bitcoin/client.rs

@@ -0,0 +1,115 @@
+use bdk::{
+    bitcoin::{psbt::PartiallySignedTransaction, secp256k1::PublicKey, Network, Transaction},
+    blockchain::EsploraBlockchain,
+    database::MemoryDatabase,
+    keys::ExtendedKey,
+    template::Bip84,
+    wallet::AddressInfo,
+    KeychainKind, SignOptions, SyncOptions, Wallet,
+};
+
+use crate::{bitcoin::keys, utils};
+
+/// A client that uses Esplora to interact with the Bitcoin network.
+pub struct BitcoinClient {
+    pub(crate) blockchain: bdk::blockchain::EsploraBlockchain,
+    name: String,
+    pub(crate) wallet: Wallet<MemoryDatabase>,
+    pub(crate) public_key: PublicKey,
+}
+
+impl BitcoinClient {
+    pub(crate) fn sync(&self) -> anyhow::Result<()> {
+        tracing::info!("syncing {}'s wallet", self.name);
+        self.wallet.sync(&self.blockchain, SyncOptions::default())?;
+        Ok(())
+    }
+
+    pub(crate) fn broadcast(&self, tx: &Transaction) -> anyhow::Result<()> {
+        tracing::info!(
+            "broadcasting transaction, output will be at https://mutinynet.com/tx/{}",
+            tx.txid()
+        );
+        let _ = self.blockchain.broadcast(&tx);
+        Ok(())
+    }
+
+    /// Builds and signs a send transaction to send coins between addresses.
+    ///
+    /// Does NOT send it, you must call `broadcast` to do that.
+    ///
+    /// We could split the creation and signing easily if needed.
+    pub(crate) fn build_and_sign_send_tx(
+        &mut self,
+        recipient: AddressInfo,
+        amount: u64,
+    ) -> Result<bdk::bitcoin::Transaction, anyhow::Error> {
+        let mut tx_builder = self.wallet.build_tx();
+        tx_builder
+            .add_recipient(recipient.script_pubkey(), amount)
+            .enable_rbf();
+        let (mut psbt, _) = tx_builder.finish()?;
+        let tx = self.sign(&mut psbt, true)?.extract_tx();
+        Ok(tx)
+    }
+
+    pub(crate) fn sign(
+        &self,
+        psbt: &mut PartiallySignedTransaction,
+        finalize: bool,
+    ) -> Result<PartiallySignedTransaction, anyhow::Error> {
+        tracing::info!("{} signing PSBT", self.name);
+
+        let options = SignOptions {
+            try_finalize: finalize,
+            ..Default::default()
+        };
+
+        let finalized = self.wallet.sign(psbt, options)?;
+
+        // make sure the PSBT is finalized if we asked for it
+        if finalize {
+            assert!(finalized)
+        }
+        Ok(psbt.to_owned())
+    }
+
+    /// Creates a Bitcoin descriptor wallet with the mnemonic in the given user directory.
+    pub(crate) fn create(name: &str, network: Network) -> anyhow::Result<BitcoinClient> {
+        let keys_dir = utils::home(name);
+
+        // let mnemonic_path = crate::utils::side_paths(keys_dir).1; // TODO: this tuple stinks
+        // let words = fs::read_to_string(mnemonic_path).expect("couldn't read bitcoin key file");
+
+        let xkey: ExtendedKey = keys::load_from_file(&keys_dir)?;
+
+        let xprv = xkey
+            .into_xprv(Network::Signet)
+            .expect("couldn't turn xkey into xprv");
+
+        let secp = bdk::bitcoin::secp256k1::Secp256k1::new();
+
+        let external_descriptor1 = Bip84(xprv.clone(), KeychainKind::External);
+        let external_descriptor2 = Bip84(xprv, KeychainKind::External);
+        let internal_descriptor = Some(Bip84(xprv, KeychainKind::Internal));
+
+        let wallet = Wallet::new(
+            external_descriptor1,
+            internal_descriptor,
+            network,
+            MemoryDatabase::default(),
+        )?;
+
+        let blockchain = EsploraBlockchain::new("https://mutinynet.com/api", 20);
+        let external_public_key = external_descriptor2.0.private_key.public_key(&secp);
+
+        let esplora = BitcoinClient {
+            name: name.to_string(),
+            wallet,
+            blockchain,
+            public_key: external_public_key,
+        };
+
+        Ok(esplora)
+    }
+}

side-node/src/bitcoin/driver/create_htlc.rs

@@ -0,0 +1,51 @@
+use crate::bitcoin::{self, driver};
+use crate::utils;
+use bdk::wallet::AddressIndex::New;
+use bdk::SignOptions;
+
+pub(crate) async fn run() -> anyhow::Result<()> {
+    tracing::info!("starting htlc flow");
+    let (dave, sammy) = driver::setup().await?;
+    let _ = dave.sync();
+    let _ = sammy.sync();
+
+    // Create an HTLC descriptor with a redeem identity, a hashlock from the preimage,
+    // a refund timelock, and a refund identity
+    let value = 500;
+    let recipient = sammy.wallet.get_address(New)?.script_pubkey();
+    let hash_preimage = "blah".to_string();
+    let hashlock = utils::sha256(hash_preimage);
+    let htlc = bitcoin::htlc::Htlc::new(dave.public_key, hashlock, 100, sammy.public_key);
+    let htlc_descriptor = htlc.to_miniscript_descriptor();
+
+    // format a new commitment transaction like in Lightning
+    let mut commitment_builder = dave.wallet.build_tx();
+    commitment_builder.enable_rbf();
+    let (psbt, _) = commitment_builder
+        .finish()
+        .expect("unable to build commitment");
+
+    // sign the commitment transaction
+    let mut dave_psbt = dave.sign(&mut psbt.clone(), false)?;
+    let sammy_psbt = sammy.sign(&mut psbt.clone(), false)?;
+
+    dave_psbt
+        .combine(sammy_psbt)
+        .expect("problem combining bitcoin PSBTs"); // these guys love mutability
+
+    let finalized = dave
+        .wallet
+        .finalize_psbt(&mut dave_psbt, SignOptions::default())
+        .expect("couldn't finalize");
+
+    assert!(finalized);
+    let tx = dave_psbt.extract_tx();
+
+    let _ = dave.broadcast(&tx)?;
+
+    let _ = sammy.sync();
+    let sammy_balance = sammy.wallet.get_balance()?;
+    tracing::info!("sammy balance: {}", sammy_balance);
+
+    Ok(())
+}

side-node/src/bitcoin/driver/mod.rs

@@ -0,0 +1,27 @@
+use super::client::BitcoinClient;
+use bdk::bitcoin::Network;
+use tracing::Level;
+use tracing_subscriber::{filter, fmt, layer::Layer, prelude::*, Registry};
+
+pub mod create_htlc;
+pub mod policy_transfer;
+pub mod simple_transfer;
+
+async fn setup() -> Result<(BitcoinClient, BitcoinClient), anyhow::Error> {
+    tracing_setup();
+    let dave = BitcoinClient::create("dave", Network::Signet)?;
+    let sammy = BitcoinClient::create("sammy", Network::Signet)?;
+
+    Ok((dave, sammy))
+}
+
+fn tracing_setup() {
+    // show only info level logs and above:
+    let info = filter::LevelFilter::from_level(Level::INFO);
+
+    // set up the tracing subscriber:
+    let subscriber = Registry::default().with(fmt::layer().with_filter(info));
+    tracing::subscriber::set_global_default(subscriber).unwrap();
+
+    tracing::info!("Tracing initialized.");
+}

side-node/src/bitcoin/driver/policy_transfer.rs

@@ -0,0 +1,23 @@
+use std::str::FromStr;
+
+use bdk::miniscript::policy;
+
+use crate::bitcoin;
+
+/// A miniscript-based simple transfer, equivalent to the `simple_transfer`
+/// but using a Bitcoin miniscript policy. TODO: finish implementation, it's not
+/// working yet.
+pub(crate) async fn run() -> anyhow::Result<()> {
+    let (dave, _sammy) = bitcoin::driver::setup().await?;
+
+    tracing::info!("starting transfer policy flow");
+
+    let policy_str = format!("addr(bc1qgw6xanldsz959z45y4dszehx4xkuzf7nfhya8x)");
+
+    let policy = policy::Concrete::<bdk::bitcoin::PublicKey>::from_str(&policy_str)
+        .expect("policy compilation failed")
+        .to_owned();
+
+    tracing::info!("policy: {}", policy);
+    Ok(())
+}

side-node/src/bitcoin/driver/simple_transfer.rs

@@ -0,0 +1,30 @@
+use crate::bitcoin::client::BitcoinClient;
+use crate::bitcoin::driver;
+use bdk::wallet::AddressIndex;
+
+/// Run the simplest transfer flow. There is no policy file,
+/// it's just a normal bitcoin transaction for a sanity check.
+pub async fn run() -> Result<(), anyhow::Error> {
+    let (mut dave, sammy) = driver::setup().await?;
+
+    let send_amount = 500;
+    let _ = ensure_enough_sats(&dave, send_amount);
+
+    let sammy_address = sammy.wallet.get_address(AddressIndex::New)?;
+    let tx = dave.build_and_sign_send_tx(sammy_address, send_amount)?;
+    dave.broadcast(&tx)?;
+
+    Ok(())
+}
+
+/// Exit if the wallet does not have enough sats to send.
+fn ensure_enough_sats(wallet: &BitcoinClient, send_amount: u64) -> anyhow::Result<()> {
+    if wallet.wallet.get_balance()?.get_total() < send_amount {
+        tracing::error!(
+            "Please send at least {} sats to the receiving address. Exiting.",
+            send_amount
+        );
+        std::process::exit(0);
+    }
+    Ok(())
+}

side-node/src/bitcoin/htlc.rs

@@ -0,0 +1,58 @@
+use std::str::FromStr;
+
+use bdk::{
+    bitcoin::secp256k1::PublicKey,
+    miniscript::{descriptor::Wsh, policy::Concrete},
+};
+
+/// A hash time locked contract between two parties.
+///
+/// If the hash preimage of the hashlock is revealed, the value is sent to the redeem_identity.
+///
+/// Alternately, if the refund timelock expires, the value can be refunded to the refund_identity.
+pub(crate) struct Htlc {
+    redeem_identity: PublicKey,
+    hashlock: String,
+    refund_timelock: u64,
+    refund_indentiy: PublicKey,
+}
+
+impl Htlc {
+    /// Create a new HTLC.
+    pub(crate) fn new(
+        redeem_identity: PublicKey,
+        hashlock: String,
+        refund_timelock: u64,
+        refund_indentiy: PublicKey,
+    ) -> Self {
+        Self {
+            redeem_identity,
+            hashlock,
+            refund_timelock,
+            refund_indentiy,
+        }
+    }
+
+    pub(crate) fn to_miniscript_descriptor(&self) -> Wsh<String> {
+        let htlc_descriptor = Wsh::new(
+            self.to_miniscript_policy()
+                .compile()
+                .expect("Policy compilation only fails on resource limits or mixed timelocks"),
+        )
+        .expect("Resource limits");
+        assert!(htlc_descriptor.sanity_check().is_ok());
+        tracing::info!("descriptor: {}", htlc_descriptor);
+
+        htlc_descriptor
+    }
+
+    fn to_miniscript_policy(&self) -> Concrete<String> {
+        Concrete::<String>::from_str(&format!(
+            "or(10@and(sha256({secret_hash}),pk({redeem_identity})),1@and(older({expiry}),pk({refund_identity})))",
+            secret_hash = self.hashlock,
+            redeem_identity = self.redeem_identity,
+            refund_identity = self.refund_indentiy,
+            expiry = self.refund_timelock
+        )).expect("Policy compilation only fails on resource limits or mixed timelocks")
+    }
+}

side-node/src/bitcoin/keys.rs

@@ -0,0 +1,50 @@
+use bdk::{
+    keys::{
+        bip39::{Language, Mnemonic, WordCount},
+        DerivableKey, ExtendedKey, GeneratableKey, GeneratedKey,
+    },
+    miniscript,
+};
+
+use std::{
+    fs::{self, File},
+    io::Write,
+    path::PathBuf,
+};
+
+/// Write the mnemonic to a file in the node's side directory
+///
+/// TODO: obviously spitting the mnemonic out to the console is not for production
+pub(crate) fn write(mnemonic_path: &PathBuf) -> Result<(), std::io::Error> {
+    let mnemonic = make_mnemonic();
+    let mut file = File::create(mnemonic_path)?;
+    println!("mnemonic: {mnemonic}");
+    file.write(mnemonic.as_bytes())?;
+
+    Ok(())
+}
+
+pub(crate) fn load_from_file(side_dir: &PathBuf) -> anyhow::Result<ExtendedKey> {
+    let mnemonic_path = crate::utils::side_paths(side_dir.clone()).1; // TODO: this tuple stinks
+    let mnemonic_words = fs::read_to_string(mnemonic_path).expect("couldn't read bitcoin key file");
+    println!("Creating extended key from mnemonic: {mnemonic_words}");
+    generate_extended_key(mnemonic_words)
+}
+
+/// Creates Signet Bitcoin descriptors from a mnemonic
+fn generate_extended_key(mnemonic_words: String) -> anyhow::Result<ExtendedKey> {
+    let mnemonic = Mnemonic::parse(mnemonic_words).unwrap();
+
+    // Generate the extended key
+    let xkey: ExtendedKey = mnemonic
+        .into_extended_key()
+        .expect("couldn't turn mnemonic into xkey");
+
+    Ok(xkey)
+}
+
+fn make_mnemonic() -> String {
+    let mnemonic: GeneratedKey<_, miniscript::Segwitv0> =
+        Mnemonic::generate((WordCount::Words12, Language::English)).unwrap();
+    mnemonic.to_string()
+}

side-node/src/bitcoin/mod.rs

@@ -0,0 +1,4 @@
+pub mod client;
+pub mod driver;
+pub mod htlc;
+pub mod keys;

side-node/src/cli/mod.rs

@@ -17,6 +17,15 @@ pub(crate) struct Args {
 
 #[derive(Subcommand)]
 pub(crate) enum Commands {
+    /// transfers bitcoin between two wallets using a driver program
+    BtcTransfer {},
+
+    /// transfers bitcoin but this time uses a Miniscript policy
+    BtcPolicyTransfer {},
+
+    /// sets up a Bitcoin HTLC
+    BtcHtlc {},
+
     /// runs the Side Node
     Run { name: String },
 

side-node/src/crdt.rs

@@ -1,34 +0,0 @@
-use bft_crdt_derive::add_crdt_fields;
-use bft_json_crdt::{
-    json_crdt::{CrdtNode, IntoCrdtNode},
-    list_crdt::ListCrdt,
-};
-use serde::{Deserialize, Serialize};
-
-#[add_crdt_fields]
-#[derive(Clone, CrdtNode, Serialize, Deserialize)]
-pub(crate) struct TransactionList {
-    pub(crate) list: ListCrdt<Transaction>,
-}
-
-/// A fake Transaction struct we can use as a simulated payload
-#[add_crdt_fields]
-#[derive(Clone, CrdtNode, Serialize, Deserialize)]
-pub(crate) struct Transaction {
-    from: String,
-    to: String,
-    amount: f64,
-}
-
-// impl TransactionList {
-//     fn create(&mut self, keys: &Ed25519KeyPair) -> bft_json_crdt::json_crdt::SignedOp {
-//         // generate a placeholder transaction
-//         let transaction = _fake_transaction(keys.public().to_string());
-
-//         // next job is to keep adding to this guy
-//         let last: &Op<Transaction>;
-//         last = self.list.ops.last().expect("couldn't find last op");
-//         let signed_op = self.list.insert(last.id, transaction.clone()).sign(&keys);
-//         signed_op
-//     }
-// }

side-node/src/init/mod.rs

@@ -2,16 +2,19 @@ use std::path::PathBuf;
 
 use config::SideNodeConfig;
 
-use crate::{keys, utils};
+use crate::{bft_crdt, bitcoin, utils};
 
 pub(crate) mod config;
 
 pub(crate) fn init(home: PathBuf, config: SideNodeConfig) -> Result<(), std::io::Error> {
     ensure_side_directory_exists(&home)?;
-    let (key_path, config_path) = utils::side_paths(home.clone());
+    let (bft_crdt_key_path, bitcoin_key_path, config_path) = utils::side_paths(home.clone());
 
-    println!("Writing key to: {:?}", key_path);
-    keys::write(key_path)?;
+    println!("Writing bft bft-crdt key to: {:?}", bft_crdt_key_path);
+    bft_crdt::keys::write(&bft_crdt_key_path)?;
+
+    println!("Writing bitcoin key to: {:?}", bitcoin_key_path);
+    bitcoin::keys::write(&bitcoin_key_path)?;
 
     println!("Writing config to: {:?}", config_path);
     config::write_toml(&config, &config_path).expect("unable to write config file");
@@ -20,7 +23,7 @@ pub(crate) fn init(home: PathBuf, config: SideNodeConfig) -> Result<(), std::io:
 }
 
 /// Ensures that the directory at side_dir exists, so we have a place
-/// to store our key file and config file.
+/// to store our key files and config file.
 fn ensure_side_directory_exists(side_dir: &PathBuf) -> Result<(), std::io::Error> {
     if side_dir.exists() {
         return Ok(());
@@ -34,7 +37,7 @@ fn ensure_side_directory_exists(side_dir: &PathBuf) -> Result<(), std::io::Error
 
 #[cfg(test)]
 mod tests {
-    use std::{fs, path::Path};
+    use std::{fs, path::Path, str::FromStr};
 
     use fastcrypto::{
         ed25519::Ed25519KeyPair,
@@ -43,50 +46,75 @@ mod tests {
 
     use super::*;
 
-    fn default_side_node_config() -> SideNodeConfig {
-        SideNodeConfig {
-            name: "alice".to_string(),
-        }
+    /// Generates a SideNodeConfig with a unique name for each test.
+    /// This is necessary because the tests run in parallel and we
+    /// don't want them to interfere with each other - without a unique
+    /// name, the tests would all try to write to the same directory and we
+    /// get test indeterminacy
+    fn side_node_config() -> (SideNodeConfig, String) {
+        let name = format!("test-{}", uuid::Uuid::new_v4()).to_string();
+        (SideNodeConfig { name: name.clone() }, name)
+    }
+
+    #[test]
+    fn creates_bitcoin_keys() {
+        let (config, name) = side_node_config();
+        let side_dir = format!("/tmp/side/{name}");
+
+        let mut bitcoin_keys_path = PathBuf::new();
+        bitcoin_keys_path.push(side_dir.clone());
+        bitcoin_keys_path.push(utils::BITCOIN_KEY_FILE);
+
+        let _ = init(PathBuf::from_str(&side_dir).unwrap(), config);
+        assert!(bitcoin_keys_path.exists());
+
+        // check that the pem is readable
+        // let data = fs::read_to_string(bitcoin_keys_path).expect("couldn't read key file");
+        // let keys = Ed25519KeyPair::decode_base64(&data).expect("couldn't load keypair from file");
+        // assert_eq!(keys.public().as_bytes().len(), 32);
     }
 
     #[test]
     fn creates_side_node_directory() {
-        let mut test_home = PathBuf::new();
-        let side_dir = "/tmp/side";
+        let (config, name) = side_node_config();
+        let side_dir = format!("/tmp/side/{name}");
 
-        // clean up any previous test runs
-        fs::remove_dir_all(side_dir).expect("couldn't remove side directory during test");
+        let mut test_home = PathBuf::new();
 
         test_home.push(side_dir);
         let node_dir = Path::new(&test_home).parent().unwrap().to_str().unwrap();
-        let _ = init(test_home.clone(), default_side_node_config());
+        let _ = init(test_home.clone(), config);
         assert!(std::path::Path::new(node_dir).exists());
     }
 
     #[test]
-    fn creates_key_file() {
-        let mut file_path = PathBuf::new();
-        file_path.push("/tmp/side");
-        let side_dir = file_path.clone();
-        file_path.push(utils::KEY_FILE);
+    fn creates_bft_crdt_key_file() {
+        let (config, name) = side_node_config();
+        let side_dir = format!("/tmp/side/{name}");
 
-        let _ = init(side_dir.clone(), default_side_node_config());
-        assert!(file_path.exists());
+        let mut key_file_path = PathBuf::new();
+        key_file_path.push(side_dir.clone());
+        key_file_path.push(utils::BFT_CRDT_KEY_FILE);
+
+        let _ = init(PathBuf::from_str(&side_dir).unwrap(), config);
+        assert!(key_file_path.exists());
 
         // check that the pem is readable
-        let data = fs::read_to_string(file_path).expect("couldn't read key file");
+        let data = fs::read_to_string(key_file_path).expect("couldn't read key file");
         let keys = Ed25519KeyPair::decode_base64(&data).expect("couldn't load keypair from file");
         assert_eq!(keys.public().as_bytes().len(), 32);
     }
 
     #[test]
     fn creates_config_file() {
-        let mut file_path = PathBuf::new();
-        file_path.push("/tmp/side");
-        let side_dir = file_path.clone();
-        file_path.push(utils::CONFIG_FILE);
+        let (config, name) = side_node_config();
+        let side_dir = format!("/tmp/side/{name}");
 
-        let _ = init(side_dir.clone(), default_side_node_config());
-        assert!(file_path.exists());
+        let mut config_file_path = PathBuf::new();
+        config_file_path.push(side_dir.clone());
+        config_file_path.push(utils::CONFIG_FILE);
+
+        let _ = init(PathBuf::from_str(&side_dir).unwrap(), config);
+        assert!(config_file_path.exists());
     }
 }

side-node/src/lib.rs

@@ -0,0 +1,76 @@
+use bft_crdt::websocket;
+use bft_crdt::TransactionList;
+use bft_json_crdt::json_crdt::{BaseCrdt, SignedOp};
+use cli::{parse_args, Commands};
+use node::SideNode;
+use tokio::{sync::mpsc, task};
+
+pub mod bft_crdt;
+pub mod bitcoin;
+pub(crate) mod cli;
+pub(crate) mod init;
+pub mod node;
+pub mod utils;
+
+#[tokio::main]
+pub async fn run() {
+    let args = parse_args();
+
+    match &args.command {
+        Some(Commands::Init { name }) => {
+            let config = init::config::SideNodeConfig {
+                name: name.to_string(),
+            };
+
+            let _ = init::init(utils::home(name), config);
+        }
+        Some(Commands::Run { name }) => {
+            let mut node = setup(name).await;
+            node.start().await;
+        }
+        Some(Commands::BtcTransfer {}) => {
+            let _ = bitcoin::driver::simple_transfer::run().await;
+        }
+
+        Some(Commands::BtcPolicyTransfer {}) => {
+            let _ = bitcoin::driver::policy_transfer::run().await;
+        }
+
+        Some(Commands::BtcHtlc {}) => {
+            let _ = bitcoin::driver::create_htlc::run().await;
+        }
+
+        None => println!("No command provided. Exiting. See --help for more information."),
+    }
+}
+
+/// Wire everything up outside the application so that we can test more easily later
+async fn setup(name: &String) -> SideNode {
+    // First, load up the keys and create a bft-bft-crdt
+    let side_dir = utils::home(name);
+    let bft_crdt_keys = bft_crdt::keys::load_from_file(&side_dir);
+    // let keys = bitcoin::keys::load_from_file(&side_dir).unwrap();
+    // let bitcoin_wallet =
+    //     bitcoin::clients::esplora::EsploraWallet::create_wallet(name, keys).unwrap();
+    let crdt = BaseCrdt::<TransactionList>::new(&bft_crdt_keys);
+
+    // Channels for internal communication, and a tokio task for stdin input
+    let (incoming_sender, incoming_receiver) = mpsc::channel::<SignedOp>(32);
+    let (stdin_sender, stdin_receiver) = std::sync::mpsc::channel();
+    task::spawn(async move {
+        bft_crdt::stdin::input(stdin_sender);
+    });
+
+    // Finally, create the node and return it
+    let handle = websocket::Client::new(incoming_sender).await;
+    let node = SideNode::new(
+        crdt,
+        bft_crdt_keys,
+        // bitcoin_wallet,
+        incoming_receiver,
+        stdin_receiver,
+        handle,
+    );
+    println!("Node setup complete.");
+    node
+}

side-node/src/main.rs

@@ -1,56 +1,5 @@
-use bft_json_crdt::json_crdt::{BaseCrdt, SignedOp};
-use cli::{parse_args, Commands};
-use crdt::TransactionList;
-use node::SideNode;
-use tokio::{sync::mpsc, task};
-use websocket::WebSocketClient;
+use side_node;
 
-pub(crate) mod cli;
-pub(crate) mod crdt;
-pub(crate) mod init;
-pub(crate) mod keys;
-pub(crate) mod node;
-pub(crate) mod stdin;
-pub(crate) mod utils;
-pub(crate) mod websocket;
-
-#[tokio::main]
-async fn main() {
-    let args = parse_args();
-
-    match &args.command {
-        Some(Commands::Init { name }) => {
-            let config = init::config::SideNodeConfig {
-                name: name.to_string(),
-            };
-
-            let _ = init::init(utils::home(name), config);
-        }
-        Some(Commands::Run { name }) => {
-            let mut node = setup(name).await;
-            node.start().await;
-        }
-        None => println!("No command provided. Exiting. See --help for more information."),
-    }
-}
-
-/// Wire everything up outside the application so we can test more easily later
-async fn setup(name: &String) -> SideNode {
-    // First, load up the keys and create a bft-crdt
-    let side_dir = utils::home(name);
-    let keys = keys::load_from_file(side_dir);
-    let crdt = BaseCrdt::<TransactionList>::new(&keys);
-
-    // Channels for internal communication, and a tokio task for stdin input
-    let (incoming_sender, incoming_receiver) = mpsc::channel::<SignedOp>(32);
-    let (stdin_sender, stdin_receiver) = std::sync::mpsc::channel();
-    task::spawn(async move {
-        stdin::input(stdin_sender);
-    });
-
-    // Finally, create the node and return it
-    let handle = WebSocketClient::new(incoming_sender).await;
-    let node = SideNode::new(crdt, keys, incoming_receiver, stdin_receiver, handle);
-    println!("Node setup complete.");
-    node
+fn main() {
+    side_node::run();
 }

side-node/src/node.rs

@@ -2,27 +2,30 @@ use bft_json_crdt::json_crdt::{BaseCrdt, SignedOp};
 use fastcrypto::ed25519::Ed25519KeyPair;
 use tokio::sync::mpsc;
 
-use crate::{crdt::TransactionList, utils, websocket::WebSocketClient};
+use crate::{bft_crdt::websocket::Client, bft_crdt::TransactionList, utils};
 
-pub(crate) struct SideNode {
+pub struct SideNode {
     crdt: BaseCrdt<TransactionList>,
-    keys: fastcrypto::ed25519::Ed25519KeyPair,
+    bft_crdt_keys: fastcrypto::ed25519::Ed25519KeyPair,
+    // _bitcoin_wallet: bdk::Wallet<MemoryDatabase>, // currently not read anywhere
     incoming_receiver: mpsc::Receiver<SignedOp>,
     stdin_receiver: std::sync::mpsc::Receiver<String>,
-    handle: ezsockets::Client<WebSocketClient>,
+    handle: ezsockets::Client<Client>,
 }
 
 impl SideNode {
-    pub(crate) fn new(
+    pub fn new(
         crdt: BaseCrdt<TransactionList>,
-        keys: Ed25519KeyPair,
+        bft_crdt_keys: Ed25519KeyPair,
+        // bitcoin_wallet: bdk::Wallet<MemoryDatabase>,
         incoming_receiver: mpsc::Receiver<SignedOp>,
         stdin_receiver: std::sync::mpsc::Receiver<String>,
-        handle: ezsockets::Client<WebSocketClient>,
+        handle: ezsockets::Client<Client>,
     ) -> Self {
         let node = Self {
             crdt,
-            keys,
+            bft_crdt_keys,
+            // _bitcoin_wallet: bitcoin_wallet,
             incoming_receiver,
             stdin_receiver,
             handle,
@@ -36,17 +39,18 @@ impl SideNode {
         loop {
             match self.stdin_receiver.try_recv() {
                 Ok(stdin) => {
-                    println!("Received stdin input: {:?}", stdin);
-                    let transaction = utils::fake_transaction(stdin);
+                    let transaction = utils::fake_generic_transaction_json(stdin);
                     let json = serde_json::to_value(transaction).unwrap();
                     let signed_op = self.add_transaction_local(json);
+                    println!("STDIN: {}", utils::shappy(signed_op.clone()));
                     self.send_to_network(signed_op).await;
                 }
                 Err(_) => {} // ignore empty channel errors in this PoC
             }
             match self.incoming_receiver.try_recv() {
                 Ok(incoming) => {
-                    self.handle_incoming(&incoming);
+                    println!("INCOMING: {}", utils::shappy(incoming.clone()));
+                    self.handle_incoming(incoming);
                 }
                 Err(_) => {} // ignore empty channel errors in this PoC
             }
@@ -54,17 +58,16 @@ impl SideNode {
     }
 
     async fn send_to_network(&self, signed_op: SignedOp) {
-        println!("sending to network: {:?}", signed_op);
         let to_send = serde_json::to_string(&signed_op).unwrap();
         self.handle.call(to_send).unwrap();
     }
 
-    fn handle_incoming(&mut self, incoming: &SignedOp) {
-        println!("WINNNINGINGINGINGINGIGNIGN");
-        self.crdt.apply(incoming.clone());
+    pub fn handle_incoming(&mut self, incoming: SignedOp) {
+        self.crdt.apply(incoming);
+        // self.trace_crdt();
     }
 
-    pub(crate) fn add_transaction_local(
+    pub fn add_transaction_local(
         &mut self,
         transaction: serde_json::Value,
     ) -> bft_json_crdt::json_crdt::SignedOp {
@@ -80,12 +83,17 @@ impl SideNode {
             .doc
             .list
             .insert(last.id, transaction)
-            .sign(&self.keys);
+            .sign(&self.bft_crdt_keys);
+        // self.trace_crdt();
         signed_op
     }
 
     /// Print the current state of the CRDT, can be used to debug
-    pub(crate) fn _trace_crdt(&self) {
-        println!("{:?}", self.crdt.doc.list);
+    pub fn trace_crdt(&self) {
+        println!("{:?}", self.crdt.doc.view_sha());
+    }
+
+    pub fn current_sha(&self) -> String {
+        self.crdt.doc.view_sha()
     }
 }

side-node/src/utils.rs

@@ -1,22 +1,27 @@
+use bft_json_crdt::json_crdt::SignedOp;
+use serde_json::{json, Value};
 use std::path::PathBuf;
 
-use serde_json::{json, Value};
-
-pub(crate) const KEY_FILE: &str = "keys.pem";
+pub(crate) const BITCOIN_KEY_FILE: &str = "bitcoin_keys.pem";
+pub(crate) const BFT_CRDT_KEY_FILE: &str = "keys.pem";
 pub(crate) const CONFIG_FILE: &str = "config.toml";
 
 /// Returns the path to the key file and config for this host OS.
-pub(crate) fn side_paths(prefix: PathBuf) -> (PathBuf, PathBuf) {
-    let mut key_path = prefix.clone();
-    key_path.push(KEY_FILE);
+pub(crate) fn side_paths(prefix: PathBuf) -> (PathBuf, PathBuf, PathBuf) {
+    let mut bft_crdt_key_path = prefix.clone();
+    bft_crdt_key_path.push(BFT_CRDT_KEY_FILE);
+
+    let mut bitcoin_key_path = prefix.clone();
+    bitcoin_key_path.push(BITCOIN_KEY_FILE);
 
     let mut config_path = prefix.clone();
     config_path.push(CONFIG_FILE);
 
-    (key_path, config_path)
+    (bft_crdt_key_path, bitcoin_key_path, config_path)
 }
 
-pub(crate) fn home(name: &String) -> std::path::PathBuf {
+/// Returns the path to the home directory for this host OS and the given node name
+pub(crate) fn home(name: &str) -> std::path::PathBuf {
     let mut path = dirs::home_dir().unwrap();
     path.push(".side");
     path.push(name);
@@ -24,10 +29,20 @@ pub(crate) fn home(name: &String) -> std::path::PathBuf {
 }
 
 /// Generate a fake transaction with customizable from_pubkey String
-pub(crate) fn fake_transaction(from_pubkey: String) -> Value {
+pub fn fake_generic_transaction_json(from: String) -> Value {
     json!({
-        "from": from_pubkey,
+        "from": from,
         "to": "Bob",
         "amount": 1
     })
 }
+
+pub fn shappy(op: SignedOp) -> String {
+    let b = serde_json::to_string(&op).unwrap().into_bytes();
+    sha256::digest(b).to_string()
+}
+
+pub fn sha256(text: String) -> String {
+    let b = text.into_bytes();
+    sha256::digest(b).to_string()
+}

side-node/tests/crdt.rs

@@ -0,0 +1,54 @@
+use bft_json_crdt::json_crdt::BaseCrdt;
+use bft_json_crdt::keypair::make_keypair;
+use bft_json_crdt::op::ROOT_ID;
+use side_node::bft_crdt::TransactionList;
+
+// case 1 - send valid updates
+#[test]
+fn test_valid_updates() {
+    // Insert to bft-crdt.doc on local node, test applying the same operation to a remote node
+    // and check that the view is the same
+    let keypair1 = make_keypair();
+    let mut crdt1 = BaseCrdt::<TransactionList>::new(&keypair1);
+
+    let val_a = side_node::utils::fake_generic_transaction_json(String::from("a"));
+    let val_b = side_node::utils::fake_generic_transaction_json(String::from("b"));
+    let val_c = side_node::utils::fake_generic_transaction_json(String::from("c"));
+
+    let _a = crdt1
+        .doc
+        .list
+        .insert(ROOT_ID, val_a.clone())
+        .sign(&keypair1);
+    let _b = crdt1
+        .doc
+        .list
+        .insert(_a.id(), val_b.clone())
+        .sign(&keypair1);
+    let _c = crdt1
+        .doc
+        .list
+        .insert(_b.id(), val_c.clone())
+        .sign(&keypair1);
+
+    let keypair2 = make_keypair();
+    let mut crdt2 = BaseCrdt::<TransactionList>::new(&keypair2);
+    crdt2.apply(_a.clone());
+    crdt2.apply(_b);
+    crdt2.apply(_c.clone());
+
+    assert_eq!(
+        crdt2.doc.list.view(),
+        crdt1.doc.list.view(),
+        "views should be equal"
+    );
+
+    crdt2.apply(_a.clone());
+    crdt2.apply(_a);
+
+    assert_eq!(
+        crdt1.doc.list.view(),
+        crdt2.doc.list.view(),
+        "views are still equal after repeated applies"
+    );
+}

side-node/tests/side_node.rs

@@ -0,0 +1,58 @@
+use bft_json_crdt::{
+    json_crdt::{BaseCrdt, SignedOp},
+    keypair::make_keypair,
+};
+use side_node::{bft_crdt::websocket::Client, bft_crdt::TransactionList, node::SideNode, utils};
+use tokio::sync::mpsc;
+
+#[tokio::test]
+async fn test_distribute_via_websockets() {
+    let mut node1 = setup("alice").await;
+    let mut node2 = setup("bob").await;
+
+    assert_eq!(node1.current_sha(), node2.current_sha());
+
+    let transaction = utils::fake_generic_transaction_json("from_alice".to_string());
+    let signed_op = node1.add_transaction_local(transaction);
+    node2.handle_incoming(signed_op);
+
+    assert_eq!(node1.current_sha(), node2.current_sha());
+
+    let transaction = utils::fake_generic_transaction_json("from_alice2".to_string());
+    let signed_op = node1.add_transaction_local(transaction);
+    node2.handle_incoming(signed_op);
+
+    assert_eq!(node1.current_sha(), node2.current_sha());
+
+    let transaction = utils::fake_generic_transaction_json("from_alice3".to_string());
+    let signed_op = node1.add_transaction_local(transaction);
+    node2.handle_incoming(signed_op);
+
+    assert_eq!(node1.current_sha(), node2.current_sha());
+}
+
+/// Wire everything up, ignoring things we are not using in the test
+async fn setup(_: &str) -> SideNode {
+    // First, load up the keys and create a bft-bft-crdt
+    let bft_crdt_keys = make_keypair();
+    // let mnemonic_words = bitcoin::keys::make_mnemonic();
+    // let keys = bitcoin::keys::get(mnemonic_words).unwrap();
+    // let bitcoin_wallet = bitcoin::clients::electrum::create_wallet(keys).unwrap();
+    let crdt = BaseCrdt::<TransactionList>::new(&bft_crdt_keys);
+
+    // Channels for internal communication, and a tokio task for stdin input
+    let (incoming_sender, incoming_receiver) = mpsc::channel::<SignedOp>(32);
+    let (_, stdin_receiver) = std::sync::mpsc::channel();
+
+    // Finally, create the node and return it
+    let handle = Client::new(incoming_sender).await;
+    let node = SideNode::new(
+        crdt,
+        bft_crdt_keys,
+        // bitcoin_wallet,
+        incoming_receiver,
+        stdin_receiver,
+        handle,
+    );
+    node
+}

side-watcher/Cargo.toml

@@ -8,6 +8,7 @@ edition = "2021"
 [dependencies]
 async-trait = "0.1.52"
 ezsockets = { version = "*", features = ["tungstenite"] }
+sha256 = "1.5.0"
 tokio = { version = "1.17.0", features = ["full"] }
 tracing = "0.1.32"
 tracing-subscriber = "0.3.9"

side-watcher/src/main.rs

@@ -85,12 +85,13 @@ impl ezsockets::ServerExt for ChatServer {
                     .unzip();
 
                 tracing::info!(
-                    "sending {text} to [{sessions}] at `{room}`",
+                    "sending {hash} to [{sessions}] at `{room}`",
                     sessions = ids
                         .iter()
                         .map(|id| id.to_string())
                         .collect::<Vec<_>>()
-                        .join(",")
+                        .join(","),
+                    hash = shappy(text.clone())
                 );
                 for session in sessions {
                     session.text(text.clone()).unwrap();
@@ -144,7 +145,7 @@ impl ezsockets::SessionExt for SessionActor {
     }
 
     async fn on_text(&mut self, text: String) -> Result<(), Error> {
-        tracing::info!("received: {text}");
+        tracing::info!("received: {}", shappy(text.clone()));
         if text.starts_with('/') {
             let mut args = text.split_whitespace();
             let command = args.next().unwrap();
@@ -183,6 +184,11 @@ impl ezsockets::SessionExt for SessionActor {
     }
 }
 
+fn shappy(text: String) -> String {
+    let b = text.into_bytes();
+    sha256::digest(b).to_string()
+}
+
 #[tokio::main]
 async fn main() {
     tracing_subscriber::fmt::init();