ed255/pod2
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Support persistent storage in Containers (#493)

Browse source 
Extend the work of https://github.com/0xPARC/pod2/pull/487 to the Containers (Dictionary, Set, Array).

The merkle tree only stores `RawValue` for both the key and the value, so it is the responsibility of the Container to store the rich value.

In order to handle containers with persistent storage efficiently (which means, cloning them or updating them should not cause an O(n) data copy) I figured we need to have a database of `Value`s indexed by their raw value; as this gives us deduplication and free cloning of containers.
The issue with this approach is that in the current design we have collisions between Value's of different types: https://github.com/0xPARC/pod2/issues/426 and the current API relies on the single type of values.

To resolve this issue I decided to change the API, instead of assuming that a Value has a fixed type, let the value be possibly multiple compatible types and let the user of the library try casting the Value to a particular type.
For this I deprecated the public access of everything related to `TypedValue` and I propose for it to be considered an implementation detail and a blackbox from the external developer point of view.  The `Value` type is now used like this:
- To create a new Value use `Value::from(...)` where you can pass any compatible type (the same types as before)
- To access the Value in typed form you cast it like `value.as_foo()` which returns `Option<Foo>`.

Previously we had a collision between `true` and `1` (and `false` and `0`).  Now it doesn't matter whether a value holds a `true` or a `1`, both should be seen as the same and both return `Some` when doing `as_int` and `as_bool`.

Similarly we had collisions with containers.  For example `set(0, 1, 2) == array[0, 1, 2]` and `set("a", "b") = dict("a": "a", "b": "b")`.  Now any container can be casted to any of `set, array, dict`.  There's a caveat here: each of these types expects a particular encoding of keys, so casting to the wrong type will return errors on some operations.

With this design it no longer matters what is being stored and recovered because the API requires the user to express the expected type and any type with collisions for particular values can be casted to the right type.

There's only one case where it's not desirable to swap one `TypedValue` for another: the `TypedValue::Raw`.  If a non-`RawValue` in the DB is replaced by the corresponding `RawValue` we erase the required information to recover the rich value.  For this reason the implementations of the database treat the `RawValue` as a special case: if an value is stored in non-`RawValue`, the corresponding `RawValue` can never overwrite it.  If a value is stored in `RawValue`, a matching non-`RawValue` will overwrite it (promoting it to a rich value).  This way we never lose data.

A consequence of this is that the serialization, `Display` and `Debug` of a container is not stable.  At any point any of the entries can be swapped for a "compatible" one if they share the storage with other containers that introduce collisions.

I rewrote all containers as wrapper to a generic `Container` which holds a `Map` from `Value` to `Value`.  The serialization of each container now uses the single implementation of the generic `Container`.
This commit is contained in:
Eduard S. 2026-03-23 12:31:28 +01:00 • committed by GitHub
parent 32f45872d7
commit 13cabdb511
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
22 changed files with 1187 additions and 621 deletions
Download patch file Download diff file Expand all files Collapse all files

2
.github/workflows/build.yml vendored
View file

@ -24,6 +24,8 @@ jobs:
run: cargo build --features metrics
- name: Build time
run: cargo build --features time
- name: Build db_rocksdb
run: cargo build --features db_rocksdb
- name: Build disk_cache
run: cargo build --no-default-features --features backend_plonky2,zk,disk_cache

3
.github/workflows/tests.yml vendored
View file

@ -17,4 +17,5 @@ jobs:
- name: Set up Rust
uses: actions-rust-lang/setup-rust-toolchain@v1
- name: Run tests
run: cargo test --release
# RocksDB is disabled by default but we still want to test it.
run: cargo test --release --features db_rocksdb

2
Cargo.toml
View file

@ -64,7 +64,7 @@ tempfile = "3"
vergen-gitcl = { version = "1.0.0", features = ["build"] }
[features]
default = ["backend_plonky2", "zk", "mem_cache", "db_rocksdb"]
default = ["backend_plonky2", "zk", "mem_cache"]
backend_plonky2 = ["plonky2"]
zk = []
metrics = []

4
src/backends/plonky2/error.rs
View file

@ -61,8 +61,8 @@ macro_rules! new {
}
use InnerError::*;
impl Error {
pub fn custom(s: String) -> Self {
new!(Custom(s))
pub fn custom(s: impl Into<String>) -> Self {
new!(Custom(s.into()))
}
pub fn plonky2_proof_fail(context: impl Into<String>, e: anyhow::Error) -> Self {
Self::Plonky2ProofFail(context.into(), e)

13
src/backends/plonky2/mainpod/mod.rs
View file

@ -225,11 +225,10 @@ pub(crate) fn extract_public_key_of(
) = (op, st)
{
let deduction_err = || MiddlewareError::invalid_deduction(op.clone(), st.clone());
let sk = SecretKey::try_from(
value_from_op(sk_s, sk_ref)
.ok_or_else(deduction_err)?
.typed(),
)?;
let value = value_from_op(sk_s, sk_ref).ok_or_else(deduction_err)?;
let sk = value
.as_secret_key()
.ok_or_else(|| Error::custom("{value} not SecretKey"))?;
aux_list[i] = OperationAux::PublicKeyOfIndex(table.len());
table.push(sk);
}
@ -283,7 +282,9 @@ pub(crate) fn extract_signatures(
aux_list[i] = OperationAux::SignedByIndex(table.len());
table.push(SignedBy {
msg: msg.raw(),
pk: PublicKey::try_from(pk.typed())?,
pk: pk
.as_public_key()
.ok_or_else(|| Error::custom(format!("{pk} is not PublicKey")))?,
sig: sig.clone(),
});
}

2
src/backends/plonky2/primitives/ec/curve.rs
View file

@ -207,7 +207,7 @@ impl Point {
u: *u,
});
points.find(|p| p.is_in_subgroup()).ok_or(Error::custom(
"One of the points must lie in the EC subgroup.".into(),
"One of the points must lie in the EC subgroup.",
))
}
pub fn as_bytes_from_subgroup(&self) -> Result<Vec<u8>, Error> {

42
src/backends/plonky2/primitives/merkletree/db/mod.rs
View file

@ -6,19 +6,20 @@ use std::{
sync::{Arc, Mutex},
};
use anyhow::{anyhow, bail, Result};
use anyhow::{anyhow, Result};
use dyn_clone::DynClone;
use crate::{
backends::plonky2::primitives::merkletree::{Leaf, Node},
middleware::{RawValue, EMPTY_VALUE},
backends::plonky2::primitives::merkletree::{Intermediate, Node},
middleware::{Hash, EMPTY_HASH},
};
#[cfg(feature = "db_rocksdb")]
pub mod rocks;
pub trait DB: Debug + DynClone + Sync + Send {
fn load_node(&self, hash: RawValue) -> Result<Node>;
/// Must always return the empty intermediate node when hash is EMPTY_HASH
fn load_node(&self, hash: Hash) -> Result<Option<Node>>;
fn store_node(&mut self, node: Node) -> Result<()>;
}
dyn_clone::clone_trait_object!(DB);
@ -26,7 +27,7 @@ dyn_clone::clone_trait_object!(DB);
/// MemDB implements the DB trait in a in-memory HashMap.
#[derive(Clone, Debug, Default)]
pub(crate) struct MemDB {
inner: Arc<Mutex<HashMap<RawValue, Node>>>,
inner: Arc<Mutex<HashMap<Hash, Node>>>,
}
impl MemDB {
@ -36,21 +37,18 @@ impl MemDB {
}
impl DB for MemDB {
fn load_node(&self, hash: RawValue) -> Result<Node> {
fn load_node(&self, hash: Hash) -> Result<Option<Node>> {
let db = self
.inner
.lock()
.map_err(|e| anyhow!("failed to acquire memdb lock for read: {}", e))?;
if let Some(node) = db.get(&hash) {
return Ok(node.clone());
if hash == EMPTY_HASH {
return Ok(Some(Node::Intermediate(Intermediate::new(
EMPTY_HASH, EMPTY_HASH,
))));
}
if hash == EMPTY_VALUE {
return Ok(Node::Leaf(Leaf::new(hash, EMPTY_VALUE)));
}
bail!("MemDB error: node not found: {}", hash);
Ok(db.get(&hash).cloned())
}
fn store_node(&mut self, node: Node) -> Result<()> {
@ -58,25 +56,15 @@ impl DB for MemDB {
.inner
.lock()
.map_err(|e| anyhow!("failed to acquire memdb lock for write: {}", e))?;
db.insert(node.hash().into(), node);
db.insert(node.hash(), node);
Ok(())
}
}
// NOTE: this can be replaced by `.to_bytes` & `from_bytes` optimized methods at `Node`
#[allow(dead_code)]
fn encode_node(node: &Node) -> Result<Vec<u8>> {
serde_json::to_vec(node).map_err(|e| anyhow!("failed to serialize node: {e}"))
}
#[allow(dead_code)]
fn decode_node(bytes: &[u8]) -> Result<Node> {
serde_json::from_slice(bytes).map_err(|e| anyhow!("failed to deserialize node: {e}"))
}
#[cfg(test)]
pub mod tests {
use super::*;
use super::{super::Leaf, *};
#[test]
fn test_db() -> Result<()> {
@ -97,7 +85,7 @@ pub mod tests {
let node = Leaf::new(1.into(), 1.into());
db.store_node(Node::Leaf(node.clone()))?;
let obtained_node = db.load_node(node.hash.into())?;
let obtained_node = db.load_node(node.hash)?.unwrap();
let leaf = match obtained_node {
Node::Leaf(l) => l,
_ => panic!("expected a leaf"),

35
src/backends/plonky2/primitives/merkletree/db/rocks.rs
View file

@ -3,10 +3,9 @@ use std::{fmt, path::Path, sync::Arc};
use anyhow::{anyhow, Result};
use rocksdb::{Options, TransactionDB, TransactionDBOptions};
use super::DB;
use crate::{
backends::plonky2::primitives::merkletree::{Leaf, Node},
middleware::{RawValue, EMPTY_VALUE},
backends::plonky2::primitives::merkletree::{self, db},
middleware::{Hash, RawValue, EMPTY_HASH},
};
#[derive(Clone)]
@ -30,29 +29,27 @@ impl fmt::Debug for RocksDB {
}
}
impl DB for RocksDB {
fn load_node(&self, hash: RawValue) -> Result<Node> {
if hash == EMPTY_VALUE {
return Ok(Node::Leaf(Leaf::new(hash, EMPTY_VALUE)));
impl db::DB for RocksDB {
fn load_node(&self, hash: Hash) -> Result<Option<merkletree::Node>> {
if hash == EMPTY_HASH {
return Ok(Some(merkletree::Node::Intermediate(
merkletree::Intermediate::new(EMPTY_HASH, EMPTY_HASH),
)));
}
let maybe_node_bytes = self
match self
.0
.get(hash.to_bytes())
.map_err(|e| anyhow!("rocksdb transaction get failed: {e}"))?;
match maybe_node_bytes {
Some(bytes) => super::decode_node(&bytes),
None => Err(anyhow!("rocksdb: node not found")),
.get(RawValue::from(hash).to_bytes())
.map_err(|e| anyhow!("rocksdb: get failed: {e}"))?
{
None => Ok(None),
Some(bytes) => Ok(Some(merkletree::Node::decode(bytes.as_ref())?)),
}
}
fn store_node(&mut self, node: Node) -> Result<()> {
fn store_node(&mut self, node: merkletree::Node) -> Result<()> {
self.0
.put(
RawValue::from(node.hash()).to_bytes(),
super::encode_node(&node)?,
)
.put(RawValue::from(node.hash()).to_bytes(), node.encode()?)
.map_err(|e| anyhow!("rocksdb transaction put failed: {e}"))
}
}

17
src/backends/plonky2/primitives/merkletree/error.rs
View file

@ -2,12 +2,16 @@
use std::{backtrace::Backtrace, fmt::Debug};
use crate::middleware::Hash;
pub type TreeResult<T, E = TreeError> = core::result::Result<T, E>;
#[derive(Debug, thiserror::Error)]
pub enum TreeInnerError {
#[error("key not found")]
KeyNotFound,
#[error("node with hash {0} not found")]
NodeNotFound(Hash),
#[error("key already exists")]
KeyExists,
#[error("max depth reached")]
@ -22,6 +26,9 @@ pub enum TreeInnerError {
StateTransitionProofFail(String),
#[error("circuit max_depth {0} is smaller than proof depth {1}")]
CircuitDepthTooSmall(usize, usize),
// Other
#[error("{0}")]
Custom(String),
}
#[derive(thiserror::Error)]
@ -31,8 +38,8 @@ pub enum TreeError {
inner: Box<TreeInnerError>,
backtrace: Box<Backtrace>,
},
#[error("anyhow::Error: {0}")]
Anyhow(#[from] anyhow::Error),
#[error("database error: {0}")]
Database(anyhow::Error),
}
impl Debug for TreeError {
@ -60,6 +67,9 @@ impl TreeError {
pub(crate) fn key_not_found() -> Self {
new!(KeyNotFound)
}
pub(crate) fn node_not_found(hash: Hash) -> Self {
new!(NodeNotFound(hash))
}
pub(crate) fn key_exists() -> Self {
new!(KeyExists)
}
@ -81,4 +91,7 @@ impl TreeError {
pub(crate) fn circuit_depth_too_small(circuit_depth: usize, proof_depth: usize) -> Self {
new!(CircuitDepthTooSmall(circuit_depth, proof_depth))
}
pub(crate) fn custom(s: impl Into<String>) -> Self {
new!(Custom(s.into()))
}
}

291
src/backends/plonky2/primitives/merkletree/mod.rs
View file

@ -2,7 +2,7 @@
//! <https://0xparc.github.io/pod2/merkletree.html> .
use std::{collections::HashMap, fmt, iter::IntoIterator};
use anyhow::{anyhow, Result};
use anyhow::anyhow;
use itertools::zip_eq;
use plonky2::{
field::types::Field,
@ -16,10 +16,15 @@ use crate::middleware::{Hash, RawValue, EMPTY_HASH, EMPTY_VALUE, F};
pub mod circuit;
pub use circuit::*;
mod db;
use db::DB;
pub mod db;
pub use db::DB;
pub mod error;
pub use error::{TreeError, TreeResult};
use error::{TreeError as Error, TreeResult as Result};
// TODO: Replace all `&RawValue` for `RawValue`. This type is very small and `Copy` so there's
// no benefit in passing a reference instead of a copy. Moreover, most of the times the value is
// being copied in methods that receive the reference: see all `*key` and `*value` in the code.
/// Theoretical max depth of a merkle tree. This limits appears because we store keys of 256 bits.
const MAX_DEPTH: usize = 256;
@ -39,6 +44,20 @@ impl PartialEq for MerkleTree {
}
impl Eq for MerkleTree {}
pub(crate) fn load_node(db: &dyn DB, hash: Hash) -> Result<Node> {
match db.load_node(hash) {
Err(e) => Err(Error::Database(e)),
Ok(None) => Err(Error::node_not_found(hash)),
Ok(Some(node)) => Ok(node),
}
}
fn store_node(db: &mut dyn DB, node: Node) -> Result<()> {
match db.store_node(node) {
Ok(_) => Ok(()),
Err(e) => Err(Error::Database(e)),
}
}
impl MerkleTree {
/// builds a new `MerkleTree` where the leaves contain the given key-values
pub fn new(kvs: &HashMap<RawValue, RawValue>) -> Self {
@ -92,18 +111,18 @@ impl MerkleTree {
new_key: RawValue, // key to be added/found at the leaf
mut siblings: Option<&mut Vec<Hash>>,
op: MerkleTreeOp,
) -> TreeResult<Option<(RawValue, RawValue)>> {
) -> Result<Option<(RawValue, RawValue)>> {
let (path, lvl) = path_and_lvl;
if lvl > MAX_DEPTH {
return Err(TreeError::max_depth());
return Err(Error::max_depth());
}
if curr_node_hash == EMPTY_HASH {
return Ok(None);
}
let node = db.load_node(curr_node_hash.into())?;
let node = load_node(db, curr_node_hash)?;
match node {
Node::Intermediate(n) => {
if path[lvl] {
@ -126,7 +145,7 @@ impl MerkleTree {
if new_key == old_leaf.key {
if op == MerkleTreeOp::Insert {
// in Insert, key should not exist
return Err(TreeError::key_exists());
return Err(Error::key_exists());
}
// we're at the operation Update/Delete case
return Ok(Some((old_leaf.key, old_leaf.value)));
@ -137,7 +156,7 @@ impl MerkleTree {
curr_node_hash.into(),
old_leaf.path,
path,
siblings.ok_or(anyhow!("expected siblings, got None"))?,
siblings.ok_or(Error::custom("expected siblings, got None"))?,
)?;
Ok(Some((old_leaf.key, old_leaf.value)))
}
@ -154,9 +173,9 @@ impl MerkleTree {
old_path: Vec<bool>,
new_path: Vec<bool>,
siblings: &mut Vec<Hash>,
) -> TreeResult<()> {
) -> Result<()> {
if lvl > MAX_DEPTH {
return Err(TreeError::max_depth());
return Err(Error::max_depth());
}
if old_path[lvl] == new_path[lvl] {
siblings.push(EMPTY_HASH);
@ -181,7 +200,7 @@ impl MerkleTree {
first_zeroes: bool,
) -> Result<Hash> {
// recall, in the delete case, the `key` is the `remaining_key`
let key_node = db.load_node(key.into())?;
let key_node = load_node(db, key)?;
if op == MerkleTreeOp::Delete
&& first_zeroes
&& matches!(key_node, Node::Leaf(..))
@ -208,7 +227,7 @@ impl MerkleTree {
let node_hash = node.hash; // variable to avoid cloning `node` later
// store in db
db.store_node(Node::Intermediate(node))?;
store_node(db, Node::Intermediate(node))?;
if curr_lvl == 0 {
return Ok(node_hash);
@ -217,7 +236,7 @@ impl MerkleTree {
}
/// returns the value at the given key
pub fn get(&self, key: &RawValue) -> TreeResult<RawValue> {
pub fn get(&self, key: &RawValue) -> Result<Option<RawValue>> {
let path = keypath(*key);
let key_resolution = Self::down(
self.db.as_ref(),
@ -228,13 +247,13 @@ impl MerkleTree {
MerkleTreeOp::ReadOnly,
)?;
match key_resolution {
Some((k, v)) if &k == key => Ok(v),
_ => Err(TreeError::key_not_found()),
Some((k, v)) if &k == key => Ok(Some(v)),
_ => Ok(None),
}
}
/// returns a boolean indicating whether the key exists in the tree
pub fn contains(&self, key: &RawValue) -> TreeResult<bool> {
pub fn contains(&self, key: &RawValue) -> Result<bool> {
let path = keypath(*key);
match Self::down(
self.db.as_ref(),
@ -253,7 +272,7 @@ impl MerkleTree {
&mut self,
key: &RawValue,
value: &RawValue,
) -> TreeResult<MerkleTreeStateTransitionProof> {
) -> Result<MerkleTreeStateTransitionProof> {
let proof_non_existence = self.prove_nonexistence(key)?;
let old_root: Hash = self.root;
@ -287,7 +306,7 @@ impl MerkleTree {
&mut self,
key: &RawValue,
value: &RawValue,
) -> TreeResult<MerkleTreeStateTransitionProof> {
) -> Result<MerkleTreeStateTransitionProof> {
let (old_value, old_proof) = self.prove(key)?;
let old_root: Hash = self.root;
@ -316,7 +335,7 @@ impl MerkleTree {
})
}
pub fn delete(&mut self, key: &RawValue) -> TreeResult<MerkleTreeStateTransitionProof> {
pub fn delete(&mut self, key: &RawValue) -> Result<MerkleTreeStateTransitionProof> {
let (value, proof_existence) = self.prove(key)?;
let old_root: Hash = self.root;
@ -346,7 +365,7 @@ impl MerkleTree {
/// returns a proof of existence, which proves that the given key exists in
/// the tree. It returns the `value` of the leaf at the given `key`, and the
/// `MerkleProof`.
pub fn prove(&self, key: &RawValue) -> TreeResult<(RawValue, MerkleProof)> {
pub fn prove(&self, key: &RawValue) -> Result<(RawValue, MerkleProof)> {
let path = keypath(*key);
let mut siblings: Vec<Hash> = Vec::new();
@ -366,7 +385,7 @@ impl MerkleTree {
other_leaf: None,
},
)),
_ => Err(TreeError::key_not_found()),
_ => Err(Error::key_not_found()),
}
}
@ -374,7 +393,7 @@ impl MerkleTree {
/// `key` does not exist in the tree. The return value specifies
/// the key-value pair in the leaf reached as a result of
/// resolving `key` as well as a `MerkleProof`.
pub fn prove_nonexistence(&self, key: &RawValue) -> TreeResult<MerkleProof> {
pub fn prove_nonexistence(&self, key: &RawValue) -> Result<MerkleProof> {
let path = keypath(*key);
let mut siblings: Vec<Hash> = Vec::new();
@ -400,22 +419,17 @@ impl MerkleTree {
siblings,
other_leaf: Some((k, v)),
}),
_ => Err(TreeError::key_exists()),
_ => Err(Error::key_exists()),
}
// both cases prove that the given key don't exist in the tree.
}
/// verifies an inclusion proof for the given `key` and `value`
pub fn verify(
root: Hash,
proof: &MerkleProof,
key: &RawValue,
value: &RawValue,
) -> TreeResult<()> {
pub fn verify(root: Hash, proof: &MerkleProof, key: &RawValue, value: &RawValue) -> Result<()> {
let h = proof.compute_root_from_leaf(key, Some(*value))?;
if h != root {
Err(TreeError::proof_fail("inclusion".to_string()))
Err(Error::proof_fail("inclusion".to_string()))
} else {
Ok(())
}
@ -423,18 +437,16 @@ impl MerkleTree {
/// verifies a non-inclusion proof for the given `key`, that is, the given
/// `key` does not exist in the tree
pub fn verify_nonexistence(root: Hash, proof: &MerkleProof, key: &RawValue) -> TreeResult<()> {
pub fn verify_nonexistence(root: Hash, proof: &MerkleProof, key: &RawValue) -> Result<()> {
match proof.other_leaf {
Some((k, _v)) if &k == key => {
Err(TreeError::invalid_proof("non-existence".to_string()))
}
Some((k, _v)) if &k == key => Err(Error::invalid_proof("non-existence".to_string())),
_ => {
let k = proof.other_leaf.map(|(k, _)| k).unwrap_or(*key);
let v: Option<RawValue> = proof.other_leaf.map(|(_, v)| v);
let h = proof.compute_root_from_leaf(&k, v)?;
if h != root {
Err(TreeError::proof_fail("exclusion".to_string()))
Err(Error::proof_fail("exclusion".to_string()))
} else {
Ok(())
}
@ -442,7 +454,7 @@ impl MerkleTree {
}
}
pub fn verify_state_transition(proof: &MerkleTreeStateTransitionProof) -> TreeResult<()> {
pub fn verify_state_transition(proof: &MerkleTreeStateTransitionProof) -> Result<()> {
let mut old_siblings = proof.op_proof.siblings.clone();
let new_siblings = proof.siblings.clone();
@ -459,7 +471,7 @@ impl MerkleTree {
}
MerkleTreeOp::Update => {
if proof.value.is_none() {
return Err(TreeError::state_transition_fail(
return Err(Error::state_transition_fail(
"Invalid proof of update: proof.value should not be None".to_string(),
));
}
@ -485,7 +497,7 @@ impl MerkleTree {
// All siblings should agree
(proof.siblings == proof.op_proof.siblings)
.then_some(())
.ok_or(TreeError::state_transition_fail(format!(
.ok_or(Error::state_transition_fail(format!(
"Invalid proof of update for key {}: Siblings don't match.",
proof.op_key
)))
@ -514,11 +526,11 @@ impl MerkleTree {
let divergence_lvl: usize =
match zip_eq(old_path, new_path).position(|(x, y)| x != y) {
Some(d) => d,
None => return Err(TreeError::max_depth()),
None => return Err(Error::max_depth()),
};
if divergence_lvl != new_siblings.len() - 1 {
return Err(TreeError::state_transition_fail(
return Err(Error::state_transition_fail(
"paths divergence does not match".to_string(),
));
}
@ -534,7 +546,7 @@ impl MerkleTree {
if new_siblings.is_empty() {
return (old_siblings.is_empty() && proof.old_root == EMPTY_HASH)
.then_some(())
.ok_or(TreeError::state_transition_fail(
.ok_or(Error::state_transition_fail(
"new tree has no siblings yet old tree is not the empty tree"
.to_string(),
));
@ -544,14 +556,14 @@ impl MerkleTree {
old_siblings.resize(d + 1, EMPTY_HASH);
for i in 0..d {
if old_siblings[i] != new_siblings[i] {
return Err(TreeError::state_transition_fail(
return Err(Error::state_transition_fail(
"siblings don't match: old[i]!=new[i] ∀ i (except at i==d)".to_string(),
));
}
}
if old_siblings[d] != new_siblings[d] {
if old_siblings[d] != EMPTY_HASH {
return Err(TreeError::state_transition_fail(
return Err(Error::state_transition_fail(
"siblings don't match: old[d]!=empty".to_string(),
));
}
@ -559,20 +571,20 @@ impl MerkleTree {
.op_proof
.other_leaf
.map(|(k, _)| k)
.ok_or(TreeError::state_transition_fail(
.ok_or(Error::state_transition_fail(
"proof.proof_non_existence.other_leaf can not be empty for the case old_siblings[d]!=new_siblings[d]".to_string()
))?;
let v: Option<RawValue> = proof.op_proof.other_leaf.map(|(_, v)| v);
let old_leaf_hash = kv_hash(&k, v);
if new_siblings[d] != old_leaf_hash {
return Err(TreeError::state_transition_fail(
return Err(Error::state_transition_fail(
"siblings don't match: new[d]!=old_leaf_hash".to_string(),
));
}
}
Ok(())
}
_ => Err(TreeError::invalid_proof("proof.op".to_string())),
_ => Err(Error::invalid_proof("proof.op".to_string())),
}
}
}
@ -586,27 +598,25 @@ impl MerkleTree {
root: Hash,
k: RawValue,
maybe_value: Option<RawValue>,
) -> TreeResult<Hash> {
) -> Result<Hash> {
// Rule out invalid arguments
match (op, maybe_value) {
(MerkleTreeOp::Insert, None) | (MerkleTreeOp::Update, None) => {
Err(TreeError::invalid_state_transition_proof_arg(format!(
Err(Error::invalid_state_transition_proof_arg(format!(
"{:?} op requires a value argument.",
op
)))
}
(MerkleTreeOp::Delete, Some(_)) => {
Err(TreeError::invalid_state_transition_proof_arg(format!(
Err(Error::invalid_state_transition_proof_arg(format!(
"{:?} op requires no value argument, yet one was provided.",
op
)))
}
(MerkleTreeOp::ReadOnly, _) => {
Err(TreeError::invalid_state_transition_proof_arg(format!(
(MerkleTreeOp::ReadOnly, _) => Err(Error::invalid_state_transition_proof_arg(format!(
"{:?} 'read only' op should not reach the 'apply_op' method",
op
)))
}
))),
_ => Ok(()),
}?;
@ -627,8 +637,7 @@ impl MerkleTree {
Node::Leaf(Leaf::new(k, value))
}
(MerkleTreeOp::Delete, None) => {
// return an intermediate node whose hash is 'empty', to
// indicate that there is no leaf
// return a node whose hash is 'empty', to indicate that there is no leaf
Node::Intermediate(Intermediate {
hash: EMPTY_HASH,
left: EMPTY_HASH,
@ -636,16 +645,16 @@ impl MerkleTree {
})
}
_ => {
return Err(TreeError::invalid_state_transition_proof_arg(format!(
return Err(Error::invalid_state_transition_proof_arg(format!(
"{:?} op has invalid value type: {:?}",
op, maybe_value
)))
}
};
let node_hash = node.hash(); // variable to avoid cloning `node` later
db.store_node(node)?;
let node_hash = node.hash(); // variable to avoid cloning `leaf` later
store_node(db, node)?;
if siblings.is_empty() {
// return the node's hash as root
// return the leaf's hash as root
return Ok(node_hash);
}
@ -654,7 +663,7 @@ impl MerkleTree {
// we're at the root-1 level, there is only a sibling, and we're
// removing the current leaf.
// If the sibling is a Leaf, the sibling (leaf) is now the new root
let sibling_node = db.load_node(siblings[0].into())?;
let sibling_node = load_node(db, siblings[0])?;
if matches!(sibling_node, Node::Leaf(..)) {
return Ok(siblings[0]);
}
@ -669,7 +678,7 @@ impl MerkleTree {
let node_hash = node.hash; // variable to avoid cloning `node` later
// store in db
db.store_node(Node::Intermediate(node))?;
store_node(db, Node::Intermediate(node))?;
return Ok(node_hash);
}
// use the last sibling as the key that we will push up from
@ -743,48 +752,39 @@ fn hash_with_flag(flag: F, inputs: &[F]) -> Hash {
impl MerkleTree {
/// returns an iterator over the leaves of the tree
pub fn iter(&self) -> Iter<'_> {
pub fn iter(&self) -> Iter {
Iter {
state: if self.root == EMPTY_HASH {
vec![]
} else {
vec![self.root]
},
db: self.db.as_ref(),
db: self.db.clone(),
}
}
}
impl<'a> IntoIterator for &'a MerkleTree {
impl IntoIterator for &MerkleTree {
type Item = (RawValue, RawValue);
type IntoIter = Iter<'a>;
type IntoIter = Iter;
fn into_iter(self) -> Self::IntoIter {
self.iter()
}
}
pub struct Iter<'a> {
pub struct Iter {
state: Vec<Hash>,
db: &'a dyn DB,
db: Box<dyn DB>,
}
impl<'a> Iterator for Iter<'a> {
impl Iterator for Iter {
type Item = (RawValue, RawValue);
fn next(&mut self) -> Option<Self::Item> {
let node_hash = self.state.pop()?;
// Inspect node
let node = self.db.load_node(node_hash.into()).ok()?;
let node = load_node(self.db.as_ref(), node_hash).ok()?;
match node {
Node::Leaf(Leaf {
hash: _,
path: _,
key,
value,
}) => Some((key, value)),
Node::Intermediate(Intermediate {
hash: _,
left,
right,
}) => {
Node::Leaf(Leaf { key, value, .. }) => Some((key, value)),
Node::Intermediate(Intermediate { left, right, .. }) => {
[right, left].into_iter().for_each(|h| {
if h != EMPTY_HASH {
self.state.push(h)
@ -814,7 +814,7 @@ fn print_graph_viz(f: &mut fmt::Formatter<'_>, db: &dyn DB, hash: Hash) -> fmt::
return Ok(());
}
let node = db.load_node(hash.into()).map_err(|_| fmt::Error)?;
let node = load_node(db, hash).map_err(|_| fmt::Error)?;
match node {
Node::Intermediate(n) => {
let left_hash: String = if n.left == EMPTY_HASH {
@ -878,14 +878,14 @@ impl MerkleProof {
/// Computes the root of the Merkle tree suggested by a Merkle proof given a
/// key & value. If a value is not provided, the terminal node is assumed to
/// be empty.
fn compute_root_from_leaf(&self, key: &RawValue, value: Option<RawValue>) -> TreeResult<Hash> {
fn compute_root_from_leaf(&self, key: &RawValue, value: Option<RawValue>) -> Result<Hash> {
let path = keypath(*key);
let h = kv_hash(key, value);
self.compute_root_from_node(&h, path)
}
fn compute_root_from_node(&self, node_hash: &Hash, path: Vec<bool>) -> TreeResult<Hash> {
fn compute_root_from_node(&self, node_hash: &Hash, path: Vec<bool>) -> Result<Hash> {
if self.siblings.len() > MAX_DEPTH {
return Err(TreeError::max_depth());
return Err(Error::max_depth());
}
let mut h = *node_hash;
for (i, sibling) in self.siblings.iter().enumerate().rev() {
@ -1001,19 +1001,17 @@ pub enum Node {
impl Node {
pub fn hash(&self) -> Hash {
match self {
Node::Leaf(Leaf {
hash,
path: _,
key: _,
value: _,
}) => *hash,
Node::Intermediate(Intermediate {
hash,
left: _,
right: _,
}) => *hash,
Node::Leaf(Leaf { hash, .. }) => *hash,
Node::Intermediate(Intermediate { hash, .. }) => *hash,
}
}
// NOTE: this can be replaced by `.to_bytes` & `from_bytes` optimized methods at `Node`
pub fn encode(&self) -> Result<Vec<u8>, anyhow::Error> {
serde_json::to_vec(self).map_err(|e| anyhow!("failed to serialize node: {e}"))
}
pub fn decode(bytes: &[u8]) -> Result<Node, anyhow::Error> {
serde_json::from_slice(bytes).map_err(|e| anyhow!("failed to deserialize node: {e}"))
}
}
#[derive(Clone, Debug, Serialize, Deserialize)]
@ -1023,7 +1021,7 @@ pub struct Intermediate {
right: Hash,
}
impl Intermediate {
fn new(left: Hash, right: Hash) -> Self {
pub fn new(left: Hash, right: Hash) -> Self {
if left == EMPTY_HASH && right == EMPTY_HASH {
return Self {
hash: EMPTY_HASH,
@ -1045,7 +1043,7 @@ pub struct Leaf {
pub(crate) value: RawValue,
}
impl Leaf {
fn new(key: RawValue, value: RawValue) -> Self {
pub fn new(key: RawValue, value: RawValue) -> Self {
Self {
hash: kv_hash(&key, Some(value)),
path: keypath(key),
@ -1079,21 +1077,21 @@ pub mod tests {
use super::*;
#[test]
fn test_merkletree() -> TreeResult<()> {
fn test_merkletree() -> Result<()> {
let db = Box::new(db::MemDB::new());
test_merkletree_opt(db)?;
#[cfg(feature = "db_rocksdb")]
{
let db = Box::new(db::rocks::RocksDB::open(
tempfile::TempDir::new().unwrap().path(),
)?);
let db = Box::new(
db::rocks::RocksDB::open(tempfile::TempDir::new().unwrap().path()).unwrap(),
);
test_merkletree_opt(db)?;
}
Ok(())
}
fn test_merkletree_opt(db: Box<dyn DB>) -> TreeResult<()> {
fn test_merkletree_opt(db: Box<dyn DB>) -> Result<()> {
let mut kvs = HashMap::new();
for i in 0..8 {
if i == 1 {
@ -1168,21 +1166,40 @@ pub mod tests {
}
#[test]
fn test_delete_to_empty() -> TreeResult<()> {
fn test_key_not_found() -> Result<()> {
let db = Box::new(db::MemDB::new());
let mut tree = MerkleTree::empty_with_db(db.clone());
let value_option = tree.get(&RawValue::from(5)).unwrap();
assert_eq!(None, value_option);
tree.insert(&RawValue::from(1), &RawValue::from(42))?;
let value_option = tree.get(&RawValue::from(5)).unwrap();
assert_eq!(None, value_option);
// If the root doesn't exist there should be an error
let tree = MerkleTree::from_db(Hash::from(RawValue::from(42)), db);
let result = tree.get(&RawValue::from(5));
assert!(result.is_err());
Ok(())
}
#[test]
fn test_delete_to_empty() -> Result<()> {
let db = Box::new(db::MemDB::new());
test_delete_to_empty_opt(db)?;
#[cfg(feature = "db_rocksdb")]
{
let db = Box::new(db::rocks::RocksDB::open(
tempfile::TempDir::new().unwrap().path(),
)?);
let db = Box::new(
db::rocks::RocksDB::open(tempfile::TempDir::new().unwrap().path()).unwrap(),
);
test_delete_to_empty_opt(db)?;
}
Ok(())
}
fn test_delete_to_empty_opt(db: Box<dyn DB>) -> TreeResult<()> {
fn test_delete_to_empty_opt(db: Box<dyn DB>) -> Result<()> {
let mut tree = MerkleTree::new_with_db(db, &HashMap::new())?;
let (key, value) = (RawValue::from(2), RawValue::from(1002));
@ -1212,21 +1229,21 @@ pub mod tests {
}
#[test]
fn test_prove_verify() -> TreeResult<()> {
fn test_prove_verify() -> Result<()> {
let db = Box::new(db::MemDB::new());
test_prove_verify_opt(db)?;
#[cfg(feature = "db_rocksdb")]
{
let db = Box::new(db::rocks::RocksDB::open(
tempfile::TempDir::new().unwrap().path(),
)?);
let db = Box::new(
db::rocks::RocksDB::open(tempfile::TempDir::new().unwrap().path()).unwrap(),
);
test_prove_verify_opt(db)?;
}
Ok(())
}
fn test_prove_verify_opt(db: Box<dyn DB>) -> TreeResult<()> {
fn test_prove_verify_opt(db: Box<dyn DB>) -> Result<()> {
let kvs = [
(1.into(), 55.into()),
(2.into(), 88.into()),
@ -1255,21 +1272,21 @@ pub mod tests {
}
#[test]
fn test_update_leaf() -> TreeResult<()> {
fn test_update_leaf() -> Result<()> {
let db = Box::new(db::MemDB::new());
test_update_leaf_opt(db)?;
#[cfg(feature = "db_rocksdb")]
{
let db = Box::new(db::rocks::RocksDB::open(
tempfile::TempDir::new().unwrap().path(),
)?);
let db = Box::new(
db::rocks::RocksDB::open(tempfile::TempDir::new().unwrap().path()).unwrap(),
);
test_update_leaf_opt(db)?;
}
Ok(())
}
fn test_update_leaf_opt(db: Box<dyn DB>) -> TreeResult<()> {
fn test_update_leaf_opt(db: Box<dyn DB>) -> Result<()> {
let kvs = [
(1.into(), 1.into()),
(9.into(), 9.into()),
@ -1304,21 +1321,21 @@ pub mod tests {
}
#[test]
fn test_update_delete_leaf() -> TreeResult<()> {
fn test_update_delete_leaf() -> Result<()> {
let db = Box::new(db::MemDB::new());
test_update_delete_leaf_opt(db)?;
#[cfg(feature = "db_rocksdb")]
{
let db = Box::new(db::rocks::RocksDB::open(
tempfile::TempDir::new().unwrap().path(),
)?);
let db = Box::new(
db::rocks::RocksDB::open(tempfile::TempDir::new().unwrap().path()).unwrap(),
);
test_update_delete_leaf_opt(db)?;
}
Ok(())
}
fn test_update_delete_leaf_opt(db: Box<dyn DB>) -> TreeResult<()> {
fn test_update_delete_leaf_opt(db: Box<dyn DB>) -> Result<()> {
let kvs: HashMap<RawValue, RawValue> = (0..10)
.map(|i| (i.into(), i.into()))
.collect::<HashMap<_, _>>();
@ -1348,21 +1365,21 @@ pub mod tests {
}
#[test]
fn test_delete_leaf() -> TreeResult<()> {
fn test_delete_leaf() -> Result<()> {
let db = Box::new(db::MemDB::new());
test_delete_leaf_opt(db)?;
#[cfg(feature = "db_rocksdb")]
{
let db = Box::new(db::rocks::RocksDB::open(
tempfile::TempDir::new().unwrap().path(),
)?);
let db = Box::new(
db::rocks::RocksDB::open(tempfile::TempDir::new().unwrap().path()).unwrap(),
);
test_delete_leaf_opt(db)?;
}
Ok(())
}
fn test_delete_leaf_opt(db: Box<dyn DB>) -> TreeResult<()> {
fn test_delete_leaf_opt(db: Box<dyn DB>) -> Result<()> {
let kvs = [(1.into(), 1.into()), (9.into(), 9.into())]
.into_iter()
.collect();
@ -1403,21 +1420,21 @@ pub mod tests {
}
#[test]
fn test_delete_from_two_leaves() -> TreeResult<()> {
fn test_delete_from_two_leaves() -> Result<()> {
let db = Box::new(db::MemDB::new());
test_delete_from_two_leaves_opt(db)?;
#[cfg(feature = "db_rocksdb")]
{
let db = Box::new(db::rocks::RocksDB::open(
tempfile::TempDir::new().unwrap().path(),
)?);
let db = Box::new(
db::rocks::RocksDB::open(tempfile::TempDir::new().unwrap().path()).unwrap(),
);
test_delete_from_two_leaves_opt(db)?;
}
Ok(())
}
fn test_delete_from_two_leaves_opt(db: Box<dyn DB>) -> TreeResult<()> {
fn test_delete_from_two_leaves_opt(db: Box<dyn DB>) -> Result<()> {
// tree with two leaves whose keys diverge at the first bit, so that when
// deleting one key leads to a tree with a single Leaf as a root
let mut kvs = HashMap::new();
@ -1439,21 +1456,21 @@ pub mod tests {
}
#[test]
fn test_state_transition() -> TreeResult<()> {
fn test_state_transition() -> Result<()> {
let db = Box::new(db::MemDB::new());
test_state_transition_opt(db)?;
#[cfg(feature = "db_rocksdb")]
{
let db = Box::new(db::rocks::RocksDB::open(
tempfile::TempDir::new().unwrap().path(),
)?);
let db = Box::new(
db::rocks::RocksDB::open(tempfile::TempDir::new().unwrap().path()).unwrap(),
);
test_state_transition_opt(db)?;
}
Ok(())
}
fn test_state_transition_opt(db: Box<dyn DB>) -> TreeResult<()> {
fn test_state_transition_opt(db: Box<dyn DB>) -> Result<()> {
let mut kvs = HashMap::new();
for i in 0..8 {
kvs.insert(RawValue::from(i), RawValue::from(1000 + i));

6
src/examples/mod.rs
View file

@ -180,11 +180,7 @@ impl EthDosHelper {
};
assert_eq!(int, Value::from(int_attestation.public_key));
let n_i64 = if let TypedValue::Int(x) = n.typed() {
*x
} else {
panic!("distance value is not Int")
};
let n_i64 = n.as_int().unwrap();
// eth_dos src->dst dist=n+1
self.n_plus_1(&mut pod, eth_dos_int_to_dst, int_attestation, n_i64)?;

49
src/frontend/mod.rs
View file

@ -13,10 +13,11 @@ use serde::{Deserialize, Serialize};
pub use serialization::SerializedMainPod;
use crate::middleware::{
self, check_custom_pred, containers::Dictionary, fill_wildcard_values, hash_op, max_op,
prod_op, sum_op, AnchoredKey, Hash, Key, MainPodInputs, MainPodProver, NativeOperation,
OperationAux, OperationType, Params, PublicKey, RawValue, Signature, Signer, Statement,
StatementArg, VDSet, Value, ValueRef,
self, check_custom_pred,
containers::{Container, Dictionary},
fill_wildcard_values, hash_op, max_op, prod_op, sum_op, AnchoredKey, Hash, Key, MainPodInputs,
MainPodProver, NativeOperation, OperationAux, OperationType, Params, PublicKey, RawValue,
Signature, Signer, Statement, StatementArg, VDSet, Value, ValueRef, EMPTY_VALUE,
};
mod custom;
@ -92,8 +93,11 @@ impl fmt::Display for SignedDict {
// https://0xparc.github.io/pod2/merkletree.html will not need it since it will be
// deterministic based on the keys values not on the order of the keys when added into the
// tree.
for (k, v) in self.dict.kvs().iter().sorted_by_key(|kv| kv.0.hash()) {
writeln!(f, " - {} = {}", k, v)?;
for kv in self.dict.iter() {
match kv {
Ok((k, v)) => writeln!(f, " - {} = {}", k, v)?,
Err(e) => writeln!(f, " - ERR: {}", e)?,
}
}
Ok(())
}
@ -106,16 +110,13 @@ impl SignedDict {
.then_some(())
.ok_or(Error::custom("Invalid signature!"))
}
pub fn kvs(&self) -> &HashMap<Key, Value> {
self.dict.kvs()
}
pub fn get(&self, key: impl Into<Key>) -> Option<&Value> {
self.kvs().get(&key.into())
pub fn get(&self, key: impl Into<Key>) -> Option<Value> {
self.dict.get(&key.into()).unwrap()
}
// Returns the Contains statement that defines key if it exists.
pub fn get_statement(&self, key: impl Into<Key>) -> Option<Statement> {
let key: Key = key.into();
self.kvs().get(&key).map(|value| {
self.dict.get(&key).unwrap().map(|value| {
Statement::Contains(
ValueRef::Literal(Value::from(self.dict.clone())),
ValueRef::Literal(Value::from(key.name())),
@ -156,6 +157,11 @@ impl fmt::Display for MainPodBuilder {
}
}
fn as_container_or_err(v: &Value) -> Result<Container> {
v.as_container()
.ok_or_else(|| Error::custom(format!("{v} not a container")))
}
impl MainPodBuilder {
pub fn new(params: &Params, vd_set: &VDSet) -> Self {
Self {
@ -347,11 +353,12 @@ impl MainPodBuilder {
.ok_or(Error::custom(format!(
"Invalid key argument for op {}.",
op
)))?;
)))?
.raw();
let proof = if op_type == &Native(ContainsFromEntries) {
container.prove_existence(key)?.1
as_container_or_err(container)?.prove(key)?.1
} else {
container.prove_nonexistence(key)?
as_container_or_err(container)?.prove_nonexistence(key)?
};
Ok(Operation(op_type.clone(), op.1, OpAux::MerkleProof(proof)))
}
@ -375,18 +382,16 @@ impl MainPodBuilder {
let value =
op.1.get(3)
.and_then(|arg| arg.value())
.ok_or(Error::custom(format!(
"Invalid key argument for op {}.",
op
)));
.cloned()
.unwrap_or(Value::from(EMPTY_VALUE));
let proof = match op_type {
Native(ContainerInsertFromEntries) => {
old_container.prove_insertion(key, value?)?
as_container_or_err(old_container)?.insert(key.clone(), value)?
}
Native(ContainerUpdateFromEntries) => {
old_container.prove_update(key, value?)?
as_container_or_err(old_container)?.update(key.raw(), value)?
}
_ => old_container.prove_deletion(key)?,
_ => as_container_or_err(old_container)?.delete(key.raw())?,
};
Ok(Operation(
op_type.clone(),

11
src/frontend/operation.rs
View file

@ -4,7 +4,7 @@ use crate::{
frontend::SignedDict,
middleware::{
containers::Dictionary, root_key_to_ak, CustomPredicateRef, NativeOperation, OperationAux,
OperationType, Signature, Statement, TypedValue, Value, ValueRef,
OperationType, Signature, Statement, Value, ValueRef,
},
};
@ -39,10 +39,9 @@ impl OperationArg {
}
pub(crate) fn int_value_and_ref(&self) -> Option<(ValueRef, i64)> {
self.value_and_ref().and_then(|(r, v)| match v.typed() {
&TypedValue::Int(i) => Some((r, i)),
_ => None,
})
self.value_and_ref()
.and_then(|(r, v)| v.as_int().map(|i| Some((r, i))))
.flatten()
}
}
@ -71,7 +70,7 @@ impl From<&Value> for OperationArg {
impl From<(&Dictionary, &str)> for OperationArg {
fn from((dict, key): (&Dictionary, &str)) -> Self {
// TODO: Use TryFrom
let value = dict.get(&key.into()).cloned().unwrap();
let value = dict.get(&key.into()).unwrap().unwrap();
Self::Statement(Statement::Contains(
dict.clone().into(),
key.into(),

33
src/frontend/serialization.rs
View file

@ -83,7 +83,7 @@ mod tests {
middleware::{
self,
containers::{Array, Dictionary, Set},
Params, Signer as _, TypedValue, DEFAULT_VD_LIST,
Params, Signer as _, Value, DEFAULT_VD_LIST,
},
};
@ -91,16 +91,15 @@ mod tests {
fn test_value_serialization() {
// Pairs of values and their expected serialized representations
let values = vec![
(TypedValue::String("hello".to_string()), "\"hello\""),
(TypedValue::Int(42), "{\"Int\":\"42\"}"),
(TypedValue::Bool(true), "true"),
(Value::from("hello"), "\"hello\""),
(Value::from(42), "{\"Int\":\"42\"}"),
(Value::from(true), r#"{"Int":"1"}"#),
(
TypedValue::Array(Array::new(vec!["foo".into(), false.into()])),
"{\"array\":[\"foo\",false]}",
Value::from(Array::new(vec![Value::from("foo"), Value::from(false)])),
r#"{"inner":[[{"Int":"0"},"foo"],[{"Int":"1"},{"Int":"0"}]]}"#,
),
(
TypedValue::Dictionary(
Dictionary::new(HashMap::from([
Value::from(Dictionary::new(HashMap::from([
// The set of valid keys is equal to the set of valid JSON keys
("foo".into(), 123.into()),
// Empty strings are valid JSON keys
@ -113,26 +112,25 @@ mod tests {
(("\0".into()), "".into()),
// Keys can contain emojis
(("🥳".into()), "party time!".into()),
]))
),
"{\"kvs\":{\"\":\"baz\",\"\\u0000\":\"\",\" hi\":false,\"!@£$%^&&*()\":\"\",\"foo\":{\"Int\":\"123\"},\"🥳\":\"party time!\"}}",
]))),
r#"{"inner":[["!@£$%^&&*()",""],["🥳","party time!"],[" hi",{"Int":"0"}],["foo",{"Int":"123"}],["\u0000",""],["","baz"]]}"#,
),
(
TypedValue::Set(Set::new(HashSet::from(["foo".into(), "bar".into()]))),
"{\"set\":[\"bar\",\"foo\"]}",
Value::from(Set::new(HashSet::from(["foo".into(), "bar".into()]))),
r#"{"inner":[["bar"],["foo"]]}"#,
),
];
for (value, expected) in values {
let serialized = serde_json::to_string(&value).unwrap();
assert_eq!(serialized, expected);
let deserialized: TypedValue = serde_json::from_str(&serialized).unwrap();
let deserialized: Value = serde_json::from_str(&serialized).unwrap();
assert_eq!(
value, deserialized,
"value {:#?} should equal deserialized {:#?}",
value, deserialized
);
let expected_deserialized: TypedValue = serde_json::from_str(expected).unwrap();
let expected_deserialized: Value = serde_json::from_str(expected).unwrap();
assert_eq!(value, expected_deserialized);
}
}
@ -177,7 +175,10 @@ mod tests {
"deserialized: {}",
serde_json::to_string_pretty(&deserialized).unwrap()
);
assert_eq!(signed_dict.dict.kvs(), deserialized.dict.kvs());
assert_eq!(
signed_dict.dict.dump().unwrap(),
deserialized.dict.dump().unwrap()
);
assert_eq!(signed_dict.public_key, deserialized.public_key);
assert_eq!(signed_dict.signature, deserialized.signature);
assert_eq!(signed_dict.verify().is_ok(), deserialized.verify().is_ok());

2
src/lang/pretty_print.rs
View file

@ -131,7 +131,7 @@ impl CustomPredicateBatch {
impl PrettyPrint for Value {
fn fmt_podlang_with_indent(&self, w: &mut dyn Write, _indent: usize) -> std::fmt::Result {
write!(w, "{}", self.typed())
write!(w, "{}", self.typed)
}
}

657
src/middleware/containers.rs
View file

@ -1,29 +1,260 @@
//! This file implements the types defined at
//! <https://0xparc.github.io/pod2/values.html#dictionary-array-set> .
use std::collections::{HashMap, HashSet};
use std::{
collections::{HashMap, HashSet},
fmt::{self, Debug},
};
use schemars::JsonSchema;
use serde::{Deserialize, Deserializer, Serialize};
use serde::{
de::{Error as _, SeqAccess, Visitor},
ser, Deserialize, Deserializer, Serialize,
};
use super::serialization::{ordered_map, ordered_set};
#[cfg(feature = "backend_plonky2")]
use crate::backends::plonky2::primitives::merkletree::{MerkleProof, MerkleTree};
use crate::backends::plonky2::primitives::merkletree::{self, MerkleProof, MerkleTree};
use crate::{
backends::plonky2::primitives::merkletree::MerkleTreeStateTransitionProof,
middleware::{Error, Hash, Key, RawValue, Result, Value},
middleware::{
db::{mem::MemDB, DB},
Error, Hash, Key, RawValue, Result, TypedValue, Value, EMPTY_HASH,
},
};
#[derive(Clone, Debug)]
pub struct Container {
root: Hash,
db: Box<dyn DB>,
}
impl JsonSchema for Container {
fn schema_name() -> String {
"Container".to_string()
}
fn json_schema(gen: &mut schemars::gen::SchemaGenerator) -> schemars::schema::Schema {
// Just use the schema of Vec<Vec<Value>> since that's what we're actually serializing
Vec::<Vec<Value>>::json_schema(gen)
}
}
impl Serialize for Container {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: serde::Serializer,
{
let mut pairs = self
.iter()
.collect::<Result<Vec<(Value, Value)>>>()
.map_err(ser::Error::custom)?;
pairs.sort_by(|(k1, _), (k2, _)| k1.raw().cmp(&k2.raw()));
// Serialize as an array
use serde::ser::SerializeSeq;
let mut seq = serializer.serialize_seq(Some(pairs.len()))?;
for (k, v) in pairs {
if k == v {
seq.serialize_element(&[&v])?;
} else {
seq.serialize_element(&[&k, &v])?;
}
}
seq.end()
}
}
struct ContainerVisitor;
impl<'de> Visitor<'de> for ContainerVisitor {
type Value = HashMap<Value, Value>;
fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
formatter.write_str("a sequence of `[Value]` or `[Value, Value]`")
}
fn visit_seq<A>(self, mut seq: A) -> Result<Self::Value, A::Error>
where
A: SeqAccess<'de>,
{
let mut kvs = HashMap::<Value, Value>::new();
while let Some(mut elem) = seq.next_element::<Vec<Value>>()? {
match elem.len() {
1 => {
let v = elem.pop().unwrap();
kvs.insert(v.clone(), v);
}
2 => {
let (v, k) = (elem.pop().unwrap(), elem.pop().unwrap());
kvs.insert(k, v);
}
n => {
return Err(A::Error::custom(format!(
"invalid vec length of {n} in container entry"
)))
}
}
}
Ok(kvs)
}
}
impl<'de> Deserialize<'de> for Container {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: Deserializer<'de>,
{
let kvs = deserializer.deserialize_seq(ContainerVisitor)?;
Ok(Container::new(kvs))
}
}
impl PartialEq for Container {
fn eq(&self, other: &Self) -> bool {
self.root == other.root
}
}
impl Eq for Container {}
fn store_container_mt(db: &mut dyn DB, container: &Container) -> Result<()> {
match db.load_node(container.root) {
Err(e) => return Err(Error::Database(e)),
// Container already exists in the DB
Ok(Some(_)) => return Ok(()),
// Container not existing, we need to save it
Ok(None) => {}
};
let mut container_copy = Container::empty_with_db(db.clone_box());
for kv_result in container.iter() {
let (k, v) = kv_result?;
container_copy.insert(k, v)?;
}
Ok(())
}
fn store_value(db: &mut dyn DB, v: Value) -> Result<()> {
match &v.typed {
TypedValue::Set(Set { inner })
| TypedValue::Dictionary(Dictionary { inner })
| TypedValue::Array(Array { inner }) => {
if db.is_persistent() {
store_container_mt(db, inner)?;
}
db.store_value(v).map_err(Error::Database)?
}
_ => db.store_value(v).map_err(Error::Database)?,
}
Ok(())
}
fn load_value(db: &dyn DB, value_raw: RawValue) -> Result<Value> {
match db.load_value(value_raw) {
Err(e) => Err(Error::Database(e)),
Ok(Some(v)) => Ok(v),
Ok(None) => Err(Error::custom(format!(
"Value from {value_raw} not found in DB"
))),
}
}
impl Container {
fn mt(&self) -> MerkleTree {
MerkleTree::from_db(self.root, self.db.clone())
}
pub fn new(kvs: HashMap<Value, Value>) -> Self {
let db = Box::new(MemDB::new());
let mut container = Self::empty_with_db(db);
for (k, v) in kvs {
container.insert(k, v).expect("no duplicates, no db errors");
}
container
}
pub fn empty_with_db(db: Box<dyn DB>) -> Self {
Self::from_db(EMPTY_HASH, db).expect("EMPTY_HASH exists implicitly")
}
pub fn from_db(root: Hash, db: Box<dyn DB>) -> Result<Self> {
// Make sure the root exists in the db
let _ = merkletree::load_node(db.as_ref(), root)?;
Ok(Self { root, db })
}
pub fn commitment(&self) -> Hash {
self.root
}
pub fn get(&self, key_raw: RawValue) -> Result<Option<Value>> {
Ok(match self.mt().get(&key_raw)? {
Some(value_raw) => Some(load_value(self.db.as_ref(), value_raw)?),
None => None,
})
}
pub fn prove(&self, key_raw: RawValue) -> Result<(Value, MerkleProof)> {
let (value_raw, mtp) = self.mt().prove(&key_raw)?;
let value = load_value(self.db.as_ref(), value_raw)?;
Ok((value, mtp))
}
pub fn prove_nonexistence(&self, key_raw: RawValue) -> Result<MerkleProof> {
Ok(self.mt().prove_nonexistence(&key_raw)?)
}
pub fn insert(&mut self, key: Value, value: Value) -> Result<MerkleTreeStateTransitionProof> {
let (key_raw, value_raw) = (key.raw(), value.raw());
store_value(self.db.as_mut(), key)?;
store_value(self.db.as_mut(), value)?;
let mut mt = self.mt();
let mtp = mt.insert(&key_raw, &value_raw)?;
self.root = mt.root();
Ok(mtp)
}
pub fn update(
&mut self,
key_raw: RawValue,
value: Value,
) -> Result<MerkleTreeStateTransitionProof> {
let value_raw = value.raw();
store_value(self.db.as_mut(), value)?;
let mut mt = self.mt();
let mtp = mt.update(&key_raw, &value_raw)?;
self.root = mt.root();
Ok(mtp)
}
pub fn delete(&mut self, key_raw: RawValue) -> Result<MerkleTreeStateTransitionProof> {
let mut mt = self.mt();
let mtp = mt.delete(&key_raw)?;
self.root = mt.root();
Ok(mtp)
}
pub fn verify(
root: Hash,
proof: &MerkleProof,
key_raw: RawValue,
value_raw: RawValue,
) -> Result<()> {
Ok(MerkleTree::verify(root, proof, &key_raw, &value_raw)?)
}
pub fn verify_nonexistence(root: Hash, proof: &MerkleProof, key_raw: RawValue) -> Result<()> {
Ok(MerkleTree::verify_nonexistence(root, proof, &key_raw)?)
}
pub fn verify_state_transition(proof: &MerkleTreeStateTransitionProof) -> Result<()> {
MerkleTree::verify_state_transition(proof).map_err(|e| e.into())
}
pub fn iter(&self) -> impl Iterator<Item = Result<(Value, Value)>> {
let db = self.db.clone();
self.mt().iter().map(move |(key_raw, value_raw)| {
let key = load_value(db.as_ref(), key_raw)?;
let value = load_value(db.as_ref(), value_raw)?;
Ok((key, value))
})
}
/// This is an expensive operation
pub fn dump(&self) -> Result<HashMap<Value, Value>> {
self.iter().collect()
}
}
/// Dictionary: the user original keys and values are hashed to be used in the leaf.
/// leaf.key=hash(original_key)
/// leaf.value=hash(original_value)
#[derive(Clone, Debug, Serialize, JsonSchema)]
#[derive(Clone, Debug, Serialize, Deserialize, JsonSchema)]
pub struct Dictionary {
#[serde(skip)]
#[schemars(skip)]
mt: MerkleTree,
#[serde(serialize_with = "ordered_map")]
kvs: HashMap<Key, Value>,
pub(crate) inner: Container,
}
#[macro_export]
@ -38,251 +269,367 @@ macro_rules! dict {
});
}
// TODO: Replace all methods that receive a `&Key` by either `impl Into<String>` for write
// methods and `impl AsRef<str>` for read methods.
// TODO: Replace all methods that receive a `&Value` in write methods for `Value`. Consider a
// trait?
impl Dictionary {
pub fn new(kvs: HashMap<Key, Value>) -> Self {
let kvs_raw: HashMap<RawValue, RawValue> =
kvs.iter().map(|(k, v)| (k.raw(), v.raw())).collect();
Self {
mt: MerkleTree::new(&kvs_raw),
kvs,
inner: Container::new(
kvs.into_iter()
.map(|(k, v)| (Value::from(k.name), v))
.collect(),
),
}
}
pub fn empty_with_db(db: Box<dyn DB>) -> Self {
Self {
inner: Container::empty_with_db(db),
}
}
pub fn from_db(root: Hash, db: Box<dyn DB>) -> Result<Self> {
Ok(Self {
inner: Container::from_db(root, db)?,
})
}
pub fn commitment(&self) -> Hash {
self.mt.root()
self.inner.commitment()
}
pub fn get(&self, key: &Key) -> Result<&Value> {
self.kvs
.get(key)
.ok_or_else(|| Error::custom(format!("key \"{}\" not found", key.name())))
pub fn get(&self, key: &Key) -> Result<Option<Value>> {
self.inner.get(key.raw())
}
pub fn prove(&self, key: &Key) -> Result<(&Value, MerkleProof)> {
let (_, mtp) = self.mt.prove(&key.raw())?;
let value = self.kvs.get(key).expect("key exists");
Ok((value, mtp))
pub fn prove(&self, key: &Key) -> Result<(Value, MerkleProof)> {
self.inner.prove(key.raw())
}
pub fn prove_nonexistence(&self, key: &Key) -> Result<MerkleProof> {
Ok(self.mt.prove_nonexistence(&key.raw())?)
self.inner.prove_nonexistence(key.raw())
}
pub fn insert(&mut self, key: &Key, value: &Value) -> Result<MerkleTreeStateTransitionProof> {
let mtp = self.mt.insert(&key.raw(), &value.raw())?;
self.kvs.insert(key.clone(), value.clone());
Ok(mtp)
self.inner
.insert(Value::from(key.name.clone()), value.clone())
}
pub fn update(&mut self, key: &Key, value: &Value) -> Result<MerkleTreeStateTransitionProof> {
let mtp = self.mt.update(&key.raw(), &value.raw())?;
self.kvs.insert(key.clone(), value.clone());
Ok(mtp)
self.inner.update(key.raw(), value.clone())
}
pub fn delete(&mut self, key: &Key) -> Result<MerkleTreeStateTransitionProof> {
let mtp = self.mt.delete(&key.raw())?;
self.kvs.remove(key);
Ok(mtp)
self.inner.delete(key.raw())
}
pub fn verify(root: Hash, proof: &MerkleProof, key: &Key, value: &Value) -> Result<()> {
let key = key.raw();
Ok(MerkleTree::verify(root, proof, &key, &value.raw())?)
Container::verify(root, proof, key.raw(), value.raw())
}
pub fn verify_nonexistence(root: Hash, proof: &MerkleProof, key: &Key) -> Result<()> {
let key = key.raw();
Ok(MerkleTree::verify_nonexistence(root, proof, &key)?)
Container::verify_nonexistence(root, proof, key.raw())
}
pub fn verify_state_transition(proof: &MerkleTreeStateTransitionProof) -> Result<()> {
MerkleTree::verify_state_transition(proof).map_err(|e| e.into())
Container::verify_state_transition(proof)
}
// TODO: Rename to dict to be consistent maybe?
pub fn kvs(&self) -> &HashMap<Key, Value> {
&self.kvs
pub fn iter(&self) -> impl Iterator<Item = Result<(String, Value)>> + use<'_> {
self.inner.iter().map(|r| match r {
Ok((key, value)) => Ok((
key.as_string()
.ok_or_else(|| Error::custom("dictionary: key is not string"))?,
value,
)),
Err(e) => Err(e),
})
}
/// This is an expensive operation
pub fn dump(&self) -> Result<HashMap<String, Value>> {
self.iter().collect()
}
}
impl PartialEq for Dictionary {
fn eq(&self, other: &Self) -> bool {
self.mt.root() == other.mt.root()
self.inner.eq(&other.inner)
}
}
impl Eq for Dictionary {}
impl<'de> Deserialize<'de> for Dictionary {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: Deserializer<'de>,
{
#[derive(Deserialize)]
struct Aux {
#[serde(serialize_with = "ordered_map")]
kvs: HashMap<Key, Value>,
}
let aux = Aux::deserialize(deserializer)?;
Ok(Dictionary::new(aux.kvs))
}
}
/// Set: the value field of the leaf is unused, and the key contains the hash of the element.
/// leaf.key=hash(original_value)
/// leaf.value=0
#[derive(Clone, Debug, Serialize, JsonSchema)]
#[derive(Clone, Debug, Serialize, Deserialize, JsonSchema)]
pub struct Set {
#[serde(skip)]
#[schemars(skip)]
mt: MerkleTree,
#[serde(serialize_with = "ordered_set")]
set: HashSet<Value>,
pub(crate) inner: Container,
}
impl Set {
pub fn new(set: HashSet<Value>) -> Self {
let kvs_raw: HashMap<RawValue, RawValue> = set
.iter()
.map(|e| {
let rv = e.raw();
(rv, rv)
})
.collect();
Self {
mt: MerkleTree::new(&kvs_raw),
set,
inner: Container::new(set.into_iter().map(|v| (v.clone(), v)).collect()),
}
}
pub fn empty_with_db(db: Box<dyn DB>) -> Self {
Self {
inner: Container::empty_with_db(db),
}
}
pub fn from_db(root: Hash, db: Box<dyn DB>) -> Result<Self> {
Ok(Self {
inner: Container::from_db(root, db)?,
})
}
pub fn commitment(&self) -> Hash {
self.mt.root()
self.inner.commitment()
}
pub fn contains(&self, value: &Value) -> bool {
self.set.contains(value)
pub fn contains(&self, value: &Value) -> Result<bool> {
Ok(self.inner.get(value.raw())?.is_some())
}
pub fn prove(&self, value: &Value) -> Result<MerkleProof> {
let rv = value.raw();
let (_, proof) = self.mt.prove(&rv)?;
let (_, proof) = self.inner.prove(value.raw())?;
Ok(proof)
}
pub fn prove_nonexistence(&self, value: &Value) -> Result<MerkleProof> {
let rv = value.raw();
Ok(self.mt.prove_nonexistence(&rv)?)
self.inner.prove_nonexistence(value.raw())
}
pub fn insert(&mut self, value: &Value) -> Result<MerkleTreeStateTransitionProof> {
let raw_value = value.raw();
let mtp = self.mt.insert(&raw_value, &raw_value)?;
self.set.insert(value.clone());
Ok(mtp)
self.inner.insert(value.clone(), value.clone())
}
pub fn delete(&mut self, value: &Value) -> Result<MerkleTreeStateTransitionProof> {
let mtp = self.mt.delete(&value.raw())?;
self.set.remove(value);
Ok(mtp)
self.inner.delete(value.raw())
}
pub fn verify(root: Hash, proof: &MerkleProof, value: &Value) -> Result<()> {
let rv = value.raw();
Ok(MerkleTree::verify(root, proof, &rv, &rv)?)
Container::verify(root, proof, value.raw(), value.raw())
}
pub fn verify_nonexistence(root: Hash, proof: &MerkleProof, value: &Value) -> Result<()> {
let rv = value.raw();
Ok(MerkleTree::verify_nonexistence(root, proof, &rv)?)
Container::verify_nonexistence(root, proof, value.raw())
}
pub fn verify_state_transition(proof: &MerkleTreeStateTransitionProof) -> Result<()> {
MerkleTree::verify_state_transition(proof).map_err(|e| e.into())
Container::verify_state_transition(proof)
}
pub fn set(&self) -> &HashSet<Value> {
&self.set
pub fn iter(&self) -> impl Iterator<Item = Result<Value>> + use<'_> {
self.inner.iter().map(|r| match r {
Ok((key, value)) => {
if key != value {
return Err(Error::custom("set: key != value"));
}
Ok(value)
}
Err(e) => Err(e),
})
}
/// This is an expensive operation
pub fn dump(&self) -> Result<HashSet<Value>> {
self.iter().collect()
}
}
impl PartialEq for Set {
fn eq(&self, other: &Self) -> bool {
self.mt.root() == other.mt.root()
self.inner.eq(&other.inner)
}
}
impl Eq for Set {}
impl<'de> Deserialize<'de> for Set {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: Deserializer<'de>,
{
#[derive(Deserialize, JsonSchema)]
struct Aux {
#[serde(serialize_with = "ordered_set")]
set: HashSet<Value>,
}
let aux = Aux::deserialize(deserializer)?;
Ok(Set::new(aux.set))
}
}
/// Array: the elements are placed at the value field of each leaf, and the key field is just the
/// array index (integer).
/// leaf.key=i
/// leaf.value=original_value
#[derive(Clone, Debug, Serialize, JsonSchema)]
/// Due to its construction this should be seen as a sparse array, where there can be gaps
/// (unused indices).
#[derive(Clone, Debug, Serialize, Deserialize, JsonSchema)]
pub struct Array {
#[serde(skip)]
#[schemars(skip)]
mt: MerkleTree,
array: Vec<Value>,
pub(crate) inner: Container,
}
impl Array {
pub fn new(array: Vec<Value>) -> Self {
let kvs_raw: HashMap<RawValue, RawValue> = array
.iter()
.enumerate()
.map(|(i, e)| (RawValue::from(i as i64), e.raw()))
.collect();
Self {
mt: MerkleTree::new(&kvs_raw),
array,
inner: Container::new(
array
.into_iter()
.enumerate()
.map(|(i, v)| (Value::from(i as i64), v))
.collect(),
),
}
}
pub fn commitment(&self) -> Hash {
self.mt.root()
pub fn empty_with_db(db: Box<dyn DB>) -> Self {
Self {
inner: Container::empty_with_db(db),
}
pub fn get(&self, i: usize) -> Result<&Value> {
self.array.get(i).ok_or_else(|| {
Error::custom(format!("index {} out of bounds 0..{}", i, self.array.len()))
}
pub fn from_db(root: Hash, db: Box<dyn DB>) -> Result<Self> {
Ok(Self {
inner: Container::from_db(root, db)?,
})
}
pub fn prove(&self, i: usize) -> Result<(&Value, MerkleProof)> {
let (_, mtp) = self.mt.prove(&RawValue::from(i as i64))?;
let value = self.array.get(i).expect("valid index");
Ok((value, mtp))
pub fn commitment(&self) -> Hash {
self.inner.commitment()
}
pub fn get(&self, i: usize) -> Result<Option<Value>> {
self.inner.get(Value::from(i as i64).raw())
}
pub fn prove(&self, i: usize) -> Result<(Value, MerkleProof)> {
self.inner.prove(Value::from(i as i64).raw())
}
pub fn insert(&mut self, i: usize, value: Value) -> Result<MerkleTreeStateTransitionProof> {
self.inner.insert(Value::from(i as i64), value)
}
pub fn delete(&mut self, i: usize) -> Result<MerkleTreeStateTransitionProof> {
self.inner.delete(Value::from(i as i64).raw())
}
pub fn update(&mut self, i: usize, value: &Value) -> Result<MerkleTreeStateTransitionProof> {
let mtp = self.mt.update(&(i as i64).into(), &value.raw())?;
self.array[i] = value.clone();
Ok(mtp)
self.inner
.update(Value::from(i as i64).raw(), value.clone())
}
pub fn verify(root: Hash, proof: &MerkleProof, i: usize, value: &Value) -> Result<()> {
Ok(MerkleTree::verify(
root,
proof,
&RawValue::from(i as i64),
&value.raw(),
)?)
Container::verify(root, proof, Value::from(i as i64).raw(), value.raw())
}
pub fn verify_state_transition(proof: &MerkleTreeStateTransitionProof) -> Result<()> {
MerkleTree::verify_state_transition(proof).map_err(|e| e.into())
Container::verify_state_transition(proof)
}
pub fn array(&self) -> &[Value] {
&self.array
pub fn iter(&self) -> impl Iterator<Item = Result<(usize, Value)>> + use<'_> {
self.inner.iter().map(|r| match r {
Ok((key, value)) => {
let index = key
.as_int()
.ok_or_else(|| Error::custom("array: key is not int"))?;
Ok((index as usize, value))
}
Err(e) => Err(e),
})
}
/// This is an expensive operation
pub fn dump(&self) -> Result<HashMap<usize, Value>> {
self.iter().collect()
}
}
impl PartialEq for Array {
fn eq(&self, other: &Self) -> bool {
self.mt.root() == other.mt.root()
self.inner.eq(&other.inner)
}
}
impl Eq for Array {}
impl<'de> Deserialize<'de> for Array {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: Deserializer<'de>,
#[cfg(test)]
mod tests {
use super::*;
use crate::middleware::db::mem::MemDB;
fn test_databases(test_fn: &dyn Fn(Box<dyn DB>)) {
let db = MemDB::new();
test_fn(Box::new(db));
#[cfg(feature = "db_rocksdb")]
{
#[derive(Deserialize, JsonSchema)]
struct Aux {
array: Vec<Value>,
}
let aux = Aux::deserialize(deserializer)?;
Ok(Array::new(aux.array))
use crate::middleware::db;
let db = db::rocks::RocksDB::open(tempfile::TempDir::new().unwrap().path()).unwrap();
test_fn(Box::new(db));
}
}
fn _test_dict(db: Box<dyn DB>) {
let mut dict0 = Dictionary::empty_with_db(db.clone());
dict0.insert(&Key::from("a"), &Value::from(1)).unwrap();
dict0.insert(&Key::from("b"), &Value::from(2)).unwrap();
dict0.update(&Key::from("a"), &Value::from(3)).unwrap();
dict0.insert(&Key::from("c"), &Value::from(4)).unwrap();
dict0.delete(&Key::from("c")).unwrap();
let kvs0 = dict0.dump().unwrap();
assert_eq!(
kvs0,
[
("a".to_string(), Value::from(3)),
("b".to_string(), Value::from(2))
]
.into_iter()
.collect()
);
let dict1 = Dictionary::from_db(dict0.commitment(), db).unwrap();
let kvs1 = dict1.dump().unwrap();
assert_eq!(kvs0, kvs1);
}
fn _test_set(db: Box<dyn DB>) {
let mut set0 = Set::empty_with_db(db.clone());
set0.insert(&Value::from(1)).unwrap();
set0.insert(&Value::from(2)).unwrap();
set0.insert(&Value::from(3)).unwrap();
set0.delete(&Value::from(2)).unwrap();
let s0 = set0.dump().unwrap();
assert_eq!(s0, [Value::from(1), Value::from(3)].into_iter().collect());
let set1 = Set::from_db(set0.commitment(), db).unwrap();
let s1 = set1.dump().unwrap();
assert_eq!(s0, s1);
}
fn _test_array(db: Box<dyn DB>) {
let mut arr0 = Array::empty_with_db(db.clone());
arr0.insert(0, Value::from("a")).unwrap();
arr0.insert(1, Value::from("b")).unwrap();
arr0.insert(2, Value::from("c")).unwrap();
arr0.delete(1).unwrap();
let a0 = arr0.dump().unwrap();
assert_eq!(
a0,
[(0, Value::from("a")), (2, Value::from("c"))]
.into_iter()
.collect()
);
let arr1 = Array::from_db(arr0.commitment(), db).unwrap();
let a1 = arr1.dump().unwrap();
assert_eq!(a0, a1);
}
fn _test_nested(db: Box<dyn DB>) {
let mut nested = Dictionary::empty_with_db(db.clone());
nested.insert(&Key::from("a"), &Value::from(1)).unwrap();
nested.insert(&Key::from("b"), &Value::from(2)).unwrap();
let nested_kvs0 = nested.dump().unwrap();
let mut dict0 = Dictionary::empty_with_db(db.clone());
dict0.insert(&Key::from("x"), &Value::from(1)).unwrap();
dict0
.insert(&Key::from("y"), &Value::from(nested.clone()))
.unwrap();
let kvs0 = dict0.dump().unwrap();
assert_eq!(
kvs0,
[
("x".to_string(), Value::from(1)),
("y".to_string(), Value::from(nested))
]
.into_iter()
.collect()
);
let dict1 = Dictionary::from_db(dict0.commitment(), db).unwrap();
let kvs1 = dict1.dump().unwrap();
assert_eq!(kvs0, kvs1);
match &kvs1["y"].typed {
TypedValue::Dictionary(d) => {
let nested_kvs1 = d.dump().unwrap();
assert_eq!(nested_kvs0, nested_kvs1);
}
_ => unreachable!(),
}
}
#[test]
fn test_dict() {
test_databases(&_test_dict);
}
#[test]
fn test_set() {
test_databases(&_test_set);
}
#[test]
fn test_array() {
test_databases(&_test_array);
}
#[test]
fn test_nested() {
test_databases(&_test_nested);
}
}

60
src/middleware/db/mem.rs Normal file
View file

@ -0,0 +1,60 @@
use super::*;
/// MemDB implements the DB trait in a in-memory HashMap.
#[derive(Clone, Debug, Default)]
pub struct MemDB {
nodes: Arc<RwLock<HashMap<Hash, merkletree::Node>>>,
values: Arc<RwLock<HashMap<RawValue, Value>>>,
}
impl MemDB {
pub fn new() -> Self {
Self::default()
}
}
impl merkletree::db::DB for MemDB {
fn load_node(&self, hash: Hash) -> anyhow::Result<Option<merkletree::Node>> {
let nodes = self.nodes.read().expect("lock not poisoned");
if hash == EMPTY_HASH {
return Ok(Some(merkletree::Node::Intermediate(
merkletree::Intermediate::new(EMPTY_HASH, EMPTY_HASH),
)));
}
Ok(nodes.get(&hash).cloned())
}
fn store_node(&mut self, node: merkletree::Node) -> anyhow::Result<()> {
let mut nodes = self.nodes.write().expect("lock not poisoned");
nodes.insert(node.hash(), node);
Ok(())
}
}
impl DB for MemDB {
fn load_value(&self, raw: RawValue) -> anyhow::Result<Option<Value>> {
let values = self.values.read().expect("lock not poisoned");
Ok(values.get(&raw).cloned())
}
fn store_value(&mut self, value: Value) -> anyhow::Result<()> {
let mut values = self.values.write().expect("lock not poisoned");
let value_raw = value.raw();
if let Some(old_value) = values.get(&value_raw) {
// If we had a non-raw value stored never overwrite it with a raw value
if !old_value.is_raw() && value.is_raw() {
return Ok(());
}
}
values.insert(value_raw, value);
Ok(())
}
fn is_persistent(&self) -> bool {
false
}
fn clone_box(&self) -> Box<dyn DB> {
Box::new(self.clone())
}
}

30
src/middleware/db/mod.rs Normal file
View file

@ -0,0 +1,30 @@
use std::{
collections::HashMap,
fmt::Debug,
sync::{Arc, RwLock},
};
use dyn_clone::DynClone;
#[cfg(feature = "backend_plonky2")]
use crate::backends::plonky2::primitives::merkletree::{self};
use crate::middleware::{Hash, RawValue, Value, EMPTY_HASH};
pub mod mem;
#[cfg(feature = "db_rocksdb")]
pub mod rocks;
// Trait for database that stores values. Must be cheap to clone.
pub trait DB: Debug + DynClone + Sync + Send + merkletree::db::DB {
fn load_value(&self, raw: RawValue) -> anyhow::Result<Option<Value>>;
// If the DB is persistent, for containers only the root needs to be stored because the
// Container type makes sure the underlying merkle tree is stored in the DB independently, so
// that it can be recovered back just with the root and the DB.
// If the value is RawValue and a previous non-RawValue exists, no store overwrite it.
// should be done. If the value is non-RawValue and a previous RawValue exists, store
// should overwrite it.
fn store_value(&mut self, value: Value) -> anyhow::Result<()>;
fn is_persistent(&self) -> bool;
fn clone_box(&self) -> Box<dyn DB>;
}
dyn_clone::clone_trait_object!(DB);

107
src/middleware/db/rocks.rs Normal file
View file

@ -0,0 +1,107 @@
use std::{fmt, path::Path, sync::Arc};
use anyhow::{anyhow, Result};
use rocksdb::{Options, TransactionDB, TransactionDBOptions};
use super::*;
fn node_key(hash: Hash) -> Vec<u8> {
let mut k = Vec::with_capacity(2 + 4);
k.extend_from_slice(b"n/");
k.extend_from_slice(&RawValue::from(hash).to_bytes());
k
}
fn value_key(raw: RawValue) -> Vec<u8> {
let mut k = Vec::with_capacity(2 + 4);
k.extend_from_slice(b"v/");
k.extend_from_slice(&raw.to_bytes());
k
}
#[derive(Clone)]
pub struct RocksDB {
db: Arc<TransactionDB>,
}
impl fmt::Debug for RocksDB {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
writeln!(f, "RocksDB(path: {:?})", self.db.path())
}
}
impl RocksDB {
pub fn open(path: impl AsRef<Path>) -> Result<Self> {
let mut options = Options::default();
options.create_if_missing(true);
let txn_options = TransactionDBOptions::default();
let inner =
TransactionDB::open(&options, &txn_options, path).map_err(|e| anyhow!("{e}"))?;
Ok(Self {
db: Arc::new(inner),
})
}
}
impl merkletree::db::DB for RocksDB {
fn load_node(&self, hash: Hash) -> Result<Option<merkletree::Node>> {
if hash == EMPTY_HASH {
return Ok(Some(merkletree::Node::Intermediate(
merkletree::Intermediate::new(EMPTY_HASH, EMPTY_HASH),
)));
}
match self.db.get(node_key(hash))? {
None => Ok(None),
Some(bytes) => Ok(Some(merkletree::Node::decode(bytes.as_ref())?)),
}
}
fn store_node(&mut self, node: merkletree::Node) -> Result<()> {
self.db
.put(node_key(node.hash()), node.encode()?)
.map_err(|e| anyhow!("rocksdb transaction put failed: {e}"))
}
}
impl DB for RocksDB {
fn load_value(&self, raw: RawValue) -> anyhow::Result<Option<Value>> {
match self.db.get(value_key(raw))? {
None => Ok(None),
Some(bytes) => Ok(Some({
if bytes.is_empty() {
Value::from(raw)
} else {
Value::from_bytes(bytes.as_ref(), self.clone_box())?
}
})),
}
}
fn store_value(&mut self, value: Value) -> anyhow::Result<()> {
let value_key = value_key(value.raw());
let tx = self.db.transaction();
if let Some(old_value_bytes) = tx.get_for_update(&value_key, true)? {
let is_raw = old_value_bytes.is_empty();
// If we had a non-RawValue stored don't overwrite it (specially not with a
// RawValue). Also skip redundant RawValue overwrite.
if !is_raw || (is_raw && value.is_raw()) {
return Ok(());
}
}
let value_bytes = if value.is_raw() {
// For RawValue we store an empty vector because it's a duplicate of the key.
// This way we can easily check for RawValue without decoding.
vec![]
} else {
Value::to_bytes(&value)
};
tx.put(value_key, value_bytes)?;
Ok(tx.commit()?)
}
fn is_persistent(&self) -> bool {
true
}
fn clone_box(&self) -> Box<dyn DB> {
Box::new(self.clone())
}
}

8
src/middleware/error.rs
View file

@ -72,6 +72,10 @@ pub enum Error {
},
#[error(transparent)]
Tree(#[from] crate::backends::plonky2::primitives::merkletree::error::TreeError),
#[error(transparent)]
Json(#[from] serde_json::Error),
#[error("database error: {0}")]
Database(anyhow::Error),
}
impl Debug for Error {
@ -164,7 +168,7 @@ impl Error {
pub(crate) fn unsatisfied_custom_predicate_disjunction(pred: CustomPredicate) -> Self {
new!(UnsatisfiedCustomPredicateDisjunction(pred))
}
pub(crate) fn custom(s: String) -> Self {
new!(Custom(s))
pub(crate) fn custom(s: impl Into<String>) -> Self {
new!(Custom(s.into()))
}
}

359
src/middleware/mod.rs
View file

@ -1,16 +1,13 @@
//! The middleware includes the type definitions and the traits used to connect the frontend and
//! the backend.
use std::sync::Arc;
use hex::ToHex;
use itertools::Itertools;
use strum_macros::FromRepr;
mod basetypes;
use std::{cmp::PartialEq, hash};
use containers::{Array, Dictionary, Set};
use containers::{Array, Container, Dictionary, Set};
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
pub mod containers;
@ -22,6 +19,7 @@ pub mod serialization;
mod statement;
use std::{any::Any, fmt};
pub mod db;
pub use basetypes::*;
pub use custom::*;
use dyn_clone::DynClone;
@ -31,14 +29,10 @@ pub use pod_deserialization::*;
use serialization::*;
pub use statement::*;
use crate::backends::plonky2::primitives::merkletree::{
MerkleProof, MerkleTreeStateTransitionProof,
};
// TODO: Move all value-related types to to `value.rs`
#[derive(Clone, Debug, Serialize, Deserialize, Eq, PartialEq)]
// TODO #[schemars(transform = serialization::transform_value_schema)]
pub enum TypedValue {
pub(crate) enum TypedValue {
// Serde cares about the order of the enum variants, with untagged variants
// appearing at the end.
// Variants without "untagged" will be serialized as "tagged" values by
@ -73,8 +67,6 @@ pub enum TypedValue {
Array(Array),
#[serde(untagged)]
String(String),
#[serde(untagged)]
Bool(bool),
}
impl From<&str> for TypedValue {
@ -97,7 +89,11 @@ impl From<i64> for TypedValue {
impl From<bool> for TypedValue {
fn from(b: bool) -> Self {
TypedValue::Bool(b)
if b {
TypedValue::Int(1)
} else {
TypedValue::Int(0)
}
}
}
@ -149,70 +145,6 @@ impl From<RawValue> for TypedValue {
}
}
impl TryFrom<&TypedValue> for i64 {
type Error = Error;
fn try_from(v: &TypedValue) -> std::result::Result<Self, Self::Error> {
if let TypedValue::Int(n) = v {
Ok(*n)
} else {
Err(Error::custom("Value not an int".to_string()))
}
}
}
impl TryFrom<&TypedValue> for String {
type Error = Error;
fn try_from(tv: &TypedValue) -> Result<Self> {
match tv {
TypedValue::String(s) => Ok(s.clone()),
_ => Err(Error::custom(format!(
"Value {} cannot be converted to a string.",
tv
))),
}
}
}
impl TryFrom<&TypedValue> for Key {
type Error = Error;
fn try_from(tv: &TypedValue) -> Result<Self> {
Ok(Key::new(String::try_from(tv)?))
}
}
impl TryFrom<&TypedValue> for PublicKey {
type Error = Error;
fn try_from(v: &TypedValue) -> std::result::Result<Self, Self::Error> {
if let TypedValue::PublicKey(pk) = v {
Ok(*pk)
} else {
Err(Error::custom("Value not a public key".to_string()))
}
}
}
impl TryFrom<&TypedValue> for SecretKey {
type Error = Error;
fn try_from(v: &TypedValue) -> std::result::Result<Self, Self::Error> {
if let TypedValue::SecretKey(sk) = v {
Ok(sk.clone())
} else {
Err(Error::custom("Value not a secret key".to_string()))
}
}
}
impl TryFrom<&TypedValue> for Predicate {
type Error = Error;
fn try_from(v: &TypedValue) -> std::result::Result<Self, Self::Error> {
if let TypedValue::Predicate(p) = v {
Ok(p.clone())
} else {
Err(Error::custom("Value not a Predicate".to_string()))
}
}
}
impl fmt::Display for TypedValue {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
@ -224,36 +156,54 @@ impl fmt::Display for TypedValue {
Err(_) => write!(f, "\"{}\"", s),
}
}
TypedValue::Bool(b) => write!(f, "{}", b),
TypedValue::Array(a) => {
write!(f, "[")?;
for (i, v) in a.array().iter().enumerate() {
for (i, r) in a.iter().enumerate() {
if i > 0 {
write!(f, ", ")?;
}
write!(f, "{}", v)?;
if i == 8 {
write!(f, "…")?;
break;
}
match r {
Ok((index, value)) => write!(f, "{}: {}", index, value)?,
Err(e) => write!(f, "{e}")?,
}
}
write!(f, "]")
}
TypedValue::Dictionary(d) => {
write!(f, "{{ ")?;
let kvs: Vec<_> = d.kvs().iter().sorted_by_key(|(k, _)| k.name()).collect();
for (i, (k, v)) in kvs.iter().enumerate() {
for (i, r) in d.iter().enumerate() {
if i > 0 {
write!(f, ", ")?;
}
write!(f, "{}: {}", k, v)?;
if i == 8 {
write!(f, "…")?;
break;
}
match r {
Ok((key, value)) => write!(f, "{}: {}", key, value)?,
Err(e) => write!(f, "{e}")?,
}
}
write!(f, " }}")
}
TypedValue::Set(s) => {
write!(f, "#[")?;
let values: Vec<_> = s.set().iter().sorted_by_key(|k| k.raw()).collect();
for (i, v) in values.iter().enumerate() {
for (i, r) in s.iter().enumerate() {
if i > 0 {
write!(f, ", ")?;
}
write!(f, "{}", v)?;
if i == 8 {
write!(f, "…")?;
break;
}
match r {
Ok(value) => write!(f, "{}", value)?,
Err(e) => write!(f, "{e}")?,
}
}
write!(f, "]")
}
@ -272,7 +222,6 @@ impl From<&TypedValue> for RawValue {
match v {
TypedValue::String(s) => RawValue::from(hash_str(s)),
TypedValue::Int(v) => RawValue::from(*v),
TypedValue::Bool(b) => RawValue::from(*b as i64),
TypedValue::Dictionary(d) => RawValue::from(d.commitment()),
TypedValue::Set(s) => RawValue::from(s.commitment()),
TypedValue::Array(a) => RawValue::from(a.commitment()),
@ -405,9 +354,8 @@ impl JsonSchema for TypedValue {
#[derive(Clone, Debug)]
pub struct Value {
// The `TypedValue` is under `Arc` so that cloning a `Value` is cheap.
typed: Arc<TypedValue>,
raw: RawValue,
pub(crate) typed: TypedValue,
pub(crate) raw: RawValue,
}
// Values are serialized as their TypedValue.
@ -441,6 +389,55 @@ impl JsonSchema for Value {
}
}
/// Dual of TypedValue that is not recursive: for container types no entry only the commitment
/// (merkle tree root of underlying data) is available. Used for byte serialization for
/// persistent storage.
#[derive(Serialize, Deserialize)]
enum TypedValueNoRec {
Raw(RawValue),
Int(i64),
PublicKey(PublicKey),
SecretKey(SecretKey),
Predicate(Predicate),
Set(Hash),
Dictionary(Hash),
Array(Hash),
String(String),
}
// NOTE: byte serialization is using json. Using a byte-native serialization would improve
// performance and storage usage.
impl Value {
pub fn to_bytes(&self) -> Vec<u8> {
let v = match &self.typed {
TypedValue::Int(v) => TypedValueNoRec::Int(*v),
TypedValue::Raw(v) => TypedValueNoRec::Raw(*v),
TypedValue::PublicKey(v) => TypedValueNoRec::PublicKey(*v),
TypedValue::SecretKey(v) => TypedValueNoRec::SecretKey(v.clone()),
TypedValue::Predicate(v) => TypedValueNoRec::Predicate(v.clone()),
TypedValue::Set(v) => TypedValueNoRec::Set(v.commitment()),
TypedValue::Dictionary(v) => TypedValueNoRec::Dictionary(v.commitment()),
TypedValue::Array(v) => TypedValueNoRec::Array(v.commitment()),
TypedValue::String(v) => TypedValueNoRec::String(v.clone()),
};
serde_json::to_vec(&v).expect("json serialization succeeds")
}
pub fn from_bytes(bytes: &[u8], db: Box<dyn db::DB>) -> Result<Self> {
let v: TypedValueNoRec = serde_json::from_slice(bytes)?;
Ok(match v {
TypedValueNoRec::Int(v) => Value::from(v),
TypedValueNoRec::Raw(v) => Value::from(v),
TypedValueNoRec::PublicKey(v) => Value::from(v),
TypedValueNoRec::SecretKey(v) => Value::from(v),
TypedValueNoRec::Predicate(v) => Value::from(v),
TypedValueNoRec::Set(v) => Value::from(Set::from_db(v, db)?),
TypedValueNoRec::Dictionary(v) => Value::from(Dictionary::from_db(v, db)?),
TypedValueNoRec::Array(v) => Value::from(Array::from_db(v, db)?),
TypedValueNoRec::String(v) => Value::from(v),
})
}
}
impl PartialEq for Value {
fn eq(&self, other: &Self) -> bool {
self.raw == other.raw
@ -462,106 +459,110 @@ impl fmt::Display for Value {
}
impl Value {
pub fn new(value: TypedValue) -> Self {
pub(crate) fn new(value: TypedValue) -> Self {
let raw_value = RawValue::from(&value);
Self {
typed: Arc::new(value),
typed: value,
raw: raw_value,
}
}
pub fn typed(&self) -> &TypedValue {
&self.typed
}
pub fn raw(&self) -> RawValue {
self.raw
}
/// Determines Merkle existence proof for `key` in `self` (if applicable).
pub(crate) fn prove_existence<'a>(
&'a self,
key: &'a Value,
) -> Result<(&'a Value, MerkleProof)> {
match &self.typed() {
TypedValue::Array(a) => match key.typed() {
TypedValue::Int(i) if i >= &0 => a.prove((*i) as usize),
_ => Err(Error::custom(format!(
"Invalid key {} for container {}.",
key, self
)))?,
/// Returns true if the typed value is RawValue, which means it's a generic value with no type
/// information and no extra value data.
pub fn is_raw(&self) -> bool {
matches!(self.typed, TypedValue::Raw(_))
}
pub fn as_raw(&self) -> RawValue {
self.raw
}
pub fn as_int(&self) -> Option<i64> {
match self.typed {
TypedValue::Int(i) => Some(i),
_ => None,
}
}
pub fn as_public_key(&self) -> Option<PublicKey> {
match &self.typed {
TypedValue::PublicKey(pk) => Some(*pk),
_ => None,
}
}
pub fn as_secret_key(&self) -> Option<SecretKey> {
match &self.typed {
TypedValue::SecretKey(sk) => Some(sk.clone()),
_ => None,
}
}
pub fn as_predicate(&self) -> Option<Predicate> {
match &self.typed {
TypedValue::Predicate(p) => Some(p.clone()),
_ => None,
}
}
pub fn as_set(&self) -> Option<Set> {
match &self.typed {
TypedValue::Set(s) => Some(s.clone()),
TypedValue::Dictionary(d) => Some(Set {
inner: d.inner.clone(),
}),
TypedValue::Array(a) => Some(Set {
inner: a.inner.clone(),
}),
_ => None,
}
}
pub fn as_container(&self) -> Option<Container> {
match &self.typed {
TypedValue::Set(s) => Some(s.inner.clone()),
TypedValue::Dictionary(d) => Some(d.inner.clone()),
TypedValue::Array(a) => Some(a.inner.clone()),
_ => None,
}
}
pub fn as_dictionary(&self) -> Option<Dictionary> {
match &self.typed {
TypedValue::Set(s) => Some(Dictionary {
inner: s.inner.clone(),
}),
TypedValue::Dictionary(d) => Some(d.clone()),
TypedValue::Array(a) => Some(Dictionary {
inner: a.inner.clone(),
}),
_ => None,
}
}
pub fn as_array(&self) -> Option<Array> {
match &self.typed {
TypedValue::Set(s) => Some(Array {
inner: s.inner.clone(),
}),
TypedValue::Dictionary(d) => Some(Array {
inner: d.inner.clone(),
}),
TypedValue::Array(a) => Some(a.clone()),
_ => None,
}
}
pub fn as_str(&self) -> Option<&str> {
match &self.typed {
TypedValue::String(s) => Some(s.as_str()),
_ => None,
}
}
pub fn as_string(&self) -> Option<String> {
self.as_str().map(|s| s.to_string())
}
pub fn as_bool(&self) -> Option<bool> {
match self.typed {
TypedValue::Int(i) => match i {
0 => Some(false),
1 => Some(true),
_ => None,
},
TypedValue::Dictionary(d) => d.prove(&key.typed().try_into()?),
TypedValue::Set(s) => Ok((key, s.prove(key)?)),
_ => Err(Error::custom(format!(
"Invalid container value {}",
self.typed()
))),
}
}
/// Determines Merkle non-existence proof for `key` in `self` (if applicable).
pub(crate) fn prove_nonexistence<'a>(&'a self, key: &'a Value) -> Result<MerkleProof> {
match &self.typed() {
TypedValue::Array(_) => Err(Error::custom(
"Arrays do not support `NotContains` operation.".to_string(),
)),
TypedValue::Dictionary(d) => d.prove_nonexistence(&key.typed().try_into()?),
TypedValue::Set(s) => s.prove_nonexistence(key),
_ => Err(Error::custom(format!(
"Invalid container value {}",
self.typed()
))),
}
}
/// Returns a Merkle state transition proof for inserting a
/// key-value pair (if applicable).
pub(crate) fn prove_insertion(
&self,
key: &Value,
value: &Value,
) -> Result<MerkleTreeStateTransitionProof> {
let container = self.typed().clone();
match container {
TypedValue::Dictionary(mut d) => d.insert(&key.typed().try_into()?, value),
TypedValue::Set(mut s) => s.insert(value),
_ => Err(Error::custom(format!(
"Invalid container value {}",
self.typed()
))),
}
}
/// Returns a Merkle state transition proof for updating a
/// key-value pair (if applicable).
pub(crate) fn prove_update(
&self,
key: &Value,
value: &Value,
) -> Result<MerkleTreeStateTransitionProof> {
let container = self.typed().clone();
match container {
TypedValue::Array(mut a) => match key.typed() {
TypedValue::Int(i) if i >= &0 => a.update(*i as usize, value),
_ => Err(Error::custom(format!(
"Invalid key {} for container {}.",
key, self
)))?,
},
TypedValue::Dictionary(mut d) => d.update(&key.typed().try_into()?, value),
_ => Err(Error::custom(format!(
"Invalid container value {} for update op",
self.typed()
))),
}
}
/// Returns a Merkle state transition proof for deleting a
/// key (if applicable).
pub(crate) fn prove_deletion(&self, key: &Value) -> Result<MerkleTreeStateTransitionProof> {
let container = self.typed().clone();
match container {
TypedValue::Dictionary(mut d) => d.delete(&key.typed().try_into()?),
TypedValue::Set(mut s) => s.delete(key),
_ => Err(Error::custom(format!(
"Invalid container value {}",
self.typed()
))),
_ => None,
}
}
}

43
src/middleware/operation.rs
View file

@ -7,17 +7,14 @@ use serde::{Deserialize, Serialize};
use crate::{
backends::plonky2::primitives::{
ec::{
curve::{Point as PublicKey, GROUP_ORDER},
schnorr::{SecretKey, Signature},
},
ec::{curve::GROUP_ORDER, schnorr::Signature},
merkletree::{MerkleProof, MerkleTree, MerkleTreeOp, MerkleTreeStateTransitionProof},
},
middleware::{
hash_values, AnchoredKey, CustomPredicate, CustomPredicateRef, Error, Hash, Key,
MiddlewareInnerError, NativePredicate, Params, Predicate, PredicateOrWildcard, Result,
Statement, StatementArg, StatementTmpl, StatementTmplArg, ToFields, TypedValue, Value,
ValueRef, Wildcard, F,
Statement, StatementArg, StatementTmpl, StatementTmplArg, ToFields, Value, ValueRef,
Wildcard, F,
},
};
@ -241,6 +238,10 @@ pub(crate) fn hash_op(x: Value, y: Value) -> Value {
Value::from(hash_values(&[x, y]))
}
fn ok_or_type_err<T>(o: Option<T>, v: &Value, typ: &'static str) -> Result<T> {
o.ok_or_else(|| Error::custom(format!("{v} type is not {typ}")))
}
impl Operation {
pub fn op_type(&self) -> OperationType {
type OT = OperationType;
@ -404,20 +405,20 @@ impl Operation {
v3: &Value,
f: impl FnOnce(i64, i64) -> i64,
) -> Result<bool> {
let i1: i64 = v1.typed().try_into()?;
let i2: i64 = v2.typed().try_into()?;
let i3: i64 = v3.typed().try_into()?;
let i1 = ok_or_type_err(v1.as_int(), v1, "Int")?;
let i2 = ok_or_type_err(v2.as_int(), v2, "Int")?;
let i3 = ok_or_type_err(v3.as_int(), v3, "Int")?;
Ok(i1 == f(i2, i3))
}
pub(crate) fn check_public_key(v1: &Value, v2: &Value) -> Result<bool> {
let pk: PublicKey = v1.typed().try_into()?;
let sk: SecretKey = v2.typed().try_into()?;
let pk = ok_or_type_err(v1.as_public_key(), v1, "PublicKey")?;
let sk = ok_or_type_err(v2.as_secret_key(), v2, "SecretKey")?;
Ok(sk.0 < *GROUP_ORDER && pk == sk.public_key())
}
pub(crate) fn check_signed_by(msg: &Value, pk: &Value, sig: &Signature) -> Result<bool> {
let pk: PublicKey = pk.typed().try_into()?;
let pk = ok_or_type_err(pk.as_public_key(), pk, "PublicKey")?;
Ok(sig.verify(pk, msg.raw()))
}
@ -428,8 +429,8 @@ impl Operation {
let val = |v, s| value_from_op(s, v).ok_or_else(deduction_err);
let int_val = |v, s| {
let v_op = value_from_op(s, v).ok_or_else(deduction_err)?;
match v_op.typed() {
&TypedValue::Int(i) => Ok(i),
match v_op.as_int() {
Some(i) => Ok(i),
_ => Err(deduction_err()),
}
};
@ -494,8 +495,7 @@ impl Operation {
&& pf.op_value == value.raw())
.then_some(())
.ok_or(Error::custom(
"The provided Merkle tree state transition proof does not match the claim."
.into(),
"The provided Merkle tree state transition proof does not match the claim.",
))?;
MerkleTree::verify_state_transition(pf)?;
true
@ -515,8 +515,7 @@ impl Operation {
&& pf.op_value == value.raw())
.then_some(())
.ok_or(Error::custom(
"The provided Merkle tree state transition proof does not match the claim."
.into(),
"The provided Merkle tree state transition proof does not match the claim.",
))?;
MerkleTree::verify_state_transition(pf)?;
true
@ -534,8 +533,7 @@ impl Operation {
&& pf.op_key == key.raw())
.then_some(())
.ok_or(Error::custom(
"The provided Merkle tree state transition proof does not match the claim."
.into(),
"The provided Merkle tree state transition proof does not match the claim.",
))?;
MerkleTree::verify_state_transition(pf)?;
true
@ -789,9 +787,8 @@ impl fmt::Display for Operation {
pub(crate) fn root_key_to_ak(root: &Value, key: &Value) -> Option<AnchoredKey> {
let root_hash = Hash::from(root.raw());
Key::try_from(key.typed())
.map(|key| AnchoredKey::new(root_hash, key))
.ok()
key.as_str()
.map(|s| AnchoredKey::new(root_hash, Key::from(s)))
}
/// Returns the value associated with `output_ref`.