Featurize middleware types that are actually defined by the backend (#94)

At the middleware we were defining some types that actually are dependant on the
backend no matter how we define them in the middleware.

For example, we were hardcoding the `Hash` and `Value` types and their related
behaviour (eg. `.to_fields()`) to be based on the length of 4 field elements,
but that's not a choice of the middleware, and in fact this is determined by the
backend itself. On the same time, those types and related methods do not belong
to the backend, since conceptually they are part of the middleware reasoning.

The intention of this PR is not to prematurely abstract the library, but to
avoid inconsistencies where a type or parameter is defined in the middleware to
have certain carachteristic and later in the backend it gets used differently.
The idea is that those types and parameters (eg. lengths) have a single source
of truth in the code; and in the case of the "base types" (hash, value, etc)
this is determined by the backend being used under the hood, not by a choice of
the middleware parameters.

The idea with this approach, is that the frontend & middleware should not need
to import the proving library used by the backend (eg. plonky2, plonky3, etc).

As mentioned earlier, the `Hash` and `Value` types are types belonging at the
middleware, and is the middleware who reasons about them, but depending on the
backend being used, the `Hash` and `Value` types will have different sizes. So
it's the backend being used who actually defines their nature under the hood.
For example with a plonky2 backend, these types will have a length of 4 field
elements, whereas with a plonky3 backend they will have a length of 8 field
eleements.

Note that his approach does not introduce new traits or abstract code, just
makes use of rust features to define 'base types' that are being used in the
middleware.

Cargo.toml

@@ -8,10 +8,15 @@ name = "pod2"
 path = "src/lib.rs"
 
 [dependencies]
-plonky2 = { git = "https://github.com/0xPolygonZero/plonky2" }
 hex = "0.4.3"
 itertools = "0.14.0"
 strum = "0.26"
 strum_macros = "0.26"
 anyhow = "1.0.56"
 dyn-clone = "1.0.18"
+# enabled by features:
+plonky2 = { git = "https://github.com/0xPolygonZero/plonky2", optional = true }
+
+[features]
+default = ["backend_plonky2"]
+backend_plonky2 = ["plonky2"]

README.md

@@ -2,6 +2,8 @@
 
 ## Usage
 - Run tests: `cargo test --release`
+- Rustfmt: `cargo fmt`
+- Check [typos](https://github.com/crate-ci/typos): `typos -c .github/workflows/typos.toml`
 
 ## Book
 The `book` contains the specification of POD2. A rendered version of the site can be found at: https://0xparc.github.io/pod2/

src/backends/mod.rs

@@ -1,3 +1,2 @@
-pub mod mock_main;
-pub mod mock_signed;
+#[cfg(feature = "backend_plonky2")]
 pub mod plonky2;

src/backends/plonky2.rs

@@ -1 +0,0 @@
-

src/backends/plonky2/basetypes.rs

@@ -0,0 +1,203 @@
+//! This file exposes the middleware::basetypes to be used in the middleware when the
+//! `backend_plonky2` feature is enabled.
+//! See src/middleware/basetypes.rs for more details.
+
+use anyhow::{anyhow, Error, Result};
+use hex::{FromHex, FromHexError};
+use plonky2::field::goldilocks_field::GoldilocksField;
+use plonky2::field::types::{Field, PrimeField64};
+use plonky2::hash::poseidon::PoseidonHash;
+use plonky2::plonk::config::Hasher;
+use plonky2::plonk::config::PoseidonGoldilocksConfig;
+use std::cmp::{Ord, Ordering};
+use std::fmt;
+
+use crate::middleware::{Params, ToFields};
+
+/// F is the native field we use everywhere.  Currently it's Goldilocks from plonky2
+pub type F = GoldilocksField;
+/// C is the Plonky2 config used in POD2 to work with Plonky2 recursion.
+pub type C = PoseidonGoldilocksConfig;
+/// D defines the extension degree of the field used in the Plonky2 proofs (quadratic extension).
+pub const D: usize = 2;
+
+pub const HASH_SIZE: usize = 4;
+pub const VALUE_SIZE: usize = 4;
+
+pub const EMPTY: Value = Value([F::ZERO, F::ZERO, F::ZERO, F::ZERO]);
+pub const SELF_ID_HASH: Hash = Hash([F::ONE, F::ZERO, F::ZERO, F::ZERO]);
+pub const NULL: Hash = Hash([F::ZERO, F::ZERO, F::ZERO, F::ZERO]);
+
+#[derive(Clone, Copy, Debug, Default, Hash, PartialEq, Eq)]
+pub struct Value(pub [F; VALUE_SIZE]);
+
+impl ToFields for Value {
+    fn to_fields(&self, _params: &Params) -> (Vec<F>, usize) {
+        (self.0.to_vec(), VALUE_SIZE)
+    }
+}
+
+impl Value {
+    pub fn to_bytes(self) -> Vec<u8> {
+        self.0
+            .iter()
+            .flat_map(|e| e.to_canonical_u64().to_le_bytes())
+            .collect()
+    }
+}
+
+impl Ord for Value {
+    fn cmp(&self, other: &Self) -> Ordering {
+        for (lhs, rhs) in self.0.iter().zip(other.0.iter()).rev() {
+            let (lhs, rhs) = (lhs.to_canonical_u64(), rhs.to_canonical_u64());
+            if lhs < rhs {
+                return Ordering::Less;
+            } else if lhs > rhs {
+                return Ordering::Greater;
+            }
+        }
+        return Ordering::Equal;
+    }
+}
+
+impl PartialOrd for Value {
+    fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
+        Some(self.cmp(other))
+    }
+}
+
+impl From<i64> for Value {
+    fn from(v: i64) -> Self {
+        let lo = F::from_canonical_u64((v as u64) & 0xffffffff);
+        let hi = F::from_canonical_u64((v as u64) >> 32);
+        Value([lo, hi, F::ZERO, F::ZERO])
+    }
+}
+
+impl From<Hash> for Value {
+    fn from(h: Hash) -> Self {
+        Value(h.0)
+    }
+}
+
+impl TryInto<i64> for Value {
+    type Error = Error;
+    fn try_into(self) -> std::result::Result<i64, Self::Error> {
+        let value = self.0;
+        if &value[2..] != &[F::ZERO, F::ZERO]
+            || value[..2]
+                .iter()
+                .all(|x| x.to_canonical_u64() > u32::MAX as u64)
+        {
+            Err(anyhow!("Value not an element of the i64 embedding."))
+        } else {
+            Ok((value[0].to_canonical_u64() + value[1].to_canonical_u64() << 32) as i64)
+        }
+    }
+}
+
+impl fmt::Display for Value {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        if self.0[2].is_zero() && self.0[3].is_zero() {
+            // Assume this is an integer
+            let (l0, l1) = (self.0[0].to_canonical_u64(), self.0[1].to_canonical_u64());
+            assert!(l0 < (1 << 32));
+            assert!(l1 < (1 << 32));
+            write!(f, "{}", l0 + l1 * (1 << 32))
+        } else {
+            // Assume this is a hash
+            Hash(self.0).fmt(f)
+        }
+    }
+}
+
+#[derive(Clone, Copy, Debug, Default, Hash, Eq, PartialEq)]
+pub struct Hash(pub [F; HASH_SIZE]);
+
+pub fn hash_value(input: &Value) -> Hash {
+    Hash(PoseidonHash::hash_no_pad(&input.0).elements)
+}
+pub fn hash_fields(input: &[F]) -> Hash {
+    Hash(PoseidonHash::hash_no_pad(&input).elements)
+}
+
+impl From<Value> for Hash {
+    fn from(v: Value) -> Self {
+        Hash(v.0)
+    }
+}
+impl Hash {
+    pub fn value(self) -> Value {
+        Value(self.0)
+    }
+}
+
+impl ToFields for Hash {
+    fn to_fields(&self, _params: &Params) -> (Vec<F>, usize) {
+        (self.0.to_vec(), VALUE_SIZE)
+    }
+}
+
+impl Ord for Hash {
+    fn cmp(&self, other: &Self) -> Ordering {
+        Value(self.0).cmp(&Value(other.0))
+    }
+}
+
+impl PartialOrd for Hash {
+    fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
+        Some(self.cmp(other))
+    }
+}
+
+impl fmt::Display for Hash {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        let v0 = self.0[0].to_canonical_u64();
+        for i in 0..4 {
+            write!(f, "{:02x}", (v0 >> (i * 8)) & 0xff)?;
+        }
+        write!(f, "…")
+    }
+}
+
+impl FromHex for Hash {
+    type Error = FromHexError;
+
+    // TODO make it dependant on backend::Value len
+    fn from_hex<T: AsRef<[u8]>>(hex: T) -> Result<Self, Self::Error> {
+        // In little endian
+        let bytes = <[u8; 32]>::from_hex(hex)?;
+        let mut buf: [u8; 8] = [0; 8];
+        let mut inner = [F::ZERO; 4];
+        for i in 0..4 {
+            buf.copy_from_slice(&bytes[8 * i..8 * (i + 1)]);
+            inner[i] = F::from_canonical_u64(u64::from_le_bytes(buf));
+        }
+        Ok(Self(inner))
+    }
+}
+
+impl From<&str> for Hash {
+    fn from(s: &str) -> Self {
+        hash_str(s)
+    }
+}
+
+pub fn hash_str(s: &str) -> Hash {
+    let mut input = s.as_bytes().to_vec();
+    input.push(1); // padding
+
+    // Merge 7 bytes into 1 field, because the field is slightly below 64 bits
+    let input: Vec<F> = input
+        .chunks(7)
+        .map(|bytes| {
+            let mut v: u64 = 0;
+            for b in bytes.iter().rev() {
+                v <<= 8;
+                v += *b as u64;
+            }
+            F::from_canonical_u64(v)
+        })
+        .collect();
+    Hash(PoseidonHash::hash_no_pad(&input).elements)
+}

src/backends/plonky2/mock_main/mod.rs

@@ -327,7 +327,7 @@ impl MockMainPod {
             statements[statements.len() - params.max_public_statements..].to_vec();
 
         // get the id out of the public statements
-        let id: PodId = PodId(hash_statements(&public_statements, *params));
+        let id: PodId = PodId(hash_statements(&public_statements, params));
 
         Ok(Self {
             params: params.clone(),
@@ -362,10 +362,10 @@ impl MockMainPod {
     }
 }
 
-pub fn hash_statements(statements: &[Statement], params: Params) -> middleware::Hash {
+pub fn hash_statements(statements: &[Statement], _params: &Params) -> middleware::Hash {
     let field_elems = statements
         .into_iter()
-        .flat_map(|statement| statement.clone().to_fields(params).0)
+        .flat_map(|statement| statement.clone().to_fields(_params).0)
         .collect::<Vec<_>>();
     Hash(PoseidonHash::hash_no_pad(&field_elems).elements)
 }
@@ -376,7 +376,7 @@ impl Pod for MockMainPod {
         // get the input_statements from the self.statements
         let input_statements = &self.statements[input_statement_offset..];
         // get the id out of the public statements, and ensure it is equal to self.id
-        let ids_match = self.id == PodId(hash_statements(&self.public_statements, self.params));
+        let ids_match = self.id == PodId(hash_statements(&self.public_statements, &self.params));
         // find a ValueOf statement from the public statements with key=KEY_TYPE and check that the
         // value is PodType::MockMainPod
         let has_type_statement = self
@@ -474,7 +474,7 @@ impl Pod for MockMainPod {
 #[cfg(test)]
 pub mod tests {
     use super::*;
-    use crate::backends::mock_signed::MockSigner;
+    use crate::backends::plonky2::mock_signed::MockSigner;
     use crate::examples::{
         great_boy_pod_full_flow, tickets_pod_full_flow, zu_kyc_pod_builder,
         zu_kyc_sign_pod_builders,

src/backends/plonky2/mock_main/statement.rs

@@ -21,13 +21,13 @@ impl Statement {
 }
 
 impl ToFields for Statement {
-    fn to_fields(&self, params: Params) -> (Vec<middleware::F>, usize) {
-        let (native_statement_f, native_statement_f_len) = self.0.to_fields(params);
+    fn to_fields(&self, _params: &Params) -> (Vec<middleware::F>, usize) {
+        let (native_statement_f, native_statement_f_len) = self.0.to_fields(_params);
         let (vec_statementarg_f, vec_statementarg_f_len) = self
             .1
             .clone()
             .into_iter()
-            .map(|statement_arg| statement_arg.to_fields(params))
+            .map(|statement_arg| statement_arg.to_fields(_params))
             .fold((Vec::new(), 0), |mut acc, (f, l)| {
                 acc.0.extend(f);
                 acc.1 += l;

src/backends/plonky2/mod.rs

@@ -0,0 +1,3 @@
+pub mod basetypes;
+pub mod mock_main;
+pub mod mock_signed;

src/examples.rs

@@ -1,7 +1,7 @@
 use anyhow::Result;
 use std::collections::HashMap;
 
-use crate::backends::mock_signed::MockSigner;
+use crate::backends::plonky2::mock_signed::MockSigner;
 use crate::frontend::{MainPodBuilder, SignedPod, SignedPodBuilder, Value};
 use crate::middleware::{containers::Dictionary, Params, PodType, KEY_SIGNER, KEY_TYPE};
 use crate::op;

src/frontend/custom.rs

@@ -316,7 +316,7 @@ mod tests {
         );
 
         let eth_dos_batch_b = builder.finish();
-        let fields = eth_dos_batch_b.to_fields(params);
+        let fields = eth_dos_batch_b.to_fields(&params);
         println!("Batch b, serialized: {:?}", fields);
     }
 }

src/frontend/mod.rs

@@ -65,12 +65,12 @@ impl From<bool> for Value {
 impl From<&Value> for middleware::Value {
     fn from(v: &Value) -> Self {
         match v {
-            Value::String(s) => middleware::Value(hash_str(s).0),
+            Value::String(s) => hash_str(s).value(),
             Value::Int(v) => middleware::Value::from(*v),
             Value::Bool(b) => middleware::Value::from(*b as i64),
-            Value::Dictionary(d) => middleware::Value(d.commitment().0),
-            Value::Set(s) => middleware::Value(s.commitment().0),
-            Value::Array(a) => middleware::Value(a.commitment().0),
+            Value::Dictionary(d) => d.commitment().value(),
+            Value::Set(s) => s.commitment().value(),
+            Value::Array(a) => a.commitment().value(),
             Value::Raw(v) => v.clone(),
         }
     }
@@ -513,8 +513,8 @@ pub mod build_utils {
 #[cfg(test)]
 pub mod tests {
     use super::*;
-    use crate::backends::mock_main::MockProver;
-    use crate::backends::mock_signed::MockSigner;
+    use crate::backends::plonky2::mock_main::MockProver;
+    use crate::backends::plonky2::mock_signed::MockSigner;
     use crate::examples::{
         great_boy_pod_full_flow, tickets_pod_full_flow, zu_kyc_pod_builder,
         zu_kyc_sign_pod_builders,

src/middleware/basetypes.rs

@@ -0,0 +1,40 @@
+//! This file exposes the backend dependent basetypes as middleware types,
+//! taking them from the feature-enabled backend.
+//!
+//! This is done in order to avoid inconsistencies where a type or parameter is
+//! defined in the middleware to have certain carachteristic and later in the
+//! backend it gets used differently. The idea is that those types and
+//! parameters (eg. lengths) have a single source of truth in the code; and in
+//! the case of the "base types" this is determined by the backend being used
+//! under the hood, not by a choice of the middleware parameters.
+//!
+//! The idea with this approach, is that the frontend & middleware should not
+//! need to import the proving library used by the backend (eg. plonky2,
+//! plonky3, etc).
+//!
+//! For example, the `Hash` and `Value` types are types belonging at the
+//! middleware, and is the middleware who reasons about them, but depending on
+//! the backend being used, the `Hash` and `Value` types will have different
+//! sizes. So it's the backend being used who actually defines their nature
+//! under the hood. For example with a plonky2 backend, these types will have a
+//! length of 4 field elements, whereas with a plonky3 backend they will have a
+//! length of 8 field eleements.
+//!
+//! Note that his approach does not introduce new traits or abstract code,
+//! just makes use of rust features to define 'base types' that are being used
+//! in the middleware.
+//!
+//!
+//! NOTE (TMP): current implementation still uses plonky2 in the middleware for
+//! u64/i64 to F conversion. Eventually we will do those conversions through the
+//! approach described in this file, removing the imports of plonky2 in the
+//! middleware.
+
+/// Value, Hash and F are imported based on 'features'. For example by default
+/// we use the 'plonky2' feature, but it could be used a 'plonky3' feature, so
+/// then the Value, Hash and F types would come from the plonky3 backend.
+#[cfg(feature = "backend_plonky2")]
+pub use crate::backends::plonky2::basetypes::{
+    hash_fields, hash_str, hash_value, Hash, Value, EMPTY, F, HASH_SIZE, NULL, SELF_ID_HASH,
+    VALUE_SIZE,
+};

src/middleware/containers.rs

@@ -1,14 +1,13 @@
 /// This file implements the types defined at
 /// https://0xparc.github.io/pod2/values.html#dictionary-array-set .
 use anyhow::Result;
-use plonky2::hash::poseidon::PoseidonHash;
-use plonky2::plonk::config::Hasher;
 use std::collections::HashMap;
 
-use super::{Hash, Value, EMPTY};
 use crate::constants::MAX_DEPTH;
 use crate::primitives::merkletree::{MerkleProof, MerkleTree};
 
+use super::basetypes::{hash_value, Hash, Value, EMPTY};
+
 /// Dictionary: the user original keys and values are hashed to be used in the leaf.
 ///    leaf.key=hash(original_key)
 ///    leaf.value=hash(original_value)
@@ -76,8 +75,8 @@ impl Set {
         let kvs: HashMap<Value, Value> = set
             .into_iter()
             .map(|e| {
-                let h = PoseidonHash::hash_no_pad(&e.0).elements;
-                (Value(h), EMPTY)
+                let h = hash_value(e);
+                (Value::from(h), EMPTY)
             })
             .collect();
         Ok(Self {

src/middleware/custom.rs

@@ -3,14 +3,10 @@ use std::{fmt, hash as h, iter::zip};
 
 use anyhow::{anyhow, Result};
 use plonky2::field::types::Field;
-use plonky2::hash::poseidon::PoseidonHash;
-use plonky2::plonk::config::Hasher;
-
-use crate::middleware::{Operation, SELF};
 
 use super::{
-    hash_str, AnchoredKey, Hash, NativePredicate, Params, PodId, Statement, StatementArg, ToFields,
-    Value, F,
+    hash_fields, AnchoredKey, Hash, NativePredicate, Params, PodId, Statement, StatementArg,
+    ToFields, Value, F,
 };
 
 // BEGIN Custom 1b
@@ -44,9 +40,9 @@ impl fmt::Display for HashOrWildcard {
 }
 
 impl ToFields for HashOrWildcard {
-    fn to_fields(&self, params: Params) -> (Vec<F>, usize) {
+    fn to_fields(&self, _params: &Params) -> (Vec<F>, usize) {
         match self {
-            HashOrWildcard::Hash(h) => h.to_fields(params),
+            HashOrWildcard::Hash(h) => h.to_fields(_params),
             HashOrWildcard::Wildcard(w) => {
                 let usizes: Vec<usize> = vec![0, 0, 0, *w];
                 let fields: Vec<F> = usizes
@@ -86,7 +82,7 @@ impl StatementTmplArg {
 }
 
 impl ToFields for StatementTmplArg {
-    fn to_fields(&self, params: Params) -> (Vec<F>, usize) {
+    fn to_fields(&self, _params: &Params) -> (Vec<F>, usize) {
         // None => (0, ...)
         // Literal(value) => (1, [value], 0, 0, 0, 0)
         // Key(hash_or_wildcard1, hash_or_wildcard2)
@@ -103,15 +99,15 @@ impl ToFields for StatementTmplArg {
             }
             StatementTmplArg::Literal(v) => {
                 let fields: Vec<F> = std::iter::once(F::from_canonical_u64(1))
-                    .chain(v.to_fields(params).0.into_iter())
+                    .chain(v.to_fields(_params).0.into_iter())
                     .chain(std::iter::repeat_with(|| F::from_canonical_u64(0)).take(hash_size))
                     .collect();
                 (fields, statement_tmpl_arg_size)
             }
             StatementTmplArg::Key(hw1, hw2) => {
                 let fields: Vec<F> = std::iter::once(F::from_canonical_u64(2))
-                    .chain(hw1.to_fields(params).0.into_iter())
-                    .chain(hw2.to_fields(params).0.into_iter())
+                    .chain(hw1.to_fields(_params).0.into_iter())
+                    .chain(hw2.to_fields(_params).0.into_iter())
                     .collect();
                 (fields, statement_tmpl_arg_size)
             }
@@ -173,13 +169,18 @@ impl StatementTmpl {
 }
 
 impl ToFields for StatementTmpl {
-    fn to_fields(&self, params: Params) -> (Vec<F>, usize) {
+    fn to_fields(&self, params: &Params) -> (Vec<F>, usize) {
         // serialize as:
         // predicate (6 field elements)
         // then the StatementTmplArgs
+
+        // TODO think if this check should go into the StatementTmpl creation,
+        // instead of at the `to_fields` method, where we should assume that the
+        // values are already valid
         if self.1.len() > params.max_statement_args {
             panic!("Statement template has too many arguments");
         }
+
         let mut fields: Vec<F> = self
             .0
             .to_fields(params)
@@ -203,16 +204,21 @@ pub struct CustomPredicate {
 }
 
 impl ToFields for CustomPredicate {
-    fn to_fields(&self, params: Params) -> (Vec<F>, usize) {
+    fn to_fields(&self, params: &Params) -> (Vec<F>, usize) {
         // serialize as:
         // conjunction (one field element)
         // args_len (one field element)
         // statements
         //   (params.max_custom_predicate_arity * params.statement_tmpl_size())
         //   field elements
+
+        // TODO think if this check should go into the StatementTmpl creation,
+        // instead of at the `to_fields` method, where we should assume that the
+        // values are already valid
         if self.statements.len() > params.max_custom_predicate_arity {
             panic!("Custom predicate depends on too many statements");
         }
+
         let mut fields: Vec<F> = std::iter::once(F::from_bool(self.conjunction))
             .chain(std::iter::once(F::from_canonical_usize(self.args_len)))
             .chain(self.statements.iter().flat_map(|st| st.to_fields(params).0))
@@ -254,11 +260,16 @@ pub struct CustomPredicateBatch {
 }
 
 impl ToFields for CustomPredicateBatch {
-    fn to_fields(&self, params: Params) -> (Vec<F>, usize) {
+    fn to_fields(&self, params: &Params) -> (Vec<F>, usize) {
         // all the custom predicates in order
+
+        // TODO think if this check should go into the StatementTmpl creation,
+        // instead of at the `to_fields` method, where we should assume that the
+        // values are already valid
         if self.predicates.len() > params.max_custom_batch_size {
             panic!("Predicate batch exceeds maximum size");
         }
+
         let mut fields: Vec<F> = self
             .predicates
             .iter()
@@ -273,9 +284,9 @@ impl ToFields for CustomPredicateBatch {
 }
 
 impl CustomPredicateBatch {
-    pub fn hash(&self, params: Params) -> Hash {
-        let input = self.to_fields(params).0;
-        let h = Hash(PoseidonHash::hash_no_pad(&input).elements);
+    pub fn hash(&self, _params: &Params) -> Hash {
+        let input = self.to_fields(_params).0;
+        let h = hash_fields(&input);
         h
     }
 }
@@ -297,7 +308,7 @@ impl From<NativePredicate> for Predicate {
 }
 
 impl ToFields for Predicate {
-    fn to_fields(&self, params: Params) -> (Vec<F>, usize) {
+    fn to_fields(&self, _params: &Params) -> (Vec<F>, usize) {
         // serialize:
         // NativePredicate(id) as (0, id, 0, 0, 0, 0) -- id: usize
         // BatchSelf(i) as (1, i, 0, 0, 0, 0) -- i: usize
@@ -306,27 +317,20 @@ impl ToFields for Predicate {
         //                      -- i: usize
 
         // in every case: pad to (hash_size + 2) field elements
-        let mut fields: Vec<F> = Vec::new();
-        match self {
-            Self::Native(p) => {
-                fields = std::iter::once(F::from_canonical_u64(1))
-                    .chain(p.to_fields(params).0.into_iter())
-                    .collect();
-            }
-            Self::BatchSelf(i) => {
-                fields = std::iter::once(F::from_canonical_u64(2))
+        let mut fields: Vec<F> = match self {
+            Self::Native(p) => std::iter::once(F::from_canonical_u64(1))
+                .chain(p.to_fields(_params).0.into_iter())
+                .collect(),
+            Self::BatchSelf(i) => std::iter::once(F::from_canonical_u64(2))
                 .chain(std::iter::once(F::from_canonical_usize(*i)))
-                    .collect();
-            }
-            Self::Custom(CustomPredicateRef(pb, i)) => {
-                fields = std::iter::once(F::from_canonical_u64(3))
-                    .chain(pb.hash(params).0)
+                .collect(),
+            Self::Custom(CustomPredicateRef(pb, i)) => std::iter::once(F::from_canonical_u64(3))
+                .chain(pb.hash(_params).0)
                 .chain(std::iter::once(F::from_canonical_usize(*i)))
-                    .collect();
-            }
-        }
-        fields.resize_with(params.predicate_size(), || F::from_canonical_u64(0));
-        (fields, params.predicate_size())
+                .collect(),
+        };
+        fields.resize_with(Params::predicate_size(), || F::from_canonical_u64(0));
+        (fields, Params::predicate_size())
     }
 }
 

src/middleware/mod.rs

@@ -1,36 +1,38 @@
 //! The middleware includes the type definitions and the traits used to connect the frontend and
 //! the backend.
 
+mod basetypes;
+pub mod containers;
 mod custom;
 mod operation;
 mod statement;
+pub use basetypes::*;
 pub use custom::*;
 pub use operation::*;
 pub use statement::*;
 
-use anyhow::{anyhow, Error, Result};
+use anyhow::Result;
 use dyn_clone::DynClone;
-use hex::{FromHex, FromHexError};
-use plonky2::field::goldilocks_field::GoldilocksField;
-use plonky2::field::types::{Field, PrimeField64};
-use plonky2::hash::poseidon::PoseidonHash;
-use plonky2::plonk::config::{Hasher, PoseidonGoldilocksConfig};
 use std::any::Any;
-use std::cmp::{Ord, Ordering};
 use std::collections::HashMap;
 use std::fmt;
 
-pub mod containers;
+pub const SELF: PodId = PodId(SELF_ID_HASH);
 
-/// F is the native field we use everywhere.  Currently it's Goldilocks from plonky2
-pub type F = GoldilocksField;
-/// C is the Plonky2 config used in POD2 to work with Plonky2 recursion.
-pub type C = PoseidonGoldilocksConfig;
-/// D defines the extension degree of the field used in the Plonky2 proofs (quadratic extension).
-pub const D: usize = 2;
+impl fmt::Display for PodId {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        if *self == SELF {
+            write!(f, "self")
+        } else if self.0 == NULL {
+            write!(f, "null")
+        } else {
+            write!(f, "{}", self.0)
+        }
+    }
+}
 
-#[derive(Clone, Debug, PartialEq, Eq, Hash)]
 /// AnchoredKey is a tuple containing (OriginId: PodId, key: Hash)
+#[derive(Clone, Debug, PartialEq, Eq, Hash)]
 pub struct AnchoredKey(pub PodId, pub Hash);
 
 impl AnchoredKey {
@@ -52,168 +54,15 @@ impl fmt::Display for AnchoredKey {
 /// An entry consists of a key-value pair.
 pub type Entry = (String, Value);
 
-#[derive(Clone, Copy, Debug, Default, Hash, PartialEq, Eq)]
-pub struct Value(pub [F; 4]);
-
-impl ToFields for Value {
-    fn to_fields(&self, params: Params) -> (Vec<F>, usize) {
-        (self.0.to_vec(), 4)
-    }
-}
-
-impl Value {
-    pub fn to_bytes(self) -> Vec<u8> {
-        self.0
-            .iter()
-            .flat_map(|e| e.to_canonical_u64().to_le_bytes())
-            .collect()
-    }
-}
-
-impl Ord for Value {
-    fn cmp(&self, other: &Self) -> Ordering {
-        for (lhs, rhs) in self.0.iter().zip(other.0.iter()).rev() {
-            let (lhs, rhs) = (lhs.to_canonical_u64(), rhs.to_canonical_u64());
-            if lhs < rhs {
-                return Ordering::Less;
-            } else if lhs > rhs {
-                return Ordering::Greater;
-            }
-        }
-        return Ordering::Equal;
-    }
-}
-
-impl PartialOrd for Value {
-    fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
-        Some(self.cmp(other))
-    }
-}
-
-impl From<i64> for Value {
-    fn from(v: i64) -> Self {
-        let lo = F::from_canonical_u64((v as u64) & 0xffffffff);
-        let hi = F::from_canonical_u64((v as u64) >> 32);
-        Value([lo, hi, F::ZERO, F::ZERO])
-    }
-}
-
-impl From<Hash> for Value {
-    fn from(h: Hash) -> Self {
-        Value(h.0)
-    }
-}
-
-impl TryInto<i64> for Value {
-    type Error = Error;
-    fn try_into(self) -> std::result::Result<i64, Self::Error> {
-        let value = self.0;
-        if &value[2..] != &[F::ZERO, F::ZERO]
-            || value[..2]
-                .iter()
-                .all(|x| x.to_canonical_u64() > u32::MAX as u64)
-        {
-            Err(anyhow!("Value not an element of the i64 embedding."))
-        } else {
-            Ok((value[0].to_canonical_u64() + value[1].to_canonical_u64() << 32) as i64)
-        }
-    }
-}
-
-impl fmt::Display for Value {
-    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
-        if self.0[2].is_zero() && self.0[3].is_zero() {
-            // Assume this is an integer
-            let (l0, l1) = (self.0[0].to_canonical_u64(), self.0[1].to_canonical_u64());
-            assert!(l0 < (1 << 32));
-            assert!(l1 < (1 << 32));
-            write!(f, "{}", l0 + l1 * (1 << 32))
-        } else {
-            // Assume this is a hash
-            Hash(self.0).fmt(f)
-        }
-    }
-}
-
-#[derive(Clone, Copy, Debug, Default, Hash, Eq, PartialEq)]
-pub struct Hash(pub [F; 4]);
-
-impl ToFields for Hash {
-    fn to_fields(&self, params: Params) -> (Vec<F>, usize) {
-        (self.0.to_vec(), 4)
-    }
-}
-
-impl Ord for Hash {
-    fn cmp(&self, other: &Self) -> Ordering {
-        Value(self.0).cmp(&Value(other.0))
-    }
-}
-
-impl PartialOrd for Hash {
-    fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
-        Some(self.cmp(other))
-    }
-}
-
-pub const EMPTY: Value = Value([F::ZERO, F::ZERO, F::ZERO, F::ZERO]);
-pub const NULL: Hash = Hash([F::ZERO, F::ZERO, F::ZERO, F::ZERO]);
-
-impl fmt::Display for Hash {
-    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
-        let v0 = self.0[0].to_canonical_u64();
-        for i in 0..4 {
-            write!(f, "{:02x}", (v0 >> (i * 8)) & 0xff)?;
-        }
-        write!(f, "…")
-    }
-}
-
-impl FromHex for Hash {
-    type Error = FromHexError;
-
-    fn from_hex<T: AsRef<[u8]>>(hex: T) -> Result<Self, Self::Error> {
-        // In little endian
-        let bytes = <[u8; 32]>::from_hex(hex)?;
-        let mut buf: [u8; 8] = [0; 8];
-        let mut inner = [F::ZERO; 4];
-        for i in 0..4 {
-            buf.copy_from_slice(&bytes[8 * i..8 * (i + 1)]);
-            inner[i] = F::from_canonical_u64(u64::from_le_bytes(buf));
-        }
-        Ok(Self(inner))
-    }
-}
-
-impl From<&str> for Hash {
-    fn from(s: &str) -> Self {
-        hash_str(s)
-    }
-}
-
 #[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, Default)]
 pub struct PodId(pub Hash);
 
 impl ToFields for PodId {
-    fn to_fields(&self, params: Params) -> (Vec<F>, usize) {
+    fn to_fields(&self, params: &Params) -> (Vec<F>, usize) {
         self.0.to_fields(params)
     }
 }
 
-pub const SELF: PodId = PodId(Hash([F::ONE, F::ZERO, F::ZERO, F::ZERO]));
-
-impl fmt::Display for PodId {
-    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
-        if *self == SELF {
-            write!(f, "self")
-        } else if self.0 == NULL {
-            write!(f, "null")
-        } else {
-            write!(f, "{}", self.0)
-        }
-    }
-}
-
 pub enum PodType {
     None = 0,
     MockSigned = 1,
@@ -228,26 +77,7 @@ impl From<PodType> for Value {
     }
 }
 
-pub fn hash_str(s: &str) -> Hash {
-    let mut input = s.as_bytes().to_vec();
-    input.push(1); // padding
-
-    // Merge 7 bytes into 1 field, because the field is slightly below 64 bits
-    let input: Vec<F> = input
-        .chunks(7)
-        .map(|bytes| {
-            let mut v: u64 = 0;
-            for b in bytes.iter().rev() {
-                v <<= 8;
-                v += *b as u64;
-            }
-            F::from_canonical_u64(v)
-        })
-        .collect();
-    Hash(PoseidonHash::hash_no_pad(&input).elements)
-}
-
-#[derive(Clone, Debug, Copy, PartialEq, Eq)]
+#[derive(Clone, Debug, PartialEq, Eq)]
 pub struct Params {
     pub max_input_signed_pods: usize,
     pub max_input_main_pods: usize,
@@ -260,8 +90,6 @@ pub struct Params {
     // in a custom predicate
     pub max_custom_predicate_arity: usize,
     pub max_custom_batch_size: usize,
-    // number of field elements in a hash
-    pub hash_size: usize,
 }
 
 impl Params {
@@ -269,33 +97,33 @@ impl Params {
         self.max_statements - self.max_public_statements
     }
 
-    pub fn statement_tmpl_arg_size(self) -> usize {
-        2 * self.hash_size + 1
+    pub fn statement_tmpl_arg_size() -> usize {
+        2 * HASH_SIZE + 1
     }
 
-    pub fn predicate_size(self) -> usize {
-        self.hash_size + 2
+    pub fn predicate_size() -> usize {
+        HASH_SIZE + 2
     }
 
-    pub fn statement_tmpl_size(self) -> usize {
-        self.predicate_size() + self.max_statement_args * self.statement_tmpl_arg_size()
+    pub fn statement_tmpl_size(&self) -> usize {
+        Self::predicate_size() + self.max_statement_args * Self::statement_tmpl_arg_size()
     }
 
-    pub fn custom_predicate_size(self) -> usize {
+    pub fn custom_predicate_size(&self) -> usize {
         self.max_custom_predicate_arity * self.statement_tmpl_size() + 2
     }
 
-    pub fn custom_predicate_batch_size_field_elts(self) -> usize {
+    pub fn custom_predicate_batch_size_field_elts(&self) -> usize {
         self.max_custom_batch_size * self.custom_predicate_size()
     }
 
-    pub fn print_serialized_sizes(self) -> () {
+    pub fn print_serialized_sizes(&self) -> () {
         println!("Parameter sizes:");
         println!(
             "  Statement template argument: {}",
-            self.statement_tmpl_arg_size()
+            Self::statement_tmpl_arg_size()
         );
-        println!("  Predicate: {}", self.predicate_size());
+        println!("  Predicate: {}", Self::predicate_size());
         println!("  Statement template: {}", self.statement_tmpl_size());
         println!("  Custom predicate: {}", self.custom_predicate_size());
         println!(
@@ -318,7 +146,6 @@ impl Default for Params {
             max_operation_args: 5,
             max_custom_predicate_arity: 5,
             max_custom_batch_size: 5,
-            hash_size: 4,
         }
     }
 }
@@ -386,5 +213,5 @@ pub trait PodProver {
 pub trait ToFields {
     /// returns Vec<F> representation of the type, and a usize indicating how many field elements
     /// does the vector contain
-    fn to_fields(&self, params: Params) -> (Vec<F>, usize);
+    fn to_fields(&self, params: &Params) -> (Vec<F>, usize);
 }

src/middleware/statement.rs

@@ -1,9 +1,9 @@
 use anyhow::{anyhow, Result};
 use plonky2::field::types::Field;
-use std::{collections::HashMap, fmt};
+use std::fmt;
 use strum_macros::FromRepr;
 
-use super::{AnchoredKey, CustomPredicateRef, Hash, Params, Predicate, ToFields, Value, F};
+use super::{AnchoredKey, CustomPredicateRef, Params, Predicate, ToFields, Value, F};
 
 pub const KEY_SIGNER: &str = "_signer";
 pub const KEY_TYPE: &str = "_type";
@@ -25,7 +25,7 @@ pub enum NativePredicate {
 }
 
 impl ToFields for NativePredicate {
-    fn to_fields(&self, params: Params) -> (Vec<F>, usize) {
+    fn to_fields(&self, _params: &Params) -> (Vec<F>, usize) {
         (vec![F::from_canonical_u64(*self as u64)], 1)
     }
 }
@@ -88,12 +88,12 @@ impl Statement {
 }
 
 impl ToFields for Statement {
-    fn to_fields(&self, params: Params) -> (Vec<F>, usize) {
-        let (native_statement_f, native_statement_f_len) = self.code().to_fields(params);
+    fn to_fields(&self, _params: &Params) -> (Vec<F>, usize) {
+        let (native_statement_f, native_statement_f_len) = self.code().to_fields(_params);
         let (vec_statementarg_f, vec_statementarg_f_len) = self
             .args()
             .into_iter()
-            .map(|statement_arg| statement_arg.to_fields(params))
+            .map(|statement_arg| statement_arg.to_fields(_params))
             .fold((Vec::new(), 0), |mut acc, (f, l)| {
                 acc.0.extend(f);
                 acc.1 += l;
@@ -156,7 +156,7 @@ impl StatementArg {
 }
 
 impl ToFields for StatementArg {
-    fn to_fields(&self, params: Params) -> (Vec<F>, usize) {
+    fn to_fields(&self, _params: &Params) -> (Vec<F>, usize) {
         // NOTE: current version returns always the same amount of field elements in the returned
         // vector, which means that the `None` case is padded with 8 zeroes, and the `Literal` case
         // is padded with 4 zeroes. Since the returned vector will mostly be hashed (and reproduced
@@ -175,8 +175,8 @@ impl ToFields for StatementArg {
                 .concat()
             }
             StatementArg::Key(ak) => {
-                let (podid_f, _) = ak.0.to_fields(params);
-                let (hash_f, _) = ak.1.to_fields(params);
+                let (podid_f, _) = ak.0.to_fields(_params);
+                let (hash_f, _) = ak.1.to_fields(_params);
                 [podid_f, hash_f].concat()
             }
         };