Extend the work of https://github.com/0xPARC/pod2/pull/487 to the Containers (Dictionary, Set, Array). The merkle tree only stores `RawValue` for both the key and the value, so it is the responsibility of the Container to store the rich value. In order to handle containers with persistent storage efficiently (which means, cloning them or updating them should not cause an O(n) data copy) I figured we need to have a database of `Value`s indexed by their raw value; as this gives us deduplication and free cloning of containers. The issue with this approach is that in the current design we have collisions between Value's of different types: https://github.com/0xPARC/pod2/issues/426 and the current API relies on the single type of values. To resolve this issue I decided to change the API, instead of assuming that a Value has a fixed type, let the value be possibly multiple compatible types and let the user of the library try casting the Value to a particular type. For this I deprecated the public access of everything related to `TypedValue` and I propose for it to be considered an implementation detail and a blackbox from the external developer point of view. The `Value` type is now used like this: - To create a new Value use `Value::from(...)` where you can pass any compatible type (the same types as before) - To access the Value in typed form you cast it like `value.as_foo()` which returns `Option<Foo>`. Previously we had a collision between `true` and `1` (and `false` and `0`). Now it doesn't matter whether a value holds a `true` or a `1`, both should be seen as the same and both return `Some` when doing `as_int` and `as_bool`. Similarly we had collisions with containers. For example `set(0, 1, 2) == array[0, 1, 2]` and `set("a", "b") = dict("a": "a", "b": "b")`. Now any container can be casted to any of `set, array, dict`. There's a caveat here: each of these types expects a particular encoding of keys, so casting to the wrong type will return errors on some operations. With this design it no longer matters what is being stored and recovered because the API requires the user to express the expected type and any type with collisions for particular values can be casted to the right type. There's only one case where it's not desirable to swap one `TypedValue` for another: the `TypedValue::Raw`. If a non-`RawValue` in the DB is replaced by the corresponding `RawValue` we erase the required information to recover the rich value. For this reason the implementations of the database treat the `RawValue` as a special case: if an value is stored in non-`RawValue`, the corresponding `RawValue` can never overwrite it. If a value is stored in `RawValue`, a matching non-`RawValue` will overwrite it (promoting it to a rich value). This way we never lose data. A consequence of this is that the serialization, `Display` and `Debug` of a container is not stable. At any point any of the entries can be swapped for a "compatible" one if they share the storage with other containers that introduce collisions. I rewrote all containers as wrapper to a generic `Container` which holds a `Map` from `Value` to `Value`. The serialization of each container now uses the single implementation of the generic `Container`.
318 lines
12 KiB
Rust
318 lines
12 KiB
Rust
use schemars::JsonSchema;
|
|
use serde::{Deserialize, Serialize};
|
|
|
|
use super::Error;
|
|
use crate::{
|
|
frontend::MainPod,
|
|
middleware::{deserialize_pod, Hash, Params, Statement, VDSet},
|
|
};
|
|
|
|
#[derive(Serialize, Deserialize, JsonSchema, Debug, Clone, PartialEq)]
|
|
#[serde(rename_all = "camelCase")]
|
|
#[schemars(rename = "MainPod")]
|
|
pub struct SerializedMainPod {
|
|
params: Params,
|
|
pod_type: (usize, String),
|
|
sts_hash: Hash,
|
|
vd_set: VDSet,
|
|
public_statements: Vec<Statement>,
|
|
data: serde_json::Value,
|
|
}
|
|
|
|
impl SerializedMainPod {
|
|
pub fn statements_hash(&self) -> Hash {
|
|
self.sts_hash
|
|
}
|
|
}
|
|
|
|
impl From<MainPod> for SerializedMainPod {
|
|
fn from(pod: MainPod) -> Self {
|
|
let (pod_type, pod_type_name_str) = pod.pod.pod_type();
|
|
let data = pod.pod.serialize_data();
|
|
SerializedMainPod {
|
|
pod_type: (pod_type, pod_type_name_str.to_string()),
|
|
sts_hash: pod.statements_hash(),
|
|
vd_set: pod.pod.vd_set().clone(),
|
|
params: pod.params.clone(),
|
|
public_statements: pod.pod.pub_statements(),
|
|
data,
|
|
}
|
|
}
|
|
}
|
|
|
|
impl TryFrom<SerializedMainPod> for MainPod {
|
|
type Error = Error;
|
|
|
|
fn try_from(serialized: SerializedMainPod) -> Result<Self, Self::Error> {
|
|
let pod = deserialize_pod(
|
|
serialized.pod_type.0,
|
|
serialized.params.clone(),
|
|
serialized.sts_hash,
|
|
serialized.vd_set,
|
|
serialized.data,
|
|
)?;
|
|
let public_statements = pod.pub_statements();
|
|
Ok(Self {
|
|
pod,
|
|
public_statements,
|
|
params: serialized.params,
|
|
})
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use std::collections::{HashMap, HashSet};
|
|
|
|
use pretty_assertions::assert_eq;
|
|
use schemars::schema_for;
|
|
|
|
use super::*;
|
|
use crate::{
|
|
backends::plonky2::{
|
|
mainpod::{rec_main_pod_circuit_data, Prover},
|
|
mock::mainpod::MockProver,
|
|
primitives::ec::schnorr::SecretKey,
|
|
signer::Signer,
|
|
},
|
|
examples::{
|
|
attest_eth_friend, zu_kyc_pod_builder, zu_kyc_sign_dict_builders, EthDosHelper,
|
|
MOCK_VD_SET,
|
|
},
|
|
frontend::{Result, SignedDict, SignedDictBuilder},
|
|
middleware::{
|
|
self,
|
|
containers::{Array, Dictionary, Set},
|
|
Params, Signer as _, Value, DEFAULT_VD_LIST,
|
|
},
|
|
};
|
|
|
|
#[test]
|
|
fn test_value_serialization() {
|
|
// Pairs of values and their expected serialized representations
|
|
let values = vec![
|
|
(Value::from("hello"), "\"hello\""),
|
|
(Value::from(42), "{\"Int\":\"42\"}"),
|
|
(Value::from(true), r#"{"Int":"1"}"#),
|
|
(
|
|
Value::from(Array::new(vec![Value::from("foo"), Value::from(false)])),
|
|
r#"{"inner":[[{"Int":"0"},"foo"],[{"Int":"1"},{"Int":"0"}]]}"#,
|
|
),
|
|
(
|
|
Value::from(Dictionary::new(HashMap::from([
|
|
// The set of valid keys is equal to the set of valid JSON keys
|
|
("foo".into(), 123.into()),
|
|
// Empty strings are valid JSON keys
|
|
(("".into()), "baz".into()),
|
|
// Keys can contain whitespace
|
|
((" hi".into()), false.into()),
|
|
// Keys can contain special characters
|
|
(("!@£$%^&&*()".into()), "".into()),
|
|
// Keys can contain _very_ special characters
|
|
(("\0".into()), "".into()),
|
|
// Keys can contain emojis
|
|
(("🥳".into()), "party time!".into()),
|
|
]))),
|
|
r#"{"inner":[["!@£$%^&&*()",""],["🥳","party time!"],[" hi",{"Int":"0"}],["foo",{"Int":"123"}],["\u0000",""],["","baz"]]}"#,
|
|
),
|
|
(
|
|
Value::from(Set::new(HashSet::from(["foo".into(), "bar".into()]))),
|
|
r#"{"inner":[["bar"],["foo"]]}"#,
|
|
),
|
|
];
|
|
|
|
for (value, expected) in values {
|
|
let serialized = serde_json::to_string(&value).unwrap();
|
|
assert_eq!(serialized, expected);
|
|
let deserialized: Value = serde_json::from_str(&serialized).unwrap();
|
|
assert_eq!(
|
|
value, deserialized,
|
|
"value {:#?} should equal deserialized {:#?}",
|
|
value, deserialized
|
|
);
|
|
let expected_deserialized: Value = serde_json::from_str(expected).unwrap();
|
|
assert_eq!(value, expected_deserialized);
|
|
}
|
|
}
|
|
|
|
fn signed_dict_builder() -> SignedDictBuilder {
|
|
let params = &Params::default();
|
|
let mut builder = SignedDictBuilder::new(params);
|
|
builder.insert("name", "test");
|
|
builder.insert("age", 30);
|
|
builder.insert("very_large_int", 1152921504606846976);
|
|
builder.insert(
|
|
"a_dict_containing_one_key",
|
|
Dictionary::new(HashMap::from([
|
|
("foo".into(), 123.into()),
|
|
(
|
|
"an_array_containing_three_ints".into(),
|
|
Array::new(vec![1.into(), 2.into(), 3.into()]).into(),
|
|
),
|
|
(
|
|
"a_set_containing_two_strings".into(),
|
|
Set::new(HashSet::from([
|
|
Array::new(vec!["foo".into(), "bar".into()]).into(),
|
|
"baz".into(),
|
|
]))
|
|
.into(),
|
|
),
|
|
])),
|
|
);
|
|
builder
|
|
}
|
|
|
|
#[test]
|
|
fn test_signed_dict_serialization() {
|
|
let builder = signed_dict_builder();
|
|
let signer = Signer(SecretKey(1u32.into()));
|
|
let signed_dict = builder.sign(&signer).unwrap();
|
|
|
|
let serialized = serde_json::to_string_pretty(&signed_dict).unwrap();
|
|
println!("serialized: {}", serialized);
|
|
let deserialized: SignedDict = serde_json::from_str(&serialized).unwrap();
|
|
println!(
|
|
"deserialized: {}",
|
|
serde_json::to_string_pretty(&deserialized).unwrap()
|
|
);
|
|
assert_eq!(
|
|
signed_dict.dict.dump().unwrap(),
|
|
deserialized.dict.dump().unwrap()
|
|
);
|
|
assert_eq!(signed_dict.public_key, deserialized.public_key);
|
|
assert_eq!(signed_dict.signature, deserialized.signature);
|
|
assert_eq!(signed_dict.verify().is_ok(), deserialized.verify().is_ok());
|
|
}
|
|
|
|
fn build_mock_zukyc_pod() -> Result<MainPod> {
|
|
let params = middleware::Params::default();
|
|
let vd_set = &*MOCK_VD_SET;
|
|
|
|
let (gov_id_builder, pay_stub_builder) = zu_kyc_sign_dict_builders(¶ms);
|
|
let signer = Signer(SecretKey(1u32.into()));
|
|
let gov_id_pod = gov_id_builder.sign(&signer).unwrap();
|
|
let signer = Signer(SecretKey(2u32.into()));
|
|
let pay_stub_pod = pay_stub_builder.sign(&signer).unwrap();
|
|
let kyc_builder = zu_kyc_pod_builder(¶ms, vd_set, &gov_id_pod, &pay_stub_pod).unwrap();
|
|
|
|
let prover = MockProver {};
|
|
let kyc_pod = kyc_builder.prove(&prover).unwrap();
|
|
Ok(kyc_pod)
|
|
}
|
|
|
|
fn build_plonky2_zukyc_pod() -> Result<MainPod> {
|
|
let params = middleware::Params {
|
|
// Currently the circuit uses random access that only supports vectors of length 64.
|
|
// With max_input_main_pods=3 we need random access to a vector of length 73.
|
|
max_input_pods: 1,
|
|
..Default::default()
|
|
};
|
|
let mut vds = DEFAULT_VD_LIST.clone();
|
|
vds.push(rec_main_pod_circuit_data(¶ms).1.verifier_only.clone());
|
|
let vd_set = VDSet::new(&vds);
|
|
|
|
let (gov_id_builder, pay_stub_builder) = zu_kyc_sign_dict_builders(¶ms);
|
|
let signer = Signer(SecretKey(1u32.into()));
|
|
let gov_id_pod = gov_id_builder.sign(&signer)?;
|
|
let signer = Signer(SecretKey(2u32.into()));
|
|
let pay_stub_pod = pay_stub_builder.sign(&signer)?;
|
|
let _signer = Signer(SecretKey(3u32.into()));
|
|
let kyc_builder = zu_kyc_pod_builder(¶ms, &vd_set, &gov_id_pod, &pay_stub_pod)?;
|
|
|
|
let prover = Prover {};
|
|
let kyc_pod = kyc_builder.prove(&prover)?;
|
|
|
|
Ok(kyc_pod)
|
|
}
|
|
|
|
#[test]
|
|
fn test_mock_main_pod_serialization() -> Result<()> {
|
|
let kyc_pod = build_mock_zukyc_pod()?;
|
|
let serialized = serde_json::to_string_pretty(&kyc_pod).unwrap();
|
|
println!("serialized: {}", serialized);
|
|
let deserialized: MainPod = serde_json::from_str(&serialized).unwrap();
|
|
|
|
assert_eq!(kyc_pod.public_statements, deserialized.public_statements);
|
|
assert_eq!(
|
|
kyc_pod.pod.statements_hash(),
|
|
deserialized.pod.statements_hash()
|
|
);
|
|
assert_eq!(kyc_pod.pod.verify()?, deserialized.pod.verify()?);
|
|
|
|
Ok(())
|
|
}
|
|
|
|
#[test]
|
|
fn test_plonky2_main_pod_serialization() -> Result<()> {
|
|
let kyc_pod = build_plonky2_zukyc_pod()?;
|
|
let serialized = serde_json::to_string_pretty(&kyc_pod).unwrap();
|
|
let deserialized: MainPod = serde_json::from_str(&serialized).unwrap();
|
|
|
|
assert_eq!(kyc_pod.public_statements, deserialized.public_statements);
|
|
assert_eq!(
|
|
kyc_pod.pod.statements_hash(),
|
|
deserialized.pod.statements_hash()
|
|
);
|
|
assert_eq!(kyc_pod.pod.verify()?, deserialized.pod.verify()?);
|
|
|
|
Ok(())
|
|
}
|
|
|
|
fn build_ethdos_pod() -> Result<MainPod> {
|
|
let params = Params {
|
|
max_input_pods_public_statements: 8,
|
|
max_statements: 24,
|
|
max_public_statements: 8,
|
|
..Default::default()
|
|
};
|
|
let vd_set = &*MOCK_VD_SET;
|
|
|
|
let alice = Signer(SecretKey(1u32.into()));
|
|
let bob = Signer(SecretKey(2u32.into()));
|
|
let charlie = Signer(SecretKey(3u32.into()));
|
|
|
|
// Alice attests that she is ETH friends with Bob and Bob
|
|
// attests that he is ETH friends with Charlie.
|
|
let alice_attestation = attest_eth_friend(¶ms, &alice, bob.public_key());
|
|
let bob_attestation = attest_eth_friend(¶ms, &bob, charlie.public_key());
|
|
|
|
let helper = EthDosHelper::new(¶ms, vd_set, alice.public_key())?;
|
|
let prover = MockProver {};
|
|
let dist_1 = helper.dist_1(&alice_attestation)?.prove(&prover)?;
|
|
let dist_2 = helper
|
|
.dist_n_plus_1(&dist_1, &bob_attestation)?
|
|
.prove(&prover)?;
|
|
|
|
Ok(dist_2)
|
|
}
|
|
|
|
#[test]
|
|
// This tests that we can generate JSON Schemas for the MainPod and
|
|
// SignedDict types, and that we can validate Signed and Main Pods
|
|
// against the schemas. Since both Mock and Plonky2 PODs have the same
|
|
// public interface, we can assume that the schema works for both.
|
|
fn test_schema() {
|
|
let mainpod_schema = schema_for!(SerializedMainPod);
|
|
let signeddict_schema = schema_for!(SignedDict);
|
|
|
|
let kyc_pod = build_mock_zukyc_pod().unwrap();
|
|
let signed_dict = signed_dict_builder()
|
|
.sign(&Signer(SecretKey(1u32.into())))
|
|
.unwrap();
|
|
let ethdos_pod = build_ethdos_pod().unwrap();
|
|
let mainpod_schema_value = serde_json::to_value(&mainpod_schema).unwrap();
|
|
let signed_dict_schema_value = serde_json::to_value(&signeddict_schema).unwrap();
|
|
|
|
let kyc_pod_value = serde_json::to_value(&kyc_pod).unwrap();
|
|
let mainpod_valid = jsonschema::validate(&mainpod_schema_value, &kyc_pod_value);
|
|
assert!(mainpod_valid.is_ok(), "{:#?}", mainpod_valid);
|
|
|
|
let signed_dict_value = serde_json::to_value(&signed_dict).unwrap();
|
|
let signed_dict_valid = jsonschema::validate(&signed_dict_schema_value, &signed_dict_value);
|
|
assert!(signed_dict_valid.is_ok(), "{:#?}", signed_dict_valid);
|
|
|
|
let ethdos_pod_value = serde_json::to_value(ðdos_pod).unwrap();
|
|
let ethdos_pod_valid = jsonschema::validate(&mainpod_schema_value, ðdos_pod_value);
|
|
assert!(ethdos_pod_valid.is_ok(), "{:#?}", ethdos_pod_valid);
|
|
}
|
|
}
|