Extend the work of https://github.com/0xPARC/pod2/pull/487 to the Containers (Dictionary, Set, Array).
The merkle tree only stores `RawValue` for both the key and the value, so it is the responsibility of the Container to store the rich value.
In order to handle containers with persistent storage efficiently (which means, cloning them or updating them should not cause an O(n) data copy) I figured we need to have a database of `Value`s indexed by their raw value; as this gives us deduplication and free cloning of containers.
The issue with this approach is that in the current design we have collisions between Value's of different types: https://github.com/0xPARC/pod2/issues/426 and the current API relies on the single type of values.
To resolve this issue I decided to change the API, instead of assuming that a Value has a fixed type, let the value be possibly multiple compatible types and let the user of the library try casting the Value to a particular type.
For this I deprecated the public access of everything related to `TypedValue` and I propose for it to be considered an implementation detail and a blackbox from the external developer point of view. The `Value` type is now used like this:
- To create a new Value use `Value::from(...)` where you can pass any compatible type (the same types as before)
- To access the Value in typed form you cast it like `value.as_foo()` which returns `Option<Foo>`.
Previously we had a collision between `true` and `1` (and `false` and `0`). Now it doesn't matter whether a value holds a `true` or a `1`, both should be seen as the same and both return `Some` when doing `as_int` and `as_bool`.
Similarly we had collisions with containers. For example `set(0, 1, 2) == array[0, 1, 2]` and `set("a", "b") = dict("a": "a", "b": "b")`. Now any container can be casted to any of `set, array, dict`. There's a caveat here: each of these types expects a particular encoding of keys, so casting to the wrong type will return errors on some operations.
With this design it no longer matters what is being stored and recovered because the API requires the user to express the expected type and any type with collisions for particular values can be casted to the right type.
There's only one case where it's not desirable to swap one `TypedValue` for another: the `TypedValue::Raw`. If a non-`RawValue` in the DB is replaced by the corresponding `RawValue` we erase the required information to recover the rich value. For this reason the implementations of the database treat the `RawValue` as a special case: if an value is stored in non-`RawValue`, the corresponding `RawValue` can never overwrite it. If a value is stored in `RawValue`, a matching non-`RawValue` will overwrite it (promoting it to a rich value). This way we never lose data.
A consequence of this is that the serialization, `Display` and `Debug` of a container is not stable. At any point any of the entries can be swapped for a "compatible" one if they share the storage with other containers that introduce collisions.
I rewrote all containers as wrapper to a generic `Container` which holds a `Map` from `Value` to `Value`. The serialization of each container now uses the single implementation of the generic `Container`.
This simplifies the MerkleTree (and container) API.
Defer the max depth check when assigning the witness (merkle proof siblings) to the merkle tree circuit.
In this implementation the native Merkle Tree branches grow as much as they needed. There are no checks of max depth in the merkle tree. All keys are 256 bits (I added a debug_assert for this); so in the worst case a path will have depth 256. It can't have a longer depth because the `insert` method calls `prove_nonexistence` which errors if the key already exists; another one may exist which must be different and thus require a path <= 256 depth.
Resolve#436
* implement merkletree insert & insert-proof-verification
* add merkletree circuit to verify insertion proof
wip
* fix merkletree's GraphViz generation for cases with empty siblings
* implement tree insert-verif circuit siblings checks
Note: I've implemented also an alternative version which instead of
inputting a witness value 'divergence_level' it inputs a bitmask. Both
approaches (divergence_level and divergence_bitmask) take the same
amount of constraints (336 constraints for a tree of 32 levels, and for
an hybrid approach it takes 331 constraints but the code gets a bit less
readable). So I've kept with the current implementation (using
divergence_level) which is more easy to follow.
* [tree] modify the strategy for the insert-proof (out-circuit)
* re-implement insert-proof verification circuit
* add pending checks and polish
* add tests for disabled(&enabled) cases that should fail
* update typos.toml config
* Add test with tampering
* add check 5.3, to prevent tampering (at insertion proof circuit)
* move old_leaf_hash computation outside the loop, simplify check 5.3 booleans
* apply @ed255 review suggestions
---------
Co-authored-by: Ahmad <root@ahmadafuni.com>
* migrate from anyhow to thiserror (#190). pending polish error msgs
* Add backtrace and compartmentalize errors
- Include backtraces in the errors we generate. To get this we can't
just return a literal enum, because the backtrace requires a call.
- Related to the previous point: add methods to create errors so
we can include the backtrace conveniently without changing too much
the syntax. So instead of `Err(Error::KeyNotFound(key))` (literal
enum) it will be `Err(Error::key_not_found(key))` (method call)
- Each error should be local to its scope, and each scope should
only return its own error.
- The merkle tree should return `TreeError` and not Error
- The middleware should return `MiddlewareError` and not Error
- With a global Error we can't easily include backend/frontend types in
the error fields, so declare a `BackendError` and a `FrontendError`
and follow the pattern from the previous point
- The Pod traits should be able to return backend errors and will be
used in the frontend; for that we change them to use trait object
Error: `dyn std::error::Error`
* fix error
* apply suggestions from @arnaucube
* rename XError and XResult to Error and Result
* reorg signature
* make frontend custom error more ergonomic
* remove unnecessary feature
---------
Co-authored-by: Eduard S. <eduardsanou@posteo.net>
