Eduard S.
13cabdb511
Extend the work of https://github.com/0xPARC/pod2/pull/487 to the Containers (Dictionary, Set, Array). The merkle tree only stores `RawValue` for both the key and the value, so it is the responsibility of the Container to store the rich value. In order to handle containers with persistent storage efficiently (which means, cloning them or updating them should not cause an O(n) data copy) I figured we need to have a database of `Value`s indexed by their raw value; as this gives us deduplication and free cloning of containers. The issue with this approach is that in the current design we have collisions between Value's of different types: https://github.com/0xPARC/pod2/issues/426 and the current API relies on the single type of values. To resolve this issue I decided to change the API, instead of assuming that a Value has a fixed type, let the value be possibly multiple compatible types and let the user of the library try casting the Value to a particular type. For this I deprecated the public access of everything related to `TypedValue` and I propose for it to be considered an implementation detail and a blackbox from the external developer point of view. The `Value` type is now used like this: - To create a new Value use `Value::from(...)` where you can pass any compatible type (the same types as before) - To access the Value in typed form you cast it like `value.as_foo()` which returns `Option<Foo>`. Previously we had a collision between `true` and `1` (and `false` and `0`). Now it doesn't matter whether a value holds a `true` or a `1`, both should be seen as the same and both return `Some` when doing `as_int` and `as_bool`. Similarly we had collisions with containers. For example `set(0, 1, 2) == array[0, 1, 2]` and `set("a", "b") = dict("a": "a", "b": "b")`. Now any container can be casted to any of `set, array, dict`. There's a caveat here: each of these types expects a particular encoding of keys, so casting to the wrong type will return errors on some operations. With this design it no longer matters what is being stored and recovered because the API requires the user to express the expected type and any type with collisions for particular values can be casted to the right type. There's only one case where it's not desirable to swap one `TypedValue` for another: the `TypedValue::Raw`. If a non-`RawValue` in the DB is replaced by the corresponding `RawValue` we erase the required information to recover the rich value. For this reason the implementations of the database treat the `RawValue` as a special case: if an value is stored in non-`RawValue`, the corresponding `RawValue` can never overwrite it. If a value is stored in `RawValue`, a matching non-`RawValue` will overwrite it (promoting it to a rich value). This way we never lose data. A consequence of this is that the serialization, `Display` and `Debug` of a container is not stable. At any point any of the entries can be swapped for a "compatible" one if they share the storage with other containers that introduce collisions. I rewrote all containers as wrapper to a generic `Container` which holds a `Map` from `Value` to `Value`. The serialization of each container now uses the single implementation of the generic `Container`.
31 lines
915 B
YAML
31 lines
915 B
YAML
name: Rust Build with features
|
|
|
|
on:
|
|
pull_request:
|
|
branches: [ main ]
|
|
types: [ready_for_review, opened, synchronize, reopened]
|
|
push:
|
|
branches: [ main ]
|
|
|
|
jobs:
|
|
test:
|
|
if: github.event.pull_request.draft == false
|
|
name: Rust tests
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v3
|
|
- name: Set up Rust
|
|
uses: actions-rust-lang/setup-rust-toolchain@v1
|
|
- name: Build default
|
|
run: cargo build
|
|
- name: Build non-zk # check without the zk feature enabled
|
|
run: cargo build --no-default-features --features backend_plonky2,mem_cache
|
|
- name: Build metrics
|
|
run: cargo build --features metrics
|
|
- name: Build time
|
|
run: cargo build --features time
|
|
- name: Build db_rocksdb
|
|
run: cargo build --features db_rocksdb
|
|
- name: Build disk_cache
|
|
run: cargo build --no-default-features --features backend_plonky2,zk,disk_cache
|
|
|