- Bump rust version to `nightly-2025-07-02` because some of the nightly features we were using have been stabilized.
- Introduce feature `disk_cache` which enables caching to disk. Each time an artifact is retrieved from the cache it will be read and deserialized. On a cache miss the artifact will be created, serialized and stored to disk.
- Introduce feature `mem_cache` which enables caching to memory. All cached artifacts are kept in memory after they are created. The mem cache implementation avoids cloning of artifacts by extending their lifetime to `'static`. This is `unsafe` code, but I argue that this usage is safe.
- Add a `build.rs`
- When the feature `disk_cache` is enabled, the `build.rs` will inject env variables to the process with the git commit information, which is used to index the cached artifacts
- Replace all previous cached artifacts from `LazyStatic` methods that call the cache API
- Derive `Serialize, Deserialize` for all `*Target` types so that they can be serialized for caching to disk
- Add finer level of caching: now we cache the `CircuitData` and `VerifierData` independently. The reason for this is that `CircuitData` is a very big artifact which is not needed for verification. So by only accessing `VerifierData` in verification we don't pay a big overhead for reading from disk and deserializing
- Add missing artifacts to the cache: like the `CircuitData` for the `MainPod` indexed by `Params`
- Add helper types to serialize and deserialize `CircuitData`, `CommonData` and `VerifierData` with the set of gates and generators used in the recursive MainPod circuit
- Tweak the ids of our custom gates so that they remain unique when their generic parameters change
- Bugfix: several tests were using the standard `vd_set` but were using MainPod circuits with non-default parameters. This was working before because there was a bug: the MainPod circuit was reporting that the used verifier data was the standard one instead of picking the one corresponding to it's own Params.
Summary of breaking changes:
- One and only one of the features `mem_cache` or `disk_cache` need to be enabled. By default it's `mem_cache`
- To enable the `disk_cache` you need to disable the default features like this: `--no-default-features --features=backend_plonky2,zk,disk_cache`
- Removed `DEFAULT_PARAMS`, instead use `Params::default()`
- Removed `STANDARD_REC_MAIN_POD_CIRCUIT_DATA`, instead use `cache_get_standard_rec_main_pod_common_circuit_data`
- The library is now using `nightly-2025-07-02`. Some rust language features are unstable in previous versions.
* calculate MainPod id in a dynamic-friendly way
The MainPod id is now calculated with front padding and a fixed size
independent of max_public_statements so that introduction gadgets can be
verified by a MainPod while paying only for the number of statements
they use. This is because with front padding of none-statements we can
precompute the poseidon state corresponding to absorbing all the padding
statements and only pay constraints for the non-padding statements.
The id is calculated as follows:
`id = hash(serialize(reverse(statements || none-statements)))`
* add time feature and disable timing by default
* apply suggestions from @arnaucube
* link issues in todos
* Add RecursiveCircuit
The RecursiveCircuit verifies N proofs of itself (N=arity), together with
the logic defined at the InnerCircuit (in our case, used for the
MainPodCircuit logic).
The arity defines the maximum amount of proofs of itself that the
RecursiveCircuit verifies. When arity>1, using the RecursiveCircuit has the
shape of a tree of the same arity.
π_root
▲
┌───────┴────────┐
│RecursiveCircuit│
└─▲───▲───▲────▲─┘
┌───────┘ ┌┘ └┐ └──────┐
│π''_1 │ ... │ π''_N│
┌────────┴───────┐ ┌┴┐┌─┐┌┴┐ ┌───────┴────────┐
│RecursiveCircuit│ │.││.││.│ │RecursiveCircuit│
└──▲─────────▲───┘ └─┘└─┘└─┘ └──▲─────────▲───┘
│ │ │ │
π_1 ... π_N π'_1 ... π'_N
where
N: arity of the RecursiveCircuit
π_i: plonky2 proof of the RecursiveCircuit
* add different inner-circuits in the test, reusing the same verifier_data; polish recursion interfaces
* add InnerCircuit::Params
* rm non_base_node
* WIP refactor RecursiveCircuit
* wip. change approach on verifier_data of circuits used in recursivecircuit. arity=1 works
* recursion works fine without registering verifierdatas as publicinputs
* add hashing of verifier_data (out & in-circuit) methods with test
* connect previous and current verifier_datas hashes
* polish
* add InnerCircuit's public_inputs to the RecursiveCircuit
* extend recursive test
* polish & review suggestions
* migrate from anyhow to thiserror (#190). pending polish error msgs
* Add backtrace and compartmentalize errors
- Include backtraces in the errors we generate. To get this we can't
just return a literal enum, because the backtrace requires a call.
- Related to the previous point: add methods to create errors so
we can include the backtrace conveniently without changing too much
the syntax. So instead of `Err(Error::KeyNotFound(key))` (literal
enum) it will be `Err(Error::key_not_found(key))` (method call)
- Each error should be local to its scope, and each scope should
only return its own error.
- The merkle tree should return `TreeError` and not Error
- The middleware should return `MiddlewareError` and not Error
- With a global Error we can't easily include backend/frontend types in
the error fields, so declare a `BackendError` and a `FrontendError`
and follow the pattern from the previous point
- The Pod traits should be able to return backend errors and will be
used in the frontend; for that we change them to use trait object
Error: `dyn std::error::Error`
* fix error
* apply suggestions from @arnaucube
* rename XError and XResult to Error and Result
* reorg signature
* make frontend custom error more ergonomic
* remove unnecessary feature
---------
Co-authored-by: Eduard S. <eduardsanou@posteo.net>
* feat: add MainPod circuit skeleton
* feat: use ValueTarget in mt, verify SignedPod type
* wip
* feat: match structure with mock
* apply feedback from @arnaucube
* add 2 operations
* fix test compilation
* Add missing todo
* sync spec & code
* move primitives (merkletree) into the backend
* comment on the ops spec and link to issue #108
* typo
* fix github-ci mdbook-publish pages
At the middleware we were defining some types that actually are dependant on the
backend no matter how we define them in the middleware.
For example, we were hardcoding the `Hash` and `Value` types and their related
behaviour (eg. `.to_fields()`) to be based on the length of 4 field elements,
but that's not a choice of the middleware, and in fact this is determined by the
backend itself. On the same time, those types and related methods do not belong
to the backend, since conceptually they are part of the middleware reasoning.
The intention of this PR is not to prematurely abstract the library, but to
avoid inconsistencies where a type or parameter is defined in the middleware to
have certain carachteristic and later in the backend it gets used differently.
The idea is that those types and parameters (eg. lengths) have a single source
of truth in the code; and in the case of the "base types" (hash, value, etc)
this is determined by the backend being used under the hood, not by a choice of
the middleware parameters.
The idea with this approach, is that the frontend & middleware should not need
to import the proving library used by the backend (eg. plonky2, plonky3, etc).
As mentioned earlier, the `Hash` and `Value` types are types belonging at the
middleware, and is the middleware who reasons about them, but depending on the
backend being used, the `Hash` and `Value` types will have different sizes. So
it's the backend being used who actually defines their nature under the hood.
For example with a plonky2 backend, these types will have a length of 4 field
elements, whereas with a plonky3 backend they will have a length of 8 field
eleements.
Note that his approach does not introduce new traits or abstract code, just
makes use of rust features to define 'base types' that are being used in the
middleware.
