* Basic frontend AST and semantic validation
* Intro statement support
* Simplify validator lifetime
* Fix arity validation
* Lowering and splitting
* Remove legacy processor and use frontend AST by default
* Use builders instead of creating middleware types directly
* Typos/formatting
* Improve error messages when overflowing a batch due to splitting
* Add FromStr implementation for NativePredicate
* Remove 'raw' fields, and switch HashHex representation to byte vector rather than string
* Simpler wrapper types for batch and intro predicate hashes
* Parse secret and public keys to their respective data structures earlier
* More detail around string escape validity
* Simplify native predicate arity handling and move method to NativePredicate impl
* Store hashes using middleware::Hash, and simplify lowering by using pre-parsed values
* Simplify predicate building
* Formatting
* Better error messages/suggestions for cases where predicate splitting fails
* Formatting
* Clippy fix
* Return error if we get a too-large int
- middleware:
- Add `Statement::Intro`
- Add `SignedBy` native predicate and operation. The signature is auxiliary data to the operation
- Rename `PodSigner` to `Signer` with a new API (just for signing `RawValue`)
- Removed `NewEntry` operation. Use `ContainsFromEntries` instead
- Remove `KEY_SIGNER` and `KEY_TYPE` which are no longer used
- Merge `RecursivePod` and `Pod` traits
- Change the `Pod::deserialize_data` method to use `Self` instead of `Box<dyn Pod>`
- Extend `Pod` trait with these methods:
- `is_main`: when the pod is Main, in a (recursive) verification its vk will be checked to exist in the vd_set but not if it's intro pod
- `is_mock`: skip some verifications in the recursive mock MainPod verification
- `verifier_data_hash`
- `pod_id` renamed to `statements_hash`
- AnchoredKeys are now a pair of dictionary root and key
- Entry statements are now defined as Contains with literal arguments
- Operations that take Entries now use Contains statements with literal arguments
- frontend:
- Rename `SignedPod` to `SignedDict` (which now contains the dict, public key and signature, and can still `verify(self)`ed)
- The `SignedDict` keeps the method `get_statement` for convenience but now it returns a `Contains` statement that proves the existence of the key in the dict
- The `MainPodBuilder` automatically inserts a `Contains` statement when an operation is added that uses an entry as argument that was not yet "opened".
- Removed the `literal` methods from the `MainPodBuilder` that were loading literals to anchored keys: that was no longer needed after we introduced literal arguments
- backend
- Only verify inclusion of the verifying key into the vd_set if the pod is MainPod. A pod is not MainPod if the first statement is Intro.
- Reject intro pods that have non-intro statements
- Empty pod now returns an intro statement
- Don't insert a type statement automatically in MainPod and MockMainPod. We get rid of the type entry.
- Implement `SignedBy` operation, which uses the muxed table to store signature verifications
- Rename `PodId` to `statements_hash` or `sts_hash` for short. Now this is only used as a hash of the statements for the circuits public inputs.
- Refactor normalization of `self` statements:
- Before: replace values that contain `SELF` by the given pod_id
- After: place the verifying key hash into the Intro predicates
* Add container update ops
* Update src/middleware/operation.rs
Co-authored-by: Eduard S. <eduardsanou@posteo.net>
* Update src/backends/plonky2/mainpod/mod.rs
Co-authored-by: Eduard S. <eduardsanou@posteo.net>
* Code review
---------
Co-authored-by: Eduard S. <eduardsanou@posteo.net>
Support arrays up to 256 elements (hardcoded maximum just to avoid abuse) by combining multiple random_accesses.
The index is now split into low and high parts. It's a bit more inconvenient than using a single Target but this allows avoiding bit decomposition.
* wrote some initial code
* added way to input private key into circuit
* TypedValue::SecretKey hashed as 10 32-bit limbs
* Check PublicKeyOf in Frontend and Middleware
* Diff review
* PR review
* Finish utest
* Fix bounds check
* added giving secret key witness to circuit
* Test & doc improvements
* added private key comparison to circuit and added test cases
* cargo fmt
* Add frontend tests for PublicKeyOf
* Add public_key_of and hash_of to op! macro
* Add ownership check to ticket example
* Group order checking in tests
* More negative test cases at circuit level
* Cleanups after self review
* clippy fixes
* Fixes after merge. Temporarily remove plonky2 commit hash
* Add a nullifier to the ticket test example
* Test PublicKeyOf with a real prover (not mock)
* plonky-u32 dependency
* feat: optimize operation checks
Skip the circuits that verify operation checks other than None, Copy or
NewEntry for the public statements. This works because public
statements are created by copying private statements, so we never use
the other operation checks in those slots.
---------
Co-authored-by: Andrew Twyman <artwyman@gmail.com>
Co-authored-by: Eduard S. <eduardsanou@posteo.net>
This PR is a continuation of the work done in #276
- Fix PodType in MainPod (we were using `MockMain` instead of `Main`)
- Update anchored keys in statement template arguments to only support wildcards in the origin and literal keys as the key.
- Update the pest grammar accordingly
- Update the parser accordingly
- Rewrite the eth_dos example in a recursive manner so that we use one recursive pod for every distance increment of 1.
- I've also used the podlang to define the eth_dos custom predicates. Currently all predicates are in a single batch (previously `eth_friend` was in a different batch). With #286 we could define `eth_friend` in a different batch again.
- I was feeling a bit creative and used a format macro to pass `Value`s from rust to the podlang code.
- The eth_dos is now written using literals. This resolves https://github.com/0xPARC/pod2/issues/255
- Remove `StatementArg::WildcardValue` in favor of `StatementArg::Literal`. The `WildcardValue` was just a way to have some kind of typing for values that would be used as arguments in custom predicates. Now that we can have literals in any statement this value can be anything, so I just removed the `WildcardValue` and use `Literal` instead. On the backend it was already the case that both cases were treated the same way (after all, `WildcardValue` and `Literal` were 4 fields in the backend).
- Added a new type for Value: `PodId` so that we can use it for custom predicates that take a pod id to be used in a wildcard
- Add a mock vd_set that is empty for tests that don't use plonky2; this allows running those tests individually without paying for the expensive work of calculating the vd for various circuits.
- rename StatementTmplArg::WildcardValue to StatementTmplArg::Wildcard
* Implement Gt and GtEq as syntactic sugar
* Update src/backends/plonky2/circuits/mainpod.rs
Co-authored-by: Eduard S. <eduardsanou@posteo.net>
* Op verification circuit refactor
* Code review
* Add range check to Eq case of LtEq
* Style
* Factor out ValueOf statement argument type checks
* Formatting
* Clean-up
* Safety
* Take sign into account
* Simplify sign check
---------
Co-authored-by: Eduard S. <eduardsanou@posteo.net>
* migrate from anyhow to thiserror (#190). pending polish error msgs
* Add backtrace and compartmentalize errors
- Include backtraces in the errors we generate. To get this we can't
just return a literal enum, because the backtrace requires a call.
- Related to the previous point: add methods to create errors so
we can include the backtrace conveniently without changing too much
the syntax. So instead of `Err(Error::KeyNotFound(key))` (literal
enum) it will be `Err(Error::key_not_found(key))` (method call)
- Each error should be local to its scope, and each scope should
only return its own error.
- The merkle tree should return `TreeError` and not Error
- The middleware should return `MiddlewareError` and not Error
- With a global Error we can't easily include backend/frontend types in
the error fields, so declare a `BackendError` and a `FrontendError`
and follow the pattern from the previous point
- The Pod traits should be able to return backend errors and will be
used in the frontend; for that we change them to use trait object
Error: `dyn std::error::Error`
* fix error
* apply suggestions from @arnaucube
* rename XError and XResult to Error and Result
* reorg signature
* make frontend custom error more ergonomic
* remove unnecessary feature
---------
Co-authored-by: Eduard S. <eduardsanou@posteo.net>
* Serialization tests now pass again
* Tidy up and test more edge-cases
* Use attributes rather than custom serializer for arrays
* Add JSON Schema support
* Tests for JSON Schema generation and validation
* Add comments
* Support custom predicates
* Clippy fixes
* Make deserialization/constructor functions pub(crate)
* unify fe/be NativeOp and NativePred
* remove Origin in favour of PodId
* Combine string and hash in Key
* use middleware::AnchoredKey in frontend
* merge frontend/middleware types
* refactor custom predicates
* clean up a bit
* fix middleware custom tests
* clean up
* clean up 2
* add acronyms in typos list
* All test pass on middleware->frontend type refactor
* Convert frontend CustomPredicateRef to a named field struct
* Minor serialization improvements
* Set appropriate titles in JSON schemas
* Add names for custom predicates
* Remove PodClass from front-end Origin type
* Simplify value conversion
---------
Co-authored-by: Ahmad <root@ahmadafuni.com>
* Contains should take three arguments (root, key, value)
* Add a test for frontend Dictionaries
* Separate frontend and middleware operations
* Make tests pass: add arg to contains
* Cargo fmt
* Merkleproof verify circuit (#143)
* merkletree: add keypath circuit
* merkletree-circuit: implement proof of existence verification in-circuit
* parametrize max_depth at the tree circuit
* Constrain selectors in-circuit
* implement merketree nonexistence proof circuit, and add edgecase tests
* add non-existence proofs documentation in the mdbook, mv EMPTY->EMPTY_VALUE & NULL->EMPTY_HASH, dependency clean and public exposure methods
* review comments, some extra polishing and add a test that expects wrong proofs to fail
* Add circuit to check only merkleproofs-of-existence
With this, the merkletree_circuit module offers two different circuits:
- `MerkleProofCircuit`: allows to verify both proofs of existence and proofs
non-existence with the same circuit.
- `MerkleProofExistenceCircuit`: allows to verify proofs of existence only.
In this way, if only proofs of existence are needed,
`MerkleProofExistenceCircuit` should be used, which requires less amount
of constraints than `MerkleProofCircuit`.
* Code review
---------
Co-authored-by: Ahmad <root@ahmadafuni.com>
* Towards Contains/NotContains in middleware and backend
* Fix build
* Adding error handling to deal with op compile introduce extra ops
* Incorporate Merkle proofs into MockMainPod
* Merkleproof verify circuit (#143)
* merkletree: add keypath circuit
* merkletree-circuit: implement proof of existence verification in-circuit
* parametrize max_depth at the tree circuit
* Constrain selectors in-circuit
* implement merketree nonexistence proof circuit, and add edgecase tests
* add non-existence proofs documentation in the mdbook, mv EMPTY->EMPTY_VALUE & NULL->EMPTY_HASH, dependency clean and public exposure methods
* review comments, some extra polishing and add a test that expects wrong proofs to fail
* Add circuit to check only merkleproofs-of-existence
With this, the merkletree_circuit module offers two different circuits:
- `MerkleProofCircuit`: allows to verify both proofs of existence and proofs
non-existence with the same circuit.
- `MerkleProofExistenceCircuit`: allows to verify proofs of existence only.
In this way, if only proofs of existence are needed,
`MerkleProofExistenceCircuit` should be used, which requires less amount
of constraints than `MerkleProofCircuit`.
* Code review
---------
Co-authored-by: Ahmad <root@ahmadafuni.com>
* Towards Contains/NotContains in middleware and backend
* Frontend compound types -- allow one frontend operation to produce multiple middleware statements (in progress)
* Incorporate Merkle proofs into MockMainPod
* Incorporate Merkle proof op arg into frontend
* Compile one statement to many, in progress
* Fix remaining tests
* Minor clean-up
* Oops I did a bunch of work in the middle of a rebase, committing
* Incorporate Merkle proof op arg into frontend
* still working on frontend compound types, refactor compile() to output multiple statements
* Contains statements for frontend types: code compiles
* Tests pass
* Examples use front-end compound types
* Remove old Contains and NotContains from frontend
* Add nin to typos
* Code review
---------
Co-authored-by: arnaucube <git@arnaucube.com>
Co-authored-by: Ahmad <root@ahmadafuni.com>
At the middleware we were defining some types that actually are dependant on the
backend no matter how we define them in the middleware.
For example, we were hardcoding the `Hash` and `Value` types and their related
behaviour (eg. `.to_fields()`) to be based on the length of 4 field elements,
but that's not a choice of the middleware, and in fact this is determined by the
backend itself. On the same time, those types and related methods do not belong
to the backend, since conceptually they are part of the middleware reasoning.
The intention of this PR is not to prematurely abstract the library, but to
avoid inconsistencies where a type or parameter is defined in the middleware to
have certain carachteristic and later in the backend it gets used differently.
The idea is that those types and parameters (eg. lengths) have a single source
of truth in the code; and in the case of the "base types" (hash, value, etc)
this is determined by the backend being used under the hood, not by a choice of
the middleware parameters.
The idea with this approach, is that the frontend & middleware should not need
to import the proving library used by the backend (eg. plonky2, plonky3, etc).
As mentioned earlier, the `Hash` and `Value` types are types belonging at the
middleware, and is the middleware who reasons about them, but depending on the
backend being used, the `Hash` and `Value` types will have different sizes. So
it's the backend being used who actually defines their nature under the hood.
For example with a plonky2 backend, these types will have a length of 4 field
elements, whereas with a plonky3 backend they will have a length of 8 field
eleements.
Note that his approach does not introduce new traits or abstract code, just
makes use of rust features to define 'base types' that are being used in the
middleware.
* Print pods from SignedPodBuilder
* Add additional print to test printing SignedPodBuilder
* Mock-prove and print MainPod
* Implement ToFields for custom predicates and dependencies
* Test: print serialization of a recursive batch
* Rearrange serialization of CustomPredicate so args_len is always in the same position
* Serialize predicates with first entry nonzero to avoid collision with padding
* Off by one error in ethdos test BatchSelf(2)
* cargo fmt
* not a typo
* Typos, trying again
* Experiment with statement & op enums
* Clean-up & fixes
* More clean-up
* Add argument length checks
* More clean-up
* Place statement and operation logic in submodules
