test: reduce chain length for ci

remove workflow Update README.md Update README.md docs and refactor readme Update README.md Update README.md readme readme refactor docs, tests, error handling, type alias docs readme Update README.md Update README.md add testing workflow workflow workflow update Update README.md test: layer count docs workflows and readme workflow names Update README.md workflow badges add benches Update README.md Update README.md update TODO circuit telemetry and benches clippy doctest workflow telemetry Update README.md add RAM bench Update README.md circuit diagram Update README.md docs docs function names and docs
Dustin-Ray · Oct 17, 2024 · 1f102d8 · 1f102d8
1 parent bd05d26
commit 1f102d8
Show file tree

Hide file tree

Showing 10 changed files with 731 additions and 160 deletions.
diff --git a/.github/workflows/rust_test.yaml b/.github/workflows/rust_test.yaml
diff --git a/.github/workflows/test_keccak_hash_chain.yml b/.github/workflows/test_keccak_hash_chain.yml
@@ -0,0 +1,25 @@
+name: Test Keccak Hash
+
+on:
+  push:
+    branches:
+      - feat/keccak
+  pull_request:
+    branches:
+      - feat/keccak
+
+jobs:
+  build_and_test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions-rs/toolchain@v1
+        with:
+          toolchain: nightly
+          profile: minimal
+          override: true
+          components: rustfmt, clippy
+      - uses: actions-rs/cargo@v1
+        with:
+          command: test
+          args: --release
diff --git a/.github/workflows/test_poseidon_hash_chain.yml b/.github/workflows/test_poseidon_hash_chain.yml
@@ -0,0 +1,25 @@
+name: Test Poseidon Hash
+
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+    branches:
+      - main
+
+jobs:
+  build_and_test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions-rs/toolchain@v1
+        with:
+          toolchain: nightly
+          profile: minimal
+          override: true
+          components: rustfmt, clippy
+      - uses: actions-rs/cargo@v1
+        with:
+          command: test
+          args: --release
diff --git a/.vscode/settings.json b/.vscode/settings.json
@@ -0,0 +1,13 @@
+{
+    "git.openRepositoryInParentFolders": "never",
+    "git.confirmSync": false,
+    "editor.formatOnSave": true,
+    "workbench.startupEditor": "none",
+    "extensions.ignoreRecommendations": true,
+    "rust-analyzer.rustfmt.overrideCommand": [
+        "rustfmt",
+        "+nightly"
+    ],
+    "rust-analyzer.server.path": null,
+    "explorer.confirmDelete": false
+}
diff --git a/Cargo.lock b/Cargo.lock
diff --git a/Cargo.toml b/Cargo.toml
@@ -19,6 +19,8 @@ log = "0.4.20"
 sha3 = "0.10.6"
 structopt = { version = "0.3.26", default-features = false }
 env_logger = "0.11.5"
+thiserror = "1.0.63"
+criterion = "0.3"
 
 [dev-dependencies]
 debug_print = { version = "1.0.0" }
@@ -32,4 +34,9 @@ plonky2 = { git = "https://github.com/drcapybara/plonky2", branch = "upstream" }
 plonky2_crypto = { git = "https://github.com/Lagrange-Labs/plonky2-crypto", branch = "patch-plonky2" }
 
 [profile.dev]
-opt-level = 3
+opt-level = 3
+
+[[bench]]
+name = "hash_chain_bench"
+path = "benches/hash_chain_benches.rs"
+harness = false
diff --git a/README.md b/README.md
@@ -1,50 +1,108 @@
 # Cyclic Hash Computation with Plonky2
 
-Have you ever been sitting behind your desk one evening and thought to yourself, "Gee whiz, I sure wish I could cryptographically prove in quantum-resistant zero knowledge that I computed a gigantic chain of hashes correctly"? Well _good news_ because this repo will fill that burning hole in your heart with a plonky2-based implementation of a recursive computation chain that proves a series of hashes a la IVC style and outputs a compressed proof of constant size, arguing the integrity of a potentially unlimited size computation.
+Have you ever thought to yourself, "Gee, I sure wish I could cryptographically prove in quantum-resistant zero knowledge that I computed a gigantic chain of hashes correctly"? Well _good news_ because this repo will fill that burning hole in your heart with a plonky2-based implementation of a prover/verifier pair for a recursive computation chain that argues the computational integrity of a series of hashes a la IVC style and outputs a compressed proof of constant size. The recursive nature of this circuit gives desireable scaling characteristics, and this work presents and effort to better understand the unique design challenges presented when working with low-level circuitry.
+
+## Quick start:
+
+You can download the repo and run the main branch with:
+```bash
+cargo test
+```
+
+Which is run in release mode by default. This repo requires the nightly toolchain. If you are seeing errors related to:
+
+```bash
+6 | #![feature(specialization)]
+  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+error[E0554]: `#![feature]` may not be used on the stable release channel
+  |
+7 | #![cfg_attr(target_arch = "x86_64", feature(stdarch_x86_avx512))]
+  |                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+error[E0554]: `#![feature]` may not be used on the stable release channel
+  |
+7 | #![cfg_attr(target_arch = "x86_64", feature(stdarch_x86_avx512))]
+  |                                             ^^^^^^^^^^^^^^^^^^
+```
+
+Then please double check your toolchain. Otherwise, this repo should work out of the box.
+
+You can also run:
+
+```bash
+RUSTFLAGS="-Ctarget-cpu=native" cargo run --release --example circuit_telemetry -- -vv --steps 20
+```
+
+To quickly benchmark prover and verifer performance, as well as examine details about the chain over a given number of steps:
+
+```text
+[2024-08-23T08:01:38Z INFO  hash_chain] Number of gates in circuit: 112960
+[2024-08-23T08:01:46Z INFO  hash_chain] Total Proof length: 133440 bytes
+[2024-08-23T08:01:46Z INFO  circuit_telemetry] Proof time: 9.733192095s
+[2024-08-23T08:01:46Z INFO  circuit_telemetry] Verification time: 4.142599ms
+[2024-08-23T08:01:46Z INFO  circuit_telemetry] Circuit depth: 20
+```
+## Supported Hashes:
+
+The following hashes are available in the recursive chain:
+
+| Hasher | Validation |
+|----------|----------|
+| Poseidon Hash    | [![Test Poseidon Hash](https://github.com/drcapybara/hash-chain/actions/workflows/test_poseidon_hash_chain.yml/badge.svg?branch=main)](https://github.com/drcapybara/hash-chain/actions/workflows/test_poseidon_hash_chain.yml) |
+| Keccak    | [![Test Keccak Hash](https://github.com/drcapybara/hash-chain/actions/workflows/test_keccak_hash_chain.yml/badge.svg?branch=feat%2Fkeccak)](https://github.com/drcapybara/hash-chain/actions/workflows/test_keccak_hash_chain.yml) |
+
+The keccak chain fails to build successfully at current, work is ongoing to fix this eventually.
 
 # Strategy
 
 Our approach is to insert the following gates into the circuit with the requisite connections. It is not enough to create a circuit that simply connects each hash output the next input, the prover must argue the hash computation _and_ verify the preceeding hash in a single step, taking into account the recursive structure of the chain:
 
+```text
++--------------------------------+    +-------------------------+    +------------------------------+
+| 1. initialize_circuit_builder  |    | 2. setup_hashes         |    | 3. common_data_for_recursion |
+|    Set up the circuit builder  |──▶| Configure initial       |──▶| Set up data for recursion    |
+|     and configuration.         |    |  and current hash       |    | and verifier data inputs.    |
++--------------------------------+    | targets and register    |    +------------------------------+                            
+          |                           | them as public inputs.  |              |
+          |                           +-------------------------+              |
+          │                                                                    │
+          │            +--------------------+                                  │
+          └──────────▶| 4. setup_condition |                                  │
+                       |  Set condition for |                                  │
+                       |  recursion base.   |                                  │
+                       +--------------------+                                  │
+                                 │                                             ▼
+                                 │            +--------------------------------------+      
+                                 └──────────▶|       5. setup_recursive_layers      |
+                                              |        Configure recursive layers    |
+                                              |        and integrate proof.          |
+                                              +--------------------------------------+
+                                                    │                      ▲
+                                                    │                      │
+                                                    │                      │
+                                                    ▼                      │
+                                          +-----------------------------+  │
+                                          | 6. process_recursive_layer  |──┘
+                                          |  Handle recursion, verify,  |
+                                          |  and loop through steps.    |
+                                          +-----------------------------+
+                                                    │
+                                                    ▼
+                                          +-------------------------+
+                                          | 7. compile_and_process  |
+                                          |  Finalize circuit and   |
+                                          |  handle processing.     |
+                                          +-------------------------+
 ```
-+------------------+    +------------------+    +-----------------+
-| Initial Hash     |    | Current Hash     |    | Verifier Data   |
-| Target Gate      |──▶| Input Target     |──▶| Target Gate     |
-+------------------+    | (Updateable)     |    +-----------------+
-         │              +------------------+            │
-         │                   │    ▲                     │
-         │                   │    │                     │
-         │                   │    └───────┐             │
-         │                   │            │             │
-         │             +-----------+      │             │
-         └───────────▶| Condition |      │             │
-                       | Check Gate|      │             │
-                       +-----------+      │             │
-                                │         │             │
-                                │   +------------------------+
-                                └───┤ Recursive Proof        |
-                                    | Integration & Loop     |
-                                    +------------------------+
-                                            │           ▲
-                                            │           │
-                                            │           │
-                                            │           │
-                                    +---------------+   │
-                                    | Step Counter  |───┘
-                                    | & Loop Check  |
-                                    +---------------+
-                                            │
-                                            │
-                                    +---------------+
-                                    | Finalize Hash |
-                                    | & Verification|
-                                    +---------------+
-```
+You can also view a rough sketch of a circuit diagram of the entire setup:
+![](https://github.com/user-attachments/assets/0d7e9a9a-a1c3-4b5e-95fd-1dbe812d1076)
+
 
 ### Initial Setup
 - **Counter Initialization**: A counter gate is initialized to track the depth of recursion.
 - **Hash Initialization**: A virtual hash target gate is inserted and registered as a public input, marking the starting point of the hash chain.
-- **Keccak Hash Gate**: An updateable hash gate is added to enable hash updates as the recursion progresses.
+- **Hash Gate**: An updateable hash gate is added to enable hash updates as the recursion progresses.
 
 ### Recursive Hashing
 - **Verifier Data Setup**: Circuit common data is prepared, including configuration and partial witnesses required for recursion.
@@ -72,24 +130,69 @@ use plonky2::{
     },
 };
 
-// setup the parameters for the circuit
 const D: usize = 2;
-type C = PoseidonGoldilocksConfig; // this is the hasher for FRI, not for the circuit
+type C = PoseidonGoldilocksConfig; // A config with poseidon as the hasher for FRI
 type F = <C as GenericConfig<D>>::F;
 
-let config = CircuitConfig::standard_recursion_config(); // Non-ZK standard recursion config
-let mut hash_chain_circuit = CircuitBuilder::<F, D>::new(config.clone());
+// a non-ZK config, commitments and proof may reveal input data
+let config = CircuitConfig::standard_recursion_config(); 
+let mut circuit = CircuitBuilder::<F, D>::new(config.clone());
+
+// Prove
+let (proof, circuit_data) =
+    <CircuitBuilder<GoldilocksField, D> as HashChain<GoldilocksField, D, C>>::build_hash_chain_circuit(
+        &mut circuit,
+        2, // number of steps in the hash chain
+    )
+    .unwrap();
 
-<CircuitBuilder<GoldilocksField, 2> as HashChain<GoldilocksField, 2, C>>::hash_chain(
-    &mut hash_chain_circuit,
-    10,  // compute a hash chain of size 10
-)
-.unwrap();
+// Verify
+let verification_result =
+    <CircuitBuilder<GoldilocksField, D> as HashChain<GoldilocksField, D, C>>::verify(proof, circuit_data);
+assert!(verification_result.is_ok());
 ```
 
-We observe a total uncomressed proof size of 133440 bytes, regardless of number of steps in the chain.
+We observe a total uncompressed proof size of 133440 bytes, regardless of number of steps in the chain. We find this is very nice because this number stays the same no matter how many hashes we compute. In theory, recursively verifiable proofs of this nature can compress extremely large computations into a very small space. Think fully-succinct blockchains, in which light clients can verify the entire state of the chain trustlessly by verifying a small and simple proof in trivial amounts of time.
+
+## Benches
+
+This crate uses criterion for formal benchmarks. Bench prover and verifier performance with:
+
+```bash
+cargo bench
+```
+Here are some prelimnary performance metrics observed thus far:
+
+| Circuit depth (steps) | Prover Runtime (s) | Verifier Runtime (ms)| System RAM Used (mb)|
+|-----------------------|--------------------|----------------------|---------------------|
+| 2                     | 3.3680 s           | 3.1013 ms            | 375.692             |
+| 4                     | 4.2126 s           | 3.1220 ms            | 381.536             |
+| 8                     | 5.7366 s           | 3.0812 ms            | 392.716             |
+| 16                    | 8.8146 s           | 3.1098 ms            | 405.516             |
+| 32                    | 14.957 s           | 3.0865 ms            | 417.704             |
+| 64                    | 27.294 s           | 3.1625 ms            | 436.424             |
+
+
+## Acknowledgments
+
+This project makes use of the following open-source libraries:
+
+Recursive STARK Wiring framework:
+- **[plonky2](https://github.com/drcapybara/plonky2)** by Polygon Labs 
+
+
+Hash Circuits:
+- **[plonky2_crypto](https://github.com/JumpCrypto/plonky2-crypto)** by Jump Crypto 
+
+## Readings
+- [Plonky2 whitepaper](https://github.com/0xPolygonZero/plonky2/blob/main/plonky2/plonky2.pdf)
+
+- The always excellent [Anatomy of a STARK](https://aszepieniec.github.io/stark-anatomy/):
+
 
-TODO
+## TODO
+- [x] add benches
+- [x] better error handling with thiserr
 - [ ] Compress the proof at the end
 - [ ] support keccak
-- [ ] add benches
+- [ ] add richer circuit telemetry