Understanding the ZK rollup is not required to use Polybase since the SDK and network take care of it.
Polybase is a state zk-rollup protocol, that provides native support for modular data storage and indexing.
Here is an overview of the Polybase protocol architecture:
If you want more detail, check out our whitepaper.
ZK State Rollup
The zk state rollup allows us to prove any record or value in the database both exists and that it is valid (according to the rules of the collection). We can prove this with only the root hash and the zk-proof.
Every record is stored as a hash (a fixed-size, random and unique representation of the record). The root hash is determined by combining the hashes for every record, and using that to create a single root hash. Because a hash represents the underlying data, any changes to any record would result in a change of that record’s hash, which in turn would change the root hash (because the root hash is based on every records’ hash).
To make a change to a record in the database, a user must prove that a new hash is valid. To do this, a zk-proof is generated that takes the existing hash (representing the current state of the record), applies the change function as defined in the collection definition, and then returns the new records hash with a zk-proof, proving that the change is valid. In Polybase, the process of creating the zk-proof is performed automatically by our SDK.
The user can then submit the new hash to the network with the proof. Every block, all the new hashes are updated and combined with the existing hashes to generate a new root hash. At each stage of this rollup process, the network confirms that the proofs are valid.
The root hash and proof are verified in a Proof of Stake blockchain, to inherit the security of the L1 blockchain.
Modular Storage and Indexing
Polybase has native support for modular storage and indexing. This allows developers to choose the trade-offs that best fit their use case.
For indexing, services could be created to serve, search, aggregation or machine learning use cases. As the root state proofs can be verified, indexers and aggregators can build on top of the Polybase state proofs, to provide proof that the aggregation was performed correctly.
For storage, this could include local storage on the clients device (for ultra sensitive data), a decentralized storage network such as Filecoin/Arweave (for collaborative data), or even a centralized storage provider such as S3 (if you don’t care about decentralisation 🥲).
Polybase currently only supports the Polybase reference indexing and storage implementation this is based on NoSQL, but additional indexers and storage providers will be added soon that will support GraphQL and SQL.
The Polybase Gatweway allows fast and simple access to the Polybase network, but in a less decentralized way. While Polybase is in Alpha, this is the only way to access the Polybase network, allowing us to ensure the security of the protocol before allowing direct access. Once direct access is enabled, clients (browsers and mobiles) will be able to connect directly to the network using p2p, offering both performance and decentralization benefits.
The following diagram shows how the Polybase Gateway interacts with the Polybase network:
See our reference Polybase Gateway API documentation.