A Networked Consensus Protocol for Collaborative Model Governance with Auditable Lineage in Data Platforms
PDF

How to Cite

[1]
Khang, P.M. and Dat, V.T. 2022. A Networked Consensus Protocol for Collaborative Model Governance with Auditable Lineage in Data Platforms. Global Frontiers in Integrated Engineering, Mathematics, and Applied Sciences Research. 12, 3 (Mar. 2022), 1–24.

Abstract

Modern data platforms increasingly host many machine learning models that are jointly authored, reviewed, and operated across teams, vendors, and regulated boundaries. As model artifacts, training data snapshots, and policy decisions evolve, governance workflows must reconcile two competing requirements: collaborative change with low friction and rigorous auditability with tamper-evident lineage. This paper presents a networked consensus protocol tailored to collaborative model governance in data platforms, in which governance actions are expressed as deterministic state transitions over an append-only log that binds cryptographic attestations to lineage-anchored model metadata. The protocol combines Byzantine fault tolerant state machine replication for governance decisions with content-addressed provenance graphs for auditable lineage, enabling independent verifiers to reconstruct end-to-end derivations from data inputs to deployed model versions. We formalize a system model spanning multi-tenant registries, pipeline execution services, feature stores, and deployment controllers, and we define governance objects, admissibility rules, and policy predicates as first-class state. The design introduces quorum-certified governance blocks, deterministic policy evaluation over signed evidence bundles, and a Merkle-DAG lineage substrate that supports partial disclosure while preserving global consistency. We analyze safety, liveness, and audit completeness under realistic fault assumptions, and we develop performance and storage models that quantify latency, throughput, and verification cost. Implementation considerations are discussed for lakehouse-style platforms and hybrid cloud environments, along with an evaluation methodology emphasizing reproducibility, adversarial testing, and compliance-oriented audits.

PDF