Picture a fine-tuning dataset assembled from a public web crawl, a licensed news archive and customer support tickets. Provenance records, for each part, the source, the creator, the license or consent terms, the collection date and the processing applied (deduplication, filtering, PII removal). Without that record, nobody can later answer basic questions: may we use this commercially, did customers agree, and which data shaped this behavior?
The problem is common. The Data Provenance Initiative, a team of legal and machine learning experts, audited more than 1,800 text datasets and traced their sources, creators and license conditions, finding that the documentation many datasets ship with is inconsistent. Groups such as the Data & Trusted AI Alliance have published metadata standards for recording dataset origin, and the W3C’s PROV specifications provide a general vocabulary for provenance.
In episode 17 at 18:07, IBM’s Maryam Ashoori describes a governance process around training IBM’s Granite models that documents “the lineage of what happens” and filters out toxic and copyrighted material. That is IBM’s account; episode 17 was sponsored by Galileo. Editorially, provenance answers where data came from and on what terms, while lineage traces how it moved and changed through pipelines; see the explainer on data lineage. Both underpin copyright and AI training defenses, data governance and the AI supply chain.
Hear it from the guest
“We establish a trustworthy governance process around the training of the model that we can document the lineage of what happens”
Quotes lightly edited to remove filler words.
Sources
- Longpre et al.: The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AI — Legal and machine learning experts audit and trace 1800+ text datasets, from source and creators to license conditions and subsequent use; checked October 24, 2026.
- Data Provenance Initiative — Project site for the dataset audits and tools; checked October 24, 2026.
- Data & Trusted AI Alliance: Data Provenance Standards — Cross-industry metadata standards (v1.0.0) for documenting the origin of datasets used in traditional data and AI applications; checked October 24, 2026.
- W3C: PROV-Overview — Overview of the W3C PROV family of specifications for representing and exchanging provenance information; checked October 24, 2026.
