Apache Atlas

Apache Atlas is an extensible set of core governance services for Hadoop and the wider enterprise data ecosystem. It gives organizations a common metadata store so any metadata consumer can work together without point-to-point interfaces, and it helps teams meet compliance requirements.

It provides visibility into data through prescriptive and forensic models, technical and operational audit, and lineage enriched with business taxonomical metadata. Security is both role based and attribute based, and Apache Ranger is used to prevent non-authorized access paths to data at runtime.

Atlas is an Apache project that builds with Java and runs on a self-hosted server, with documented Docker build and run instructions. The distribution ships server and hook tarballs for HBase, Hive, Impala, Kafka, Sqoop, Storm, Falcon, and Couchbase to capture metadata from those systems.