Data Lake

A data warehouse asks you to decide the shape of your tables before data goes in. A lake does not. A company can dump clickstream logs, support tickets, scanned contracts and sensor readings into cheap cloud object storage today and decide later what to do with them. That flexibility is why data science and machine learning teams like lakes: unstructured data such as text and images, which warehouses handle poorly, is exactly what modern models train on.

The weakness is the flip side. Without catalogs, ownership and quality checks, a lake fills with files nobody can find or trust. People call that a “data swamp.” A data catalog and data quality rules are the usual remedies.

The lakehouse is the industry’s attempt to get both. Open table formats such as Apache Iceberg and Delta Lake add database-like tables, transactions and versioning to files sitting in the lake, so analysts can run SQL and ML teams can read the same data without copying it into a separate warehouse. The term was popularized by Databricks, which sells lakehouse products, so treat vendor comparisons accordingly.

In episode 55 at 4:58, Neo4j’s Sudhir Hasbe argues that putting siloed enterprise data into “a data lake or data swamp” still leaves agents facing thousands of disconnected tables; at 6:35 he proposes a knowledge graph as a semantic map over them. That is his company’s approach, and Neo4j sells graph databases. Episode 55 was sponsored by Galileo (sponsor read at 1:37). For the case against moving everything into one lake at all, see Centralizing data before AI, drawn from Starburst’s Jitender Aswani in episode 66.