Lakehouse

AKA the third-generation data platform.

                                   .---.        .---.         o
.---.                      .-----> |   | -----> |   | -----> /|\
|   |-.     --.            |       '---'        |   |        / \
|   | |-.     |    .---.   |       .---.        |   |         o
'---' | |-. --+--> |   | --+-----> |   | -----> |   | -----> /|\
  '---' | |   |    '---'   |       '---'        |   |        / \
    '---' | --'            |       .---.        |   |         o
      '---'                '-----> |   | -----> |   | -----> /|\
                                   '---'        '---'        / \

  Sources       Raw Storage      Transforms   Metadata,      Users
                                              Governance

Characteristics:

• Management and optimization a la Data Warehouse
• Low-cost storage in open formats a la Data Lake

Platforms:

• Databricks Lakehouse Platform, https://databricks.com/product/data-lakehouse
• Azure Synapse Analytics, https://azure.microsoft.com/en-us/blog/simply-unmatched-truly-limitless-announcing-azure-synapse-analytics/
• BigQuery, https://cloud.google.com/bigquery/
• Redshift Spectrum, https://docs.aws.amazon.com/redshift/latest/dg/c-using-spectrum.html

Frameworks:

• Delta Lake, https://delta.io/
• Apache Iceberg, https://iceberg.apache.org/
• Apache Hudi, https://hudi.apache.org/

References

• An evolutionary history of enterprise data architectures, https://www.ibm.com/blogs/services/2021/08/03/an-evolutionary-history-of-enterprise-data-architectures/
• Data Lakehouse, https://databricks.com/glossary/data-lakehouse
• Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics, http://cidrdb.org/cidr2021/papers/cidr2021_paper17.pdf