Data
Data Lake Design and Change Data Capture
Data flowing into the Data Lake obviously changes. Data table changes are captured by CDC or change data capture. Changes in the source database are delivered …
Read More »
DataRedshift vs Redshift Spectrum
Amazon Redshift is a petabyte scalable columnar data warehouse that is very efficient in storing raw data and collecting data from various sources. Redshift su…
Read More »
Cloud Design GoalsData products – a summary
Data products are the end result of file or data movements to the cloud; ETL; processing; de-duplication; curation and storage in a consumable layer. There is …
Read More »
Cloud Design GoalsUnderstanding Data Lakes and Data Warehouses
In simple terms we can identify the differences between Data Lakes and Data Warehouses. Data Lake: A data lake is a centralized repository, usually a platform,…
Read More »
Application TransformationDigital and Data Transformation (Why)
Digital Transformation Digital transformation not a magic solution nor a buffet of word salads. DT is roughly defined as the integration of digital technologie…
Read More »
AWS TechnologyStandard Technology Stack for an AWS based Data Lake
A typical Technology Stack for a Data Lake. S3 as the Golden Source. Snowflake as a corporate Data Share with SQL use cases. If AWS-S3 and Redshift are not pro…
Read More »
DataImproving Redshift Data Warehouse performance
(ETL engine in the above could be AWS Glue) There are various ways to define performance and what that means. A simple way to be consistent with management is …
Read More »
DataIceberg and Data Format open standards
Iceberg Cometh Open table formats, such as Apache Iceberg, enable scale-out data warehousing directly on a data lake. This architecture has become known as a d…
Read More »
AWS TechnologyData Lake Basics and AWS
A data lake is a centralized repository that allows a firm to store structured and unstructured data at any scale. You can store your data as-is, without havin…
Read More »
DataData Products, overview, summary
Traditional Data Product Management Federated data management and data product builds and sharing has little to do with traditional data product management. Tr…
Read More »
DataData Lake Architecture(s) overview
Data Lake Architecture Data lake architecture was introduced in 2010 in response to the challenges of data warehousing architecture in satisfying the new uses …
Read More »
AWS TechnologyAWS, Snowpipe, Snowflake
Moving data from S3 to Snowflake to satisfy use cases around analysis, corporate reporting, or cross-domain information collaboration is best achieved through …
Read More »