AWS Glue Use Case examples
AWS Glue can serve a wide array of data engineering use cases. Loading Data To The Data Warehouse: One of Glue’s earliest and most popular use cases is loading…
Read More »AWS, ETL with Glue
AWS Glue was designed based on these principles: Provide customers the ability to solve problems when the system can not satisfy their needs. Examples include …
Read More »
Migrating ‘Big Data’ processes to AWS
Migrating ‘ Big Data’ pipelines and data processes to AWS will allow organizations to: Scale Seamlessly: AWS services like EMR (to replace Hadoop),…
Read More »
AWS Data Lake, Streaming Architecture summary
Example project deployment at a Financial Institution. Creation of a Data Lake with a streaming/real time data ingestion requirement. AWS Data Architecture Des…
Read More »
AWS RDS summary
Amazon RDS (Relational Database Service) RDS is a managed relational database service that supports various database engines such as MySQL, PostgreSQL, Oracle,…
Read More »
AWS S3 Iceberg Tables
Amazon S3 Iceberg Tables introduced fully managed Apache Iceberg table support to S3, optimizing the storage and querying of tabular data for analytics. By cre…
Read More »
Relational, Document, Key Value Databases: summary, use cases
There are a wide variety of databases. With cloud and hybrid architectures often see the following: Relational DB A structured format with rows and columns, re…
Read More »
Batching and AWS S3 on prem to S3 cloud
A useful architecture to move data from on-premises to AWS is to consider using AWS S3 outputs and move data directly over a Direct Connect to S3 in AWS. This …
Read More »
Data Ops, a summary
In essence Data Operations is based on DevSecOps or DevOps and applies these same ideas to the life cycle of data management. DataOps advocates for agility, na…
Read More »