Aurora PostgreSQL now supports querying of Apache Iceberg and Parquet data

New Feature: Query Data Lakes with Aurora PostgreSQL

Starting today, you can query operational data together with data stored in data lakes in Apache Iceberg and Parquet formats using your existing PostgreSQL applications and tools, without ETL pipelines or data duplication.

Aurora PostgreSQL now supports creating PostgreSQL foreign tables that reference your Iceberg or Parquet data in Amazon S3, Amazon S3 Tables, or AWS Glue Data Catalog. When querying these foreign tables, Aurora uses DuckDB’s high-performance query engine to execute the query against the underlying Iceberg and Parquet data. Your existing applications and BI tools continue to use the same PostgreSQL interface.

Additionally, you can query tables from external Iceberg REST Catalog (IRC)-compatible catalogs federated through AWS Glue Data Catalog without moving or duplicating data. For latency-sensitive workloads, you can materialize Iceberg or Parquet data into native Aurora PostgreSQL tables using standard SQL statements, without an ETL pipeline.

What to do

• Use the Amazon RDS console or any PostgreSQL client to get started.
• Read the blog post or documentation for more information.

Source: AWS release notes

If you need further guidance on AWS, our experts are available at AWS@westloop.io. You may also reach us by submitting the Contact Us form.