Amazon EMR now supports Apache Spark 4.0.2 in general availability

Amazon EMR Apache Spark 4.0.2 Support

Amazon EMR now supports Apache Spark 4.0.2 across all three deployment models. This update enhances data pipeline building with ANSI SQL and VARIANT data types, enforces fine-grained access control, and strengthens compliance with Apache Iceberg v3 table format. It also improves real-time application deployment with enhanced streaming capabilities.

What to do

• Upgrade your EMR applications using the Apache Spark upgrade agent.
• Explore new features in the Amazon EMR release notes.
• Create an EMR application with Spark 4.0.2 from the AWS Management Console.

Source: AWS release notes

If you need further guidance on AWS, our experts are available at AWS@westloop.io. You may also reach us by submitting the Contact Us form.