Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 21 hours
Course Outline
Introduction:
- Apache Spark within the Hadoop Ecosystem
- Brief Overview of Python and Scala
Foundational Concepts (Theory):
- Spark Architecture
- Resilient Distributed Datasets (RDD)
- Transformations and Actions
- Stages, Tasks, and Dependencies
Exploring Fundamentals in a Databricks Environment (Hands-on Workshop):
- Practical exercises using the RDD API
- Core action and transformation functions
- PairRDD operations
- Join operations
- Caching strategies
- Practical exercises using the DataFrame API
- SparkSQL
- DataFrame operations: select, filter, group, sort
- User Defined Functions (UDF)
- Exploring the Dataset API
- Structured Streaming
Deployment in an AWS Environment (Hands-on Workshop):
- Fundamentals of AWS Glue
- Distinguishing between AWS EMR and AWS Glue
- Implementing sample jobs in both environments
- Analyzing advantages and limitations
Additional Topics:
- Introduction to Apache Airflow for orchestration
Requirements
Programming experience (ideally with Python and Scala)
Fundamentals of SQL
Testimonials (3)
Having hands on session / assignments
Poornima Chenthamarakshan - Intelligent Medical Objects
Course - Apache Spark in the Cloud
1. Right balance between high level concepts and technical details. 2. Andras is very knowledgeable about his teaching. 3. Exercise
Steven Wu - Intelligent Medical Objects
Course - Apache Spark in the Cloud
Get to learn spark streaming , databricks and aws redshift