Course Outline
NiFi Fundamentals and Data Flow Concepts
- Distinguishing data in motion from data at rest: underlying concepts and challenges
- NiFi architecture: cores, flow controller, provenance, and bulletin board
- Core components: processors, connections, controllers, and provenance tracking
Big Data Context and System Integration
- The role of NiFi in Big Data ecosystems (Hadoop, Kafka, cloud storage)
- Overview of HDFS, MapReduce, and contemporary alternatives
- Practical use cases: stream ingestion, log shipping, and event pipelines
Installation, Configuration & Cluster Setup
- Deploying NiFi in single-node and cluster modes
- Configuring clusters: node roles, ZooKeeper integration, and load balancing
- Orchestrating deployments using Ansible, Docker, or Helm
Designing and Managing Dataflows
- Routing, filtering, splitting, and merging data flows
- Processor configuration (including InvokeHTTP, QueryRecord, PutDatabaseRecord, etc.)
- Managing schemas, data enrichment, and transformation operations
- Implementing error handling, retry mechanisms, and backpressure control
Integration Scenarios
- Connecting to databases, messaging queues, and REST APIs
- Streaming data to analytics platforms: Kafka, Elasticsearch, or cloud storage
- Integration with monitoring tools like Splunk, Prometheus, or logging pipelines
Monitoring, Recovery & Provenance
- Utilizing the NiFi UI, system metrics, and the provenance visualizer
- Designing autonomous recovery strategies and graceful failure handling
- Managing backups, flow versioning, and change control
Performance Tuning & Optimization
- Optimizing JVM settings, heap size, thread pools, and cluster parameters
- Refining flow design to minimize bottlenecks
- Managing resource isolation, flow prioritization, and throughput
Best Practices & Governance
- Documentation standards, naming conventions, and modular design principles
- Security measures: TLS, authentication, access control, and data encryption
- Change management, versioning, role-based access, and audit trails
Troubleshooting & Incident Response
- Addressing common issues: deadlocks, memory leaks, and processor errors
- Performing log analysis, error diagnostics, and root cause investigation
- Executing recovery strategies and flow rollbacks
Hands-on Lab: Realistic Data Pipeline Implementation
- Constructing an end-to-end flow: ingestion, transformation, and delivery
- Implementing error handling, backpressure, and scaling capabilities
- Conducting performance testing and pipeline tuning
Summary and Future Directions
Requirements
- Proficiency with the Linux command line
- Fundamental knowledge of networking and data systems
- Familiarity with data streaming or ETL principles
Intended Audience
- System administrators
- Data engineers
- Software developers
- DevOps practitioners
Testimonials (7)
Hands on exercises. Class should have been 5 days, but the 3 days helped to clear up a lot of questions that I had from working with NiFi already
James - BHG Financial
Course - Apache NiFi for Administrators
I thought the trainer's pace was good. He left no student behind with his approach. He was very supportive with us NEWBIES that may not have had a System Administrator or Infrastructure role during our career or resurrected those skills from a prior period of our career.
Pamdrea Ivory - BHG Financial
Course - Apache NiFi for Administrators
I like the hands on section. It helped me to better retain information by completing the provided exercises. Also, the trainer's ability to engage with the entire class made me feel comfortable to ask questions on things I was not sure about.
Leila - BHG Financial
Course - Apache NiFi for Administrators
Use-cases, examples for building NiFi dataflows. We worked on troubleshooting common problems and gotchas.
Nelson - BHG Financial
Course - Apache NiFi for Administrators
I loved the structure. We dove into the basics of Nifi, concepts, use cases, etc. on day 1. On day 2, we got to put Day 1 knowledge into practice by building out flows to meet scenario requirements. Day 3 we got to see Nifi Registry and version control, mulit-tenancy, and go over Q&A.
Adam - BHG Financial
Course - Apache NiFi for Administrators
Very little, I found it very difficult, especially because I fell behind and didn't attend the first sessions.
Rolando Garcia - OIT para Mexico y Cuba
Course - Apache NiFi for Administrators
Machine Translated
Knowing that there is a program with many potential applications for the work I do
Jorge Alberto Martinez - OIT para Mexico y Cuba
Course - Apache NiFi for Administrators
Machine Translated
