Apache NiFi 1.2.0 发布,数据处理和分发系统

Apache NiFi 1.2.0 发布了,Apache NiFi 是一个易于使用、功能强大而且可靠的数据处理和分发系统。它为数据流设计,支持高度可配置的指示图的数据路由、转换和系统中介逻辑。新版本更新内容包括:


  • Now supports running multiple versions of the same components. Makes upgrades and multi-tenant flows easier to manage as sets the stage for upcoming Apache NiFi Registry work!

  • New provenance repository implementation available called 'WriteAheadProvenanceRepository'. Huge performance increase over the standard implementation and indexes in real-time.


  • New Record oriented abstraction for reading/writing schema aware event streams from CSV, JSON, AVRO, Grok, and plaintext with easy extension for other formats/schemas

  • QueryRecord processor to execute SQL queries over a stream of records powered by Apache Calcite

  • ConvertRecord processor to efficiently transform records from a given schema and format into another schema and format

  • SplitRecord processor to efficiently split huge record bundles into configurable batch sizes for divide and conquer or protect downstream systems

  • Processors to efficiently stream Records into and out of Apache Kafka in a format and schema aware manner and which automatically handle achieving high throughput and full provenance

  • Controller Services for plugging into and managing data schemas (Avro Schema Registry, Hortonworks Schema Registry) that integrate nicely into the record readers and writers

Features/improvements related to Change Data Capture (CDC), including CaptureChangeMySQL which reads from the MySQL binlogs, EnforceOrder, and PutDatabaseRecord processors, as well as a "Rollback on Failure" capability of some Put processors




Original article – 

Apache NiFi 1.2.0 发布,数据处理和分发系统