• Kafka Brokers form the messaging, data persistency and storage tier of Apache Kafka
  • Kafka Java Client APIs
    • Producer API publish a stream record
    • Consumer API subcribe to topics
    • Streams API act as stream processor
    • Connect API stream data with other systems

    使用stream api时，当部署多个application时，kafka自动处理负载均衡问题。

stream api

  • 简单的客户端library
  • 没有外部依赖
  • 支持容错的局部状态
  • 支持每条记录只处理一次
  • 一次处理一条记录 所以延迟低

time

  • event time - generated source time
  • processing time - when the record is being consumed
  • ingestion time - when an event or record is stored in a partition