Apache Pulsar 架構原理

消息系統的發展歷程

大數據實時數據流處理流程

雲原生架構特點

  1. 存儲和計算分離

  2. 節點對等 

  3. 獨立擴展 

  4. 靈活擴容 

  5. 快速容錯

企業級流存儲層：Apache BookKeeper

BookKeeper 分佈式日誌 / 流存儲具有特性：

  1. 低延時、高吞吐、持久化

  2. 強一致

  3. 高可用

4.I/O 隔離

企業級特性：大集羣、多租戶

原生的跨地域複製

Pulsar 介紹：

Apache Pulsar is a cloud-native event streaming platform

Connect, Store, and Process real-time event streams

Connect: Pulsar clients, IO connectors, Protocol handlers

Store: Apache BookKeeper, Tiered storage

Process:

Pulsar Functions - Lightweight serverless computation  

Spark / Flink - Unified data processing

Presto - Interactive queries

Tenant & Namespace 

Pulsar Architecture

Apache BookKeeper

BookKeeper — 穩定的 IO 質量

Namespace Bundles

Load Manager - Assign bundles to brokers

Topic Lookup

HTTP Topic Lookup

Topic Lookup with Proxies

Geo-Replication

Geo-replication w/o Global ZK

Aggregation Pattern

本文由 Readfog 進行 AMP 轉碼，版權歸原作者所有。
來源：https://mp.weixin.qq.com/s/bMB6OGfOu8G62RWNM3G3bQ

猜你喜歡