bitcask 性能並不差！存儲引擎性能評測

簡介

衆所周知，bitcask 存儲引擎最大的缺點是：內存需要存全量數據的索引。其他方面看起來像是優點，比如：

每次查詢需要一次 disk seek
所有數據都追加寫到 wal log

詳細介紹見鏈接：https://arpitbhayani.me/blogs/bitcask

那麼，到底 bitcask 存儲引擎與其他主流的 LSM，B 樹類的存儲引擎，性能有多大差異呢？

這裏選用了 golang 比較受歡迎的 KV 存儲，包括 LSM 類型的存儲引擎 badger、levelDB，B+ 樹類型的存儲引擎 BoltDB，以及 bitcask 類型的存儲引擎 bitcaskDB。

4KB 讀

測試用例：先寫 1 百萬個鍵值對，value 爲 4KB，key 爲 25 B，然後測試併發隨機讀的性能。測試倉庫：https://github.com/wenzhang-dev/bitcaskDB-benchmark

可以看到 bitcaskDB 平均每次讀大約需要 6.5 微秒，平均佔用內存爲 1.2 GB。Badger 大約耗時 14 微秒，平均佔用內存 1.6 GB。LevelDB 大約耗時 57 微秒，佔用內存 100 MB。BoltDB 大約耗時 5 微秒，佔用內存 5.8 GB。

可以發現，bitcaskDB 使用的內存，相比 Badger 和 BoltDB 還算少的。分析了一下它們提供的默認參數。爲了性能，它們會使用 mmap 將 vlog 或者數據文件映射到內存，會喫不少的內存。

所以，在併發隨機讀測試中，bitcask 佔用內存和讀性能都不差。

go test -bench=Read -benchtime=60s -timeout=30m -count=3
goos: linux
goarch: amd64
pkg: github.com/wenzhang-dev/bitcaskDB-benchmark
cpu: Intel(R) Xeon(R) Gold 5318N CPU @ 2.10GHz
BenchmarkReadWithBitcaskDB/read4K-8  11459024   6313 ns/op  1.217 AvgRSS(GB)  1.275 PeakRSS(GB)  10120 B/op  10 allocs/op
BenchmarkReadWithBitcaskDB/read4K-8  12512324   6522 ns/op  1.220 AvgRSS(GB)  1.234 PeakRSS(GB)  10120 B/op  10 allocs/op
BenchmarkReadWithBitcaskDB/read4K-8  12414660   6468 ns/op  1.206 AvgRSS(GB)  1.231 PeakRSS(GB)  10120 B/op  10 allocs/op
BenchmarkReadWithBadger/read4K-8      4575487  13526 ns/op  2.716 AvgRSS(GB)  4.350 PeakRSS(GB)  19416 B/op  43 allocs/op
BenchmarkReadWithBadger/read4K-8      4960239  13741 ns/op  1.629 AvgRSS(GB)  1.681 PeakRSS(GB)  19406 B/op  43 allocs/op
BenchmarkReadWithBadger/read4K-8      4851144  14429 ns/op  1.591 AvgRSS(GB)  1.650 PeakRSS(GB)  19422 B/op  44 allocs/op
BenchmarkReadWithLevelDB/read4K-8     1569663  50710 ns/op  0.111 AvgRSS(GB)  0.134 PeakRSS(GB)  55021 B/op  35 allocs/op
BenchmarkReadWithLevelDB/read4K-8     1000000  63066 ns/op  0.113 AvgRSS(GB)  0.129 PeakRSS(GB)  54264 B/op  35 allocs/op
BenchmarkReadWithLevelDB/read4K-8     1236408  57268 ns/op  0.114 AvgRSS(GB)  0.138 PeakRSS(GB)  54624 B/op  35 allocs/op
BenchmarkReadWithBoltDB/read4K-8     12587562   5269 ns/op  5.832 AvgRSS(GB)  5.838 PeakRSS(GB)    832 B/op  13 allocs/op
BenchmarkReadWithBoltDB/read4K-8     16920481   4482 ns/op  5.832 AvgRSS(GB)  5.833 PeakRSS(GB)    832 B/op  13 allocs/op
BenchmarkReadWithBoltDB/read4K-8     19141418   5276 ns/op  5.832 AvgRSS(GB)  5.835 PeakRSS(GB)    832 B/op  13 allocs/op
PASS
ok   github.com/wenzhang-dev/bitcaskDB-benchmark 1475.172s

4KB 寫

測試用例：反覆併發的寫 1 千萬個鍵值對，key 爲 25 B，value 爲 4 KB。

可以看到 bitcask 每次寫操作大約耗時 13 微秒，平均內存佔用 1 GB。Badger 平均耗時 45 微秒，平均內存佔用 10 GB。LevelDB 平均耗時 70 微秒，平均內存佔用 60 MB。BoltDB 平均耗時 22 微秒，平均內存佔用 1.5 GB。

比較下來，bitcask 併發 4KB 寫的表現也不差，同時佔用內存也不算高。

go test -bench=Write -benchtime=60s -timeout=30m -count=3
goos: linux
goarch: amd64
pkg: github.com/wenzhang-dev/bitcaskDB-benchmark
cpu: Intel(R) Xeon(R) Gold 5318N CPU @ 2.10GHz
BenchmarkWriteWithBitcaskDB/write4K-8  8334304  13217 ns/op  0.7905 AvgRSS(GB)   0.934 PeakRSS(GB)    1666 B/op   11 allocs/op
BenchmarkWriteWithBitcaskDB/write4K-8  5323338  14976 ns/op  0.9732 AvgRSS(GB)   1.058 PeakRSS(GB)    1727 B/op   12 allocs/op
BenchmarkWriteWithBitcaskDB/write4K-8  5435398  13929 ns/op  0.9639 AvgRSS(GB)   1.122 PeakRSS(GB)    1756 B/op   12 allocs/op
BenchmarkWriteWithLevelDB/write4K-8    1047753  68691 ns/op  0.0615 AvgRSS(GB)  0.0636 PeakRSS(GB)    2946 B/op   16 allocs/op
BenchmarkWriteWithLevelDB/write4K-8    1179555  71497 ns/op  0.0617 AvgRSS(GB)  0.0634 PeakRSS(GB)    3250 B/op   18 allocs/op
BenchmarkWriteWithLevelDB/write4K-8     992488  74130 ns/op  0.0613 AvgRSS(GB)  0.0625 PeakRSS(GB)    3444 B/op   19 allocs/op
BenchmarkWriteWithBadger/write4K-8     3776720  20036 ns/op   6.409 AvgRSS(GB)   7.534 PeakRSS(GB)   30062 B/op   68 allocs/op
BenchmarkWriteWithBadger/write4K-8     4106070  50959 ns/op   10.77 AvgRSS(GB)   13.63 PeakRSS(GB)  115442 B/op  152 allocs/op
BenchmarkWriteWithBadger/write4K-8     1491906  49955 ns/op   11.45 AvgRSS(GB)   13.72 PeakRSS(GB)   88941 B/op  130 allocs/op
BenchmarkWriteWithBoltDB/write4K-8     2808206  23131 ns/op   0.626 AvgRSS(GB)   0.999 PeakRSS(GB)    7579 B/op   11 allocs/op
BenchmarkWriteWithBoltDB/write4K-8     4303538  22836 ns/op   1.713 AvgRSS(GB)   2.971 PeakRSS(GB)    7765 B/op   11 allocs/op
BenchmarkWriteWithBoltDB/write4K-8     3755002  19385 ns/op   2.481 AvgRSS(GB)   2.872 PeakRSS(GB)    7896 B/op   12 allocs/op
PASS
ok   github.com/wenzhang-dev/bitcaskDB-benchmark 1541.068s

磁盤佔用大小

測試用例：批量寫 1 百萬個不同的鍵值對，key 爲 25 B，value 爲 4KB，然後查看它們的磁盤空間佔用。

測試發現，bitcaskDB 完全沒有壓縮，佔用磁盤 3.9 GB。LevelDB 默認開啓了 snappy 壓縮，佔用磁盤 250 MB。Badger 佔用磁盤 2.4 GB。BoltDB 佔用磁盤空間 5.7 GB。由於 4 KB value 是一個 128 B 循環拼接而成，開啓壓縮後，磁盤佔用率會更有優勢。

go test -bench=DiskUsage -benchtime=60s -count=1 -timeout=30m
goos: linux
goarch: amd64
pkg: github.com/wenzhang-dev/bitcaskDB-benchmark
cpu: Intel(R) Xeon(R) Gold 5318N CPU @ 2.10GHz
BenchmarkDiskUsageWithBitcaskDB/1_million-8 1000000000  0.003079 ns/op 3.880 DiskUsage(GB)
BenchmarkDiskUsageWithLevelDB/1_million-8   1000000000  0.004168 ns/op 0.252 DiskUsage(GB)
BenchmarkDiskUsageWithBadger/1_million-8    1000000000   0.02207 ns/op 2.431 DiskUsage(GB)
BenchmarkDiskUsageWithBoltDB/1_million-8    1000000000  0.002641 ns/op 5.775 DiskUsage(GB)
PASS
ok   github.com/wenzhang-dev/bitcaskDB-benchmark 109.965s

GC 暫停時間

由於 bitcask 存儲引擎，需要將全量索引放內存。那麼堆上勢必有大量的小對象，用來表示鍵值在 log 文件的位置信息。因此，GC 暫停時間也是一個重要的指標。

下面是 bitcaskDB 執行併發 4 KB 寫時 GC Pause 的表現。迭代寫約 6 千萬次，總耗時 14.5 分鐘，GC 暫停時間 86 毫秒。

go test -bench=GcOverhead -benchtime=300s -count=1 -timeout=30m
goos: linux
goarch: amd64
pkg: github.com/wenzhang-dev/bitcaskDB/bench
cpu: Intel(R) Xeon(R) Gold 5318N CPU @ 2.10GHz
BenchmarkGcOverhead-8
GC pause for startup: gc=1.246146ms
GC pause fortest: total=64.350396ms, gc=160.293µs, iter=1
GC pause for startup: gc=719.201µs
GC pause fortest: total=105.89436ms, gc=176.146µs, iter=100
GC pause for startup: gc=202.888µs
GC pause fortest: total=150.224754ms, gc=63.252µs, iter=10000
GC pause for startup: gc=323.495µs
GC pause fortest: total=5.88998191s, gc=1.734889ms, iter=1000000
GC pause for startup: gc=181.486µs
GC pause fortest: total=14m31.795948219s, gc=86.133309ms, iter=59247226
59247226      14743 ns/op     1698 B/op       10 allocs/op
PASS
ok   github.com/wenzhang-dev/bitcaskDB/bench 884.982s

固定磁盤大小

bitcaskDB 稍加配置，就可以作爲一個磁盤緩存。指定內存和磁盤上限，就可以按照近似 LRU 進行淘汰數據。比如下面的配置，限定內存容量 1 千萬個鍵值對，磁盤容量 10 GB。

func newDB(b *testing.B) {
        dir = "./bitcaskDB"
        _ = os.RemoveAll(dir)
        _ = os.MkdirAll(dir, os.ModePerm)

        opts := &bitcask.Options{
                Dir:                       dir,
                WalMaxSize:                1024 * 1024 * 1024, // 1GB
                ManifestMaxSize:           10 * 1024 * 1024,   // 10MB
                IndexCapacity:             10000000,           // 10 million
                IndexLimited:              8000000,
                IndexEvictionPoolCapacity: 64,
                IndexSampleKeys:           5,
                CompactionPicker:          nil, // default picker
                CompactionFilter:          nil, // default filter
                NsSize:                    bitcask.DefaultNsSize,
                EtagSize:                  bitcask.DefaultEtagSize,
                DisableCompaction:         false,
                DiskUsageLimited:          10 * 1024 * 1024 * 1024, // 10GB
        }

        var err error
        db, err = bitcask.NewDB(opts)
        assert.Nil(b, err)
}

在上述配置下，測試併發 4 KB 寫的性能如下。可以發現，不限速的寫，雖然限定磁盤容量 10 GB，但是實際佔用的峯值可能達到 15 GB。定時回收磁盤空間的後臺任務，會按照 wal 創建時間的先後順序進行近似 LRU 淘汰。該定時的間隔越短，限定磁盤容量約準確，其默認值爲 20 秒。

go test -bench=DiskUsage -benchtime=300s -count=1 -timeout=30m  > log
goos: linux
goarch: amd64
pkg: github.com/wenzhang-dev/bitcaskDB/bench
cpu: Intel(R) Xeon(R) Gold 5318N CPU @ 2.10GHz
BenchmarkDiskUsage/concurrentBatchPut4K-8
Write Speed 681.80 MB/s | Write Total: 2.00 GB | Disk Usage: 2.01 GB
Write Speed 414.79 MB/s | Write Total: 3.21 GB | Disk Usage: 3.24 GB
Write Speed 759.28 MB/s | Write Total: 2.22 GB | Disk Usage: 2.24 GB
Write Speed 605.14 MB/s | Write Total: 4.00 GB | Disk Usage: 4.03 GB
Write Speed 341.12 MB/s | Write Total: 5.00 GB | Disk Usage: 5.04 GB
Write Speed 336.92 MB/s | Write Total: 5.98 GB | Disk Usage: 6.04 GB
Write Speed 345.39 MB/s | Write Total: 7.00 GB | Disk Usage: 7.06 GB
Write Speed 323.72 MB/s | Write Total: 7.94 GB | Disk Usage: 8.02 GB
Write Speed 273.67 MB/s | Write Total: 8.75 GB | Disk Usage: 8.83 GB
Write Speed 149.68 MB/s | Write Total: 9.18 GB | Disk Usage: 9.27 GB
Write Speed 305.53 MB/s | Write Total: 10.08 GB | Disk Usage: 10.18 GB
Write Speed 313.14 MB/s | Write Total: 11.00 GB | Disk Usage: 11.11 GB
Write Speed 339.74 MB/s | Write Total: 11.99 GB | Disk Usage: 12.12 GB
Write Speed 341.25 MB/s | Write Total: 12.99 GB | Disk Usage: 13.13 GB
Write Speed 316.89 MB/s | Write Total: 13.92 GB | Disk Usage: 14.07 GB
Write Speed 110.53 MB/s | Write Total: 14.24 GB | Disk Usage: 9.35 GB
Write Speed 255.09 MB/s | Write Total: 14.99 GB | Disk Usage: 10.10 GB
Write Speed 351.83 MB/s | Write Total: 16.02 GB | Disk Usage: 11.14 GB
Write Speed 330.42 MB/s | Write Total: 16.99 GB | Disk Usage: 12.12 GB
Write Speed 313.67 MB/s | Write Total: 17.91 GB | Disk Usage: 13.05 GB
Write Speed 214.56 MB/s | Write Total: 18.54 GB | Disk Usage: 13.68 GB
Write Speed 333.37 MB/s | Write Total: 19.51 GB | Disk Usage: 14.67 GB
Write Speed 162.09 MB/s | Write Total: 19.99 GB | Disk Usage: 10.18 GB
Write Speed 277.88 MB/s | Write Total: 20.80 GB | Disk Usage: 9.14 GB
Write Speed 263.49 MB/s | Write Total: 21.58 GB | Disk Usage: 9.93 GB
Write Speed 276.17 MB/s | Write Total: 22.38 GB | Disk Usage: 10.75 GB
Write Speed 205.57 MB/s | Write Total: 22.99 GB | Disk Usage: 11.35 GB
Write Speed 284.71 MB/s | Write Total: 23.82 GB | Disk Usage: 12.19 GB
Write Speed 214.63 MB/s | Write Total: 24.45 GB | Disk Usage: 9.80 GB
Write Speed 394.39 MB/s | Write Total: 25.61 GB | Disk Usage: 10.97 GB
Write Speed 262.18 MB/s | Write Total: 26.37 GB | Disk Usage: 11.74 GB
Write Speed 246.88 MB/s | Write Total: 27.10 GB | Disk Usage: 12.47 GB
Write Speed 336.85 MB/s | Write Total: 28.08 GB | Disk Usage: 13.47 GB
Write Speed 390.12 MB/s | Write Total: 29.23 GB | Disk Usage: 14.63 GB
Write Speed 354.78 MB/s | Write Total: 30.27 GB | Disk Usage: 15.67 GB
Write Speed 244.80 MB/s | Write Total: 30.98 GB | Disk Usage: 10.09 GB
Write Speed 289.30 MB/s | Write Total: 31.83 GB | Disk Usage: 10.95 GB
Write Speed 359.99 MB/s | Write Total: 32.89 GB | Disk Usage: 12.02 GB
Write Speed 262.57 MB/s | Write Total: 33.65 GB | Disk Usage: 12.80 GB
Write Speed 283.72 MB/s | Write Total: 34.49 GB | Disk Usage: 13.64 GB
Write Speed 259.16 MB/s | Write Total: 35.24 GB | Disk Usage: 14.40 GB
Write Speed 293.51 MB/s | Write Total: 36.10 GB | Disk Usage: 15.27 GB
Write Speed 127.44 MB/s | Write Total: 36.48 GB | Disk Usage: 9.63 GB
Write Speed 373.13 MB/s | Write Total: 37.57 GB | Disk Usage: 10.91 GB
Write Speed 407.88 MB/s | Write Total: 38.77 GB | Disk Usage: 10.09 GB
Write Speed 398.95 MB/s | Write Total: 39.94 GB | Disk Usage: 11.28 GB
Write Speed 317.95 MB/s | Write Total: 40.87 GB | Disk Usage: 12.22 GB
Write Speed 214.49 MB/s | Write Total: 41.49 GB | Disk Usage: 12.86 GB
Write Speed 253.70 MB/s | Write Total: 42.24 GB | Disk Usage: 13.61 GB
Write Speed 252.52 MB/s | Write Total: 42.98 GB | Disk Usage: 10.31 GB
Write Speed 310.33 MB/s | Write Total: 43.89 GB | Disk Usage: 11.23 GB
Write Speed 372.21 MB/s | Write Total: 44.98 GB | Disk Usage: 12.33 GB
Write Speed 216.13 MB/s | Write Total: 45.61 GB | Disk Usage: 12.98 GB
Write Speed 285.37 MB/s | Write Total: 46.45 GB | Disk Usage: 13.82 GB
Write Speed 260.40 MB/s | Write Total: 47.21 GB | Disk Usage: 14.59 GB
Write Speed 261.78 MB/s | Write Total: 47.98 GB | Disk Usage: 15.37 GB
Write Speed 340.75 MB/s | Write Total: 48.97 GB | Disk Usage: 10.10 GB
Write Speed 241.68 MB/s | Write Total: 49.68 GB | Disk Usage: 10.82 GB
Write Speed 145.64 MB/s | Write Total: 50.11 GB | Disk Usage: 11.24 GB
Write Speed 295.16 MB/s | Write Total: 50.97 GB | Disk Usage: 12.12 GB
Write Speed 219.15 MB/s | Write Total: 51.62 GB | Disk Usage: 12.78 GB
Write Speed 223.22 MB/s | Write Total: 52.27 GB | Disk Usage: 13.43 GB
Write Speed 80.93 MB/s | Write Total: 52.51 GB | Disk Usage: 9.63 GB
Write Speed 248.19 MB/s | Write Total: 53.23 GB | Disk Usage: 10.60 GB
Write Speed 251.40 MB/s | Write Total: 53.97 GB | Disk Usage: 9.33 GB
Write Speed 341.07 MB/s | Write Total: 54.97 GB | Disk Usage: 10.34 GB
Write Speed 277.29 MB/s | Write Total: 55.78 GB | Disk Usage: 11.16 GB
Write Speed 210.54 MB/s | Write Total: 56.40 GB | Disk Usage: 11.79 GB
Write Speed 250.81 MB/s | Write Total: 57.13 GB | Disk Usage: 12.52 GB
Write Speed 285.10 MB/s | Write Total: 57.97 GB | Disk Usage: 10.34 GB
Write Speed 340.88 MB/s | Write Total: 58.97 GB | Disk Usage: 11.35 GB
Write Speed 341.07 MB/s | Write Total: 59.97 GB | Disk Usage: 12.36 GB
Write Speed 341.07 MB/s | Write Total: 60.97 GB | Disk Usage: 13.37 GB
Write Speed 217.70 MB/s | Write Total: 61.60 GB | Disk Usage: 14.02 GB
Write Speed 384.50 MB/s | Write Total: 62.73 GB | Disk Usage: 15.15 GB
Write Speed 404.66 MB/s | Write Total: 63.92 GB | Disk Usage: 16.35 GB
Write Speed 99.33 MB/s | Write Total: 64.21 GB | Disk Usage: 9.33 GB
Write Speed 286.22 MB/s | Write Total: 65.05 GB | Disk Usage: 10.18 GB
Write Speed 313.61 MB/s | Write Total: 65.96 GB | Disk Usage: 11.11 GB
Write Speed 362.82 MB/s | Write Total: 67.03 GB | Disk Usage: 12.18 GB
Write Speed 345.21 MB/s | Write Total: 68.04 GB | Disk Usage: 13.21 GB
Write Speed 315.72 MB/s | Write Total: 68.96 GB | Disk Usage: 14.14 GB
Write Speed 308.03 MB/s | Write Total: 69.87 GB | Disk Usage: 8.99 GB
Write Speed 83.76 MB/s | Write Total: 70.11 GB | Disk Usage: 9.35 GB
Write Speed 245.03 MB/s | Write Total: 70.83 GB | Disk Usage: 9.15 GB
Write Speed 386.66 MB/s | Write Total: 71.96 GB | Disk Usage: 9.29 GB
Write Speed 415.37 MB/s | Write Total: 73.18 GB | Disk Usage: 10.52 GB
Write Speed 325.77 MB/s | Write Total: 74.13 GB | Disk Usage: 11.49 GB
Write Speed 320.64 MB/s | Write Total: 75.07 GB | Disk Usage: 12.44 GB
Write Speed 302.97 MB/s | Write Total: 75.96 GB | Disk Usage: 9.29 GB
Write Speed 270.32 MB/s | Write Total: 76.75 GB | Disk Usage: 10.10 GB
Write Speed 302.38 MB/s | Write Total: 77.64 GB | Disk Usage: 10.99 GB
Write Speed 213.04 MB/s | Write Total: 78.26 GB | Disk Usage: 11.62 GB
Write Speed 315.19 MB/s | Write Total: 79.19 GB | Disk Usage: 12.55 GB
Write Speed 263.62 MB/s | Write Total: 79.96 GB | Disk Usage: 13.34 GB
Write Speed 341.40 MB/s | Write Total: 80.96 GB | Disk Usage: 14.35 GB
Write Speed 175.66 MB/s | Write Total: 81.47 GB | Disk Usage: 9.61 GB
Write Speed 228.67 MB/s | Write Total: 82.14 GB | Disk Usage: 10.28 GB
Write Speed 278.73 MB/s | Write Total: 82.96 GB | Disk Usage: 11.11 GB
Write Speed 297.78 MB/s | Write Total: 83.83 GB | Disk Usage: 12.00 GB
Write Speed 130.46 MB/s | Write Total: 84.21 GB | Disk Usage: 12.38 GB
Write Speed 253.38 MB/s | Write Total: 84.96 GB | Disk Usage: 13.13 GB
Write Speed 186.17 MB/s | Write Total: 85.50 GB | Disk Usage: 13.68 GB
Write Speed 154.90 MB/s | Write Total: 85.96 GB | Disk Usage: 10.21 GB
Write Speed 198.98 MB/s | Write Total: 86.54 GB | Disk Usage: 10.93 GB
Write Speed 262.05 MB/s | Write Total: 87.31 GB | Disk Usage: 9.68 GB
Write Speed 337.20 MB/s | Write Total: 88.29 GB | Disk Usage: 10.68 GB
Write Speed 434.23 MB/s | Write Total: 89.57 GB | Disk Usage: 11.96 GB
Write Speed 275.78 MB/s | Write Total: 90.37 GB | Disk Usage: 12.79 GB
Write Speed 238.92 MB/s | Write Total: 91.07 GB | Disk Usage: 9.45 GB
Write Speed 304.48 MB/s | Write Total: 91.97 GB | Disk Usage: 10.35 GB
Write Speed 336.54 MB/s | Write Total: 92.95 GB | Disk Usage: 11.35 GB
Write Speed 228.74 MB/s | Write Total: 93.62 GB | Disk Usage: 12.03 GB
Write Speed 282.28 MB/s | Write Total: 94.45 GB | Disk Usage: 12.86 GB
Write Speed 271.70 MB/s | Write Total: 95.25 GB | Disk Usage: 13.67 GB
Write Speed 283.99 MB/s | Write Total: 96.08 GB | Disk Usage: 14.50 GB
Write Speed 298.18 MB/s | Write Total: 96.95 GB | Disk Usage: 10.10 GB
Write Speed 265.53 MB/s | Write Total: 97.73 GB | Disk Usage: 10.89 GB
Write Speed 269.93 MB/s | Write Total: 98.52 GB | Disk Usage: 11.69 GB
Write Speed 284.25 MB/s | Write Total: 99.35 GB | Disk Usage: 12.53 GB
Write Speed 203.51 MB/s | Write Total: 99.95 GB | Disk Usage: 13.13 GB
Write Speed 341.53 MB/s | Write Total: 100.95 GB | Disk Usage: 14.14 GB
Write Speed 341.07 MB/s | Write Total: 101.95 GB | Disk Usage: 15.15 GB
Write Speed 70.23 MB/s | Write Total: 102.15 GB | Disk Usage: 9.34 GB
Write Speed 333.98 MB/s | Write Total: 103.13 GB | Disk Usage: 8.48 GB
Write Speed 289.51 MB/s | Write Total: 103.98 GB | Disk Usage: 9.34 GB
Write Speed 276.10 MB/s | Write Total: 104.79 GB | Disk Usage: 10.16 GB
Write Speed 147.35 MB/s | Write Total: 105.22 GB | Disk Usage: 10.60 GB
Write Speed 247.33 MB/s | Write Total: 105.95 GB | Disk Usage: 11.33 GB
Write Speed 289.44 MB/s | Write Total: 106.79 GB | Disk Usage: 12.19 GB
Write Speed 146.95 MB/s | Write Total: 107.23 GB | Disk Usage: 9.59 GB
Write Speed 246.08 MB/s | Write Total: 107.95 GB | Disk Usage: 10.32 GB
Write Speed 318.94 MB/s | Write Total: 108.88 GB | Disk Usage: 11.26 GB
Write Speed 192.74 MB/s | Write Total: 109.45 GB | Disk Usage: 11.84 GB
Write Speed 170.67 MB/s | Write Total: 109.95 GB | Disk Usage: 12.34 GB
Write Speed 308.49 MB/s | Write Total: 110.85 GB | Disk Usage: 13.25 GB
Write Speed 200.36 MB/s | Write Total: 111.44 GB | Disk Usage: 9.81 GB
Write Speed 375.37 MB/s | Write Total: 112.54 GB | Disk Usage: 10.92 GB
Write Speed 284.64 MB/s | Write Total: 113.37 GB | Disk Usage: 11.76 GB
Write Speed 295.09 MB/s | Write Total: 114.23 GB | Disk Usage: 12.64 GB
Write Speed 282.41 MB/s | Write Total: 115.06 GB | Disk Usage: 13.47 GB
Write Speed 380.56 MB/s | Write Total: 116.18 GB | Disk Usage: 14.60 GB
Write Speed 260.93 MB/s | Write Total: 116.94 GB | Disk Usage: 15.37 GB
Write Speed 17.74 MB/s | Write Total: 116.99 GB | Disk Usage: 9.15 GB
Write Speed 341.80 MB/s | Write Total: 117.99 GB | Disk Usage: 10.38 GB
Write Speed 309.94 MB/s | Write Total: 118.90 GB | Disk Usage: 9.28 GB
Write Speed 320.71 MB/s | Write Total: 119.84 GB | Disk Usage: 10.23 GB
Write Speed 116.21 MB/s | Write Total: 120.18 GB | Disk Usage: 10.58 GB
Write Speed 258.17 MB/s | Write Total: 120.94 GB | Disk Usage: 11.34 GB
Write Speed 147.48 MB/s | Write Total: 121.37 GB | Disk Usage: 11.78 GB
Write Speed 193.73 MB/s | Write Total: 121.94 GB | Disk Usage: 9.32 GB
Write Speed 341.07 MB/s | Write Total: 122.94 GB | Disk Usage: 10.33 GB
Write Speed 312.10 MB/s | Write Total: 123.85 GB | Disk Usage: 11.26 GB
Write Speed 215.80 MB/s | Write Total: 124.48 GB | Disk Usage: 11.90 GB
Write Speed 173.62 MB/s | Write Total: 124.99 GB | Disk Usage: 12.41 GB
Write Speed 387.65 MB/s | Write Total: 126.13 GB | Disk Usage: 13.57 GB
Write Speed 275.64 MB/s | Write Total: 126.94 GB | Disk Usage: 14.38 GB
Write Speed 228.67 MB/s | Write Total: 127.61 GB | Disk Usage: 10.01 GB
Write Speed 207.13 MB/s | Write Total: 128.21 GB | Disk Usage: 10.62 GB
Write Speed 246.21 MB/s | Write Total: 128.93 GB | Disk Usage: 11.35 GB
Write Speed 225.06 MB/s | Write Total: 129.59 GB | Disk Usage: 12.02 GB
Write Speed 246.28 MB/s | Write Total: 130.31 GB | Disk Usage: 12.74 GB
Write Speed 288.59 MB/s | Write Total: 131.16 GB | Disk Usage: 13.60 GB
Write Speed 264.08 MB/s | Write Total: 131.93 GB | Disk Usage: 14.38 GB
Write Speed 211.53 MB/s | Write Total: 132.55 GB | Disk Usage: 9.80 GB
Write Speed 255.21 MB/s | Write Total: 133.30 GB | Disk Usage: 10.68 GB
Write Speed 215.34 MB/s | Write Total: 133.93 GB | Disk Usage: 9.31 GB
Write Speed 302.78 MB/s | Write Total: 134.82 GB | Disk Usage: 10.21 GB
Write Speed 347.18 MB/s | Write Total: 135.84 GB | Disk Usage: 11.23 GB
Write Speed 193.53 MB/s | Write Total: 136.40 GB | Disk Usage: 11.81 GB
Write Speed 179.93 MB/s | Write Total: 136.93 GB | Disk Usage: 9.31 GB
Write Speed 341.21 MB/s | Write Total: 137.93 GB | Disk Usage: 10.32 GB
Write Speed 332.86 MB/s | Write Total: 138.91 GB | Disk Usage: 11.30 GB
Write Speed 273.87 MB/s | Write Total: 139.71 GB | Disk Usage: 12.12 GB
Write Speed 281.36 MB/s | Write Total: 140.53 GB | Disk Usage: 12.95 GB
Write Speed 288.52 MB/s | Write Total: 141.38 GB | Disk Usage: 13.81 GB
Write Speed 249.04 MB/s | Write Total: 142.11 GB | Disk Usage: 14.54 GB
Write Speed 209.16 MB/s | Write Total: 142.72 GB | Disk Usage: 10.11 GB
Write Speed 395.47 MB/s | Write Total: 143.88 GB | Disk Usage: 11.28 GB
Write Speed 159.24 MB/s | Write Total: 144.34 GB | Disk Usage: 11.75 GB
Write Speed 198.39 MB/s | Write Total: 144.93 GB | Disk Usage: 12.34 GB
Write Speed 341.47 MB/s | Write Total: 145.93 GB | Disk Usage: 13.35 GB
Write Speed 277.75 MB/s | Write Total: 146.74 GB | Disk Usage: 14.18 GB
Write Speed 319.99 MB/s | Write Total: 147.68 GB | Disk Usage: 15.12 GB
Write Speed 84.28 MB/s | Write Total: 147.92 GB | Disk Usage: 10.17 GB
Write Speed 142.22 MB/s | Write Total: 148.34 GB | Disk Usage: 10.70 GB
Write Speed 198.85 MB/s | Write Total: 148.92 GB | Disk Usage: 9.32 GB
Write Speed 340.94 MB/s | Write Total: 149.92 GB | Disk Usage: 10.33 GB
Write Speed 332.21 MB/s | Write Total: 150.90 GB | Disk Usage: 11.31 GB
Write Speed 254.89 MB/s | Write Total: 151.64 GB | Disk Usage: 12.08 GB
Write Speed 253.97 MB/s | Write Total: 152.39 GB | Disk Usage: 12.82 GB
Write Speed 186.04 MB/s | Write Total: 152.93 GB | Disk Usage: 10.34 GB
Write Speed 337.86 MB/s | Write Total: 153.92 GB | Disk Usage: 11.34 GB
Write Speed 334.31 MB/s | Write Total: 154.90 GB | Disk Usage: 12.33 GB
Write Speed 284.12 MB/s | Write Total: 155.73 GB | Disk Usage: 13.19 GB
Write Speed 207.52 MB/s | Write Total: 156.34 GB | Disk Usage: 13.80 GB
Write Speed 197.21 MB/s | Write Total: 156.92 GB | Disk Usage: 14.37 GB
Write Speed 182.56 MB/s | Write Total: 157.45 GB | Disk Usage: 10.92 GB
Write Speed 158.52 MB/s | Write Total: 157.92 GB | Disk Usage: 10.33 GB
Write Speed 341.34 MB/s | Write Total: 158.92 GB | Disk Usage: 11.34 GB
Write Speed 244.57 MB/s | Write Total: 159.63 GB | Disk Usage: 12.07 GB
Write Speed 240.89 MB/s | Write Total: 160.34 GB | Disk Usage: 12.79 GB
Write Speed 196.88 MB/s | Write Total: 160.92 GB | Disk Usage: 13.36 GB
Write Speed 275.12 MB/s | Write Total: 161.72 GB | Disk Usage: 14.18 GB
Write Speed 66.09 MB/s | Write Total: 161.92 GB | Disk Usage: 9.14 GB
Write Speed 208.70 MB/s | Write Total: 162.53 GB | Disk Usage: 9.88 GB
Write Speed 162.65 MB/s | Write Total: 163.01 GB | Disk Usage: 9.36 GB
Write Speed 399.15 MB/s | Write Total: 164.17 GB | Disk Usage: 9.54 GB
Write Speed 399.15 MB/s | Write Total: 165.34 GB | Disk Usage: 10.73 GB
Write Speed 196.03 MB/s | Write Total: 165.92 GB | Disk Usage: 11.31 GB
Write Speed 339.83 MB/s | Write Total: 166.91 GB | Disk Usage: 12.31 GB
Write Speed 263.69 MB/s | Write Total: 167.69 GB | Disk Usage: 10.07 GB
Write Speed 331.15 MB/s | Write Total: 168.66 GB | Disk Usage: 11.05 GB
Write Speed 227.10 MB/s | Write Total: 169.32 GB | Disk Usage: 11.72 GB
Write Speed 310.13 MB/s | Write Total: 170.23 GB | Disk Usage: 12.64 GB
Write Speed 241.88 MB/s | Write Total: 170.94 GB | Disk Usage: 13.35 GB
Write Speed 331.88 MB/s | Write Total: 171.91 GB | Disk Usage: 14.34 GB
Write Speed 177.43 MB/s | Write Total: 172.43 GB | Disk Usage: 9.67 GB
Write Speed 274.40 MB/s | Write Total: 173.23 GB | Disk Usage: 10.43 GB
Write Speed 323.67 MB/s | Write Total: 174.18 GB | Disk Usage: 11.39 GB
Write Speed 247.66 MB/s | Write Total: 174.91 GB | Disk Usage: 12.12 GB
Write Speed 341.47 MB/s | Write Total: 175.91 GB | Disk Usage: 13.13 GB
Write Speed 352.90 MB/s | Write Total: 176.94 GB | Disk Usage: 14.18 GB
Write Speed 299.95 MB/s | Write Total: 177.82 GB | Disk Usage: 15.07 GB
Write Speed 85.40 MB/s | Write Total: 178.07 GB | Disk Usage: 9.28 GB
Write Speed 315.78 MB/s | Write Total: 179.00 GB | Disk Usage: 10.38 GB
Write Speed 398.23 MB/s | Write Total: 180.16 GB | Disk Usage: 9.53 GB
Write Speed 253.38 MB/s | Write Total: 180.91 GB | Disk Usage: 10.29 GB
Write Speed 341.34 MB/s | Write Total: 181.91 GB | Disk Usage: 11.30 GB
Write Speed 341.21 MB/s | Write Total: 182.91 GB | Disk Usage: 12.31 GB
Write Speed 203.84 MB/s | Write Total: 183.50 GB | Disk Usage: 12.92 GB
Write Speed 137.03 MB/s | Write Total: 183.90 GB | Disk Usage: 10.29 GB
Write Speed 341.27 MB/s | Write Total: 184.90 GB | Disk Usage: 11.30 GB
Write Speed 341.21 MB/s | Write Total: 185.90 GB | Disk Usage: 12.31 GB
Write Speed 340.81 MB/s | Write Total: 186.90 GB | Disk Usage: 13.32 GB
Write Speed 321.10 MB/s | Write Total: 187.84 GB | Disk Usage: 14.28 GB
Write Speed 261.78 MB/s | Write Total: 188.61 GB | Disk Usage: 15.05 GB
Write Speed 135.13 MB/s | Write Total: 189.01 GB | Disk Usage: 11.45 GB
Write Speed 236.23 MB/s | Write Total: 189.70 GB | Disk Usage: 9.89 GB
Write Speed 410.51 MB/s | Write Total: 190.90 GB | Disk Usage: 11.11 GB
Write Speed 341.40 MB/s | Write Total: 191.90 GB | Disk Usage: 12.12 GB
Write Speed 151.68 MB/s | Write Total: 192.35 GB | Disk Usage: 12.57 GB
Write Speed 190.11 MB/s | Write Total: 192.90 GB | Disk Usage: 13.14 GB
Write Speed 366.43 MB/s | Write Total: 193.98 GB | Disk Usage: 14.22 GB
Write Speed 192.61 MB/s | Write Total: 194.54 GB | Disk Usage: 9.76 GB
Write Speed 220.73 MB/s | Write Total: 195.19 GB | Disk Usage: 10.51 GB
Write Speed 247.46 MB/s | Write Total: 195.91 GB | Disk Usage: 11.35 GB
Write Speed 441.71 MB/s | Write Total: 197.21 GB | Disk Usage: 10.64 GB
Write Speed 235.84 MB/s | Write Total: 197.90 GB | Disk Usage: 11.33 GB
Write Speed 341.27 MB/s | Write Total: 198.90 GB | Disk Usage: 12.35 GB
Write Speed 341.14 MB/s | Write Total: 199.90 GB | Disk Usage: 13.36 GB
Write Speed 146.63 MB/s | Write Total: 200.33 GB | Disk Usage: 9.75 GB
Write Speed 349.75 MB/s | Write Total: 201.35 GB | Disk Usage: 10.79 GB
Write Speed 235.57 MB/s | Write Total: 202.04 GB | Disk Usage: 11.48 GB
52512962             14400 ns/op            1685 B/op              10 allocs/op

另外，我嘗試在本機測試磁盤的寫入性能。結果顯示大約有 260 MB/s。綜合分析，bitcaskDB 基本可以將磁盤帶寬打滿。

dd if=/dev/zero of=/tmp/testfile bs=5G count=1 oflag=dsync
0+1 records in
0+1 records out
2147479552 bytes (2.1 GB) copied, 8.18893 s, 262 MB/s

其他測試

這裏測試了 bitcaskDB 分別執行順序讀寫、併發讀寫以及批量寫 4 KB 時的性能。

測試發現，批量比單次讀寫性能高，併發比批量讀寫性能高。

go test -bench=PutGet -benchtime=60s -count=3 -timeout=50m
goos: linux
goarch: amd64
pkg: github.com/wenzhang-dev/bitcaskDB/bench
cpu: Intel(R) Xeon(R) Gold 5318N CPU @ 2.10GHz
BenchmarkPutGet/put4K-8                  5331782   25259 ns/op   11795 B/op   21 allocs/op
BenchmarkPutGet/put4K-8                  5130870   25417 ns/op   11767 B/op   21 allocs/op
BenchmarkPutGet/put4K-8                  4898403   26676 ns/op   11742 B/op   21 allocs/op
BenchmarkPutGet/batchPut4K-8            10548615   15340 ns/op    1695 B/op   11 allocs/op
BenchmarkPutGet/batchPut4K-8             9220388   14278 ns/op    1694 B/op   11 allocs/op
BenchmarkPutGet/batchPut4K-8            10363459   15019 ns/op    1686 B/op   11 allocs/op
BenchmarkPutGet/get4K-8                  8812342    8076 ns/op   10119 B/op   10 allocs/op
BenchmarkPutGet/get4K-8                  7963098    7952 ns/op   10119 B/op   10 allocs/op
BenchmarkPutGet/get4K-8                  8480240    7997 ns/op   10119 B/op   10 allocs/op
BenchmarkPutGet/concurrentGet4K-8       17233309    4427 ns/op   10044 B/op    7 allocs/op
BenchmarkPutGet/concurrentGet4K-8       26745726    3681 ns/op   10044 B/op    7 allocs/op
BenchmarkPutGet/concurrentGet4K-8       29305041    3654 ns/op   10044 B/op    7 allocs/op
BenchmarkPutGet/concurrentPut4K-8        4558645   19829 ns/op    8340 B/op   18 allocs/op
BenchmarkPutGet/concurrentPut4K-8        4433334   18664 ns/op   10031 B/op   18 allocs/op
BenchmarkPutGet/concurrentPut4K-8        4366149   17031 ns/op    8175 B/op   17 allocs/op
BenchmarkPutGet/concurrentBatchPut4K-8   9443377   12520 ns/op    1527 B/op    9 allocs/op
BenchmarkPutGet/concurrentBatchPut4K-8  11338162   12429 ns/op    1517 B/op    9 allocs/op
BenchmarkPutGet/concurrentBatchPut4K-8  11394081   12101 ns/op    1510 B/op    9 allocs/op
PASS
ok   github.com/wenzhang-dev/bitcaskDB/bench 2310.401s

結論

Bitcask，性能並不差，且極具場景價值！

通過本次性能評測，可以得出幾個非常重要的結論：

Bitcask 類型引擎在高併發、隨機讀寫場景下，性能絲毫不遜色
對於小 Key 場景，內存佔用空間並不算高
磁盤空間佔用上，BitcaskDB 沒有提供壓縮，空間開銷大於 LSM 類引擎，但仍優於 BoltDB
GC 暫停時間可控
寫入帶寬表現亮眼

適用場景建議

冷熱分離型緩存，尤其是磁盤成本敏感、數據量超大的場景
寫多讀多、或寫多讀少的高併發緩存系統
容忍無強事務、無多列索引、無範圍查詢的 KV 場景

本文由 Readfog 進行 AMP 轉碼，版權歸原作者所有。
來源：https://mp.weixin.qq.com/s/PYgMy7DTogOSs3ZTDwHRYg

簡介