diff options
| author | bndw <ben@bdw.to> | 2026-02-14 18:56:19 -0800 |
|---|---|---|
| committer | bndw <ben@bdw.to> | 2026-02-14 18:56:19 -0800 |
| commit | 7a5d5a53e5d6878f38382c4d35f644e088d318d2 (patch) | |
| tree | 5f8ba0bee800a5998ec4167c47e4adc6602243e1 /BENCHMARKS.md | |
| parent | 7fba76d7e4e63e0c29da81d6be43330743af1aaf (diff) | |
feat: add library comparison benchmarks with build tag isolation
Add comprehensive benchmarks comparing NWIO against nbd-wtf/go-nostr and
fiatjaf.com/nostr across event operations, signing, verification, and filtering.
Use build tag 'benchcmp' to prevent competitor libraries from polluting module
dependencies - they're only downloaded when explicitly running comparison tests.
Diffstat (limited to 'BENCHMARKS.md')
| -rw-r--r-- | BENCHMARKS.md | 129 |
1 files changed, 129 insertions, 0 deletions
diff --git a/BENCHMARKS.md b/BENCHMARKS.md new file mode 100644 index 0000000..14a861c --- /dev/null +++ b/BENCHMARKS.md | |||
| @@ -0,0 +1,129 @@ | |||
| 1 | # Nostr Library Benchmarks | ||
| 2 | |||
| 3 | This directory contains comprehensive benchmarks comparing three popular Go Nostr libraries: | ||
| 4 | |||
| 5 | - **NWIO** (`northwest.io/nostr`) - This library | ||
| 6 | - **NBD** (`github.com/nbd-wtf/go-nostr`) - Popular community library | ||
| 7 | - **Fiat** (`fiatjaf.com/nostr`) - Original implementation by Fiatjaf | ||
| 8 | |||
| 9 | ## Benchmark Categories | ||
| 10 | |||
| 11 | ### Event Operations | ||
| 12 | - **Unmarshal**: Parsing JSON into Event struct | ||
| 13 | - **Marshal**: Serializing Event struct to JSON | ||
| 14 | - **Serialize**: Canonical serialization for ID computation | ||
| 15 | - **ComputeID**: Computing event ID hash | ||
| 16 | - **Sign**: Signing events with private key | ||
| 17 | - **Verify**: Verifying event signatures | ||
| 18 | |||
| 19 | ### Key Operations | ||
| 20 | - **GenerateKey**: Generating new private keys | ||
| 21 | |||
| 22 | ### Filter Operations | ||
| 23 | - **FilterMatch**: Simple filter matching (kind, author) | ||
| 24 | - **FilterMatchComplex**: Complex filter matching (with tags, prefix matching) | ||
| 25 | |||
| 26 | ## Running Benchmarks | ||
| 27 | |||
| 28 | **Important**: The comparison benchmarks require the `benchcmp` build tag to avoid polluting the module dependencies with competitor libraries. | ||
| 29 | |||
| 30 | ### Quick Start | ||
| 31 | |||
| 32 | Run all benchmarks (automatically handles dependencies): | ||
| 33 | ```bash | ||
| 34 | ./run_benchmarks.sh | ||
| 35 | ``` | ||
| 36 | |||
| 37 | Or manually: | ||
| 38 | ```bash | ||
| 39 | # First, get the comparison dependencies | ||
| 40 | go get -tags=benchcmp -t ./... | ||
| 41 | |||
| 42 | # Then run the benchmarks | ||
| 43 | go test -tags=benchcmp -bench=. -benchmem -benchtime=1s | ||
| 44 | ``` | ||
| 45 | |||
| 46 | ### Specific Benchmark Groups | ||
| 47 | |||
| 48 | Event unmarshaling: | ||
| 49 | ```bash | ||
| 50 | go test -tags=benchcmp -bench=BenchmarkEventUnmarshal -benchmem | ||
| 51 | ``` | ||
| 52 | |||
| 53 | Event signing: | ||
| 54 | ```bash | ||
| 55 | go test -tags=benchcmp -bench=BenchmarkEventSign -benchmem | ||
| 56 | ``` | ||
| 57 | |||
| 58 | Event verification: | ||
| 59 | ```bash | ||
| 60 | go test -tags=benchcmp -bench=BenchmarkEventVerify -benchmem | ||
| 61 | ``` | ||
| 62 | |||
| 63 | Filter matching: | ||
| 64 | ```bash | ||
| 65 | go test -tags=benchcmp -bench=BenchmarkFilterMatch -benchmem | ||
| 66 | ``` | ||
| 67 | |||
| 68 | ### Compare Single Library | ||
| 69 | |||
| 70 | NWIO only: | ||
| 71 | ```bash | ||
| 72 | go test -tags=benchcmp -bench='.*_NWIO' -benchmem | ||
| 73 | ``` | ||
| 74 | |||
| 75 | NBD only: | ||
| 76 | ```bash | ||
| 77 | go test -tags=benchcmp -bench='.*_NBD' -benchmem | ||
| 78 | ``` | ||
| 79 | |||
| 80 | Fiat only: | ||
| 81 | ```bash | ||
| 82 | go test -tags=benchcmp -bench='.*_Fiat' -benchmem | ||
| 83 | ``` | ||
| 84 | |||
| 85 | ## Analyzing Results | ||
| 86 | |||
| 87 | Use `benchstat` for statistical analysis: | ||
| 88 | |||
| 89 | ```bash | ||
| 90 | # Install benchstat | ||
| 91 | go install golang.org/x/perf/cmd/benchstat@latest | ||
| 92 | |||
| 93 | # Run benchmarks multiple times and compare | ||
| 94 | go test -tags=benchcmp -bench=. -benchmem -count=10 > results.txt | ||
| 95 | benchstat results.txt | ||
| 96 | ``` | ||
| 97 | |||
| 98 | Compare two specific libraries: | ||
| 99 | ```bash | ||
| 100 | go test -tags=benchcmp -bench='.*_NWIO' -benchmem -count=10 > nwio.txt | ||
| 101 | go test -tags=benchcmp -bench='.*_NBD' -benchmem -count=10 > nbd.txt | ||
| 102 | benchstat nwio.txt nbd.txt | ||
| 103 | ``` | ||
| 104 | |||
| 105 | ## Understanding the Output | ||
| 106 | |||
| 107 | Example output: | ||
| 108 | ``` | ||
| 109 | BenchmarkEventSign_NWIO-24 50000 35421 ns/op 1024 B/op 12 allocs/op | ||
| 110 | ``` | ||
| 111 | |||
| 112 | - `50000`: Number of iterations | ||
| 113 | - `35421 ns/op`: Nanoseconds per operation (lower is better) | ||
| 114 | - `1024 B/op`: Bytes allocated per operation (lower is better) | ||
| 115 | - `12 allocs/op`: Memory allocations per operation (lower is better) | ||
| 116 | |||
| 117 | ## Performance Tips | ||
| 118 | |||
| 119 | 1. **Event Unmarshaling**: Critical for relay implementations | ||
| 120 | 2. **Event Signing**: Important for client implementations | ||
| 121 | 3. **Event Verification**: Critical for all implementations | ||
| 122 | 4. **Filter Matching**: Important for relay implementations with many subscriptions | ||
| 123 | |||
| 124 | ## Notes | ||
| 125 | |||
| 126 | - All benchmarks use realistic event data | ||
| 127 | - Benchmarks run with default Go test timeout | ||
| 128 | - Results may vary based on hardware and system load | ||
| 129 | - Use `-benchtime=5s` for more stable results on noisy systems | ||
