summaryrefslogtreecommitdiffstats
path: root/BENCHMARKS.md
diff options
context:
space:
mode:
authorbndw <ben@bdw.to>2026-02-14 18:56:19 -0800
committerbndw <ben@bdw.to>2026-02-14 18:56:19 -0800
commit7a5d5a53e5d6878f38382c4d35f644e088d318d2 (patch)
tree5f8ba0bee800a5998ec4167c47e4adc6602243e1 /BENCHMARKS.md
parent7fba76d7e4e63e0c29da81d6be43330743af1aaf (diff)
feat: add library comparison benchmarks with build tag isolation
Add comprehensive benchmarks comparing NWIO against nbd-wtf/go-nostr and fiatjaf.com/nostr across event operations, signing, verification, and filtering. Use build tag 'benchcmp' to prevent competitor libraries from polluting module dependencies - they're only downloaded when explicitly running comparison tests.
Diffstat (limited to 'BENCHMARKS.md')
-rw-r--r--BENCHMARKS.md129
1 files changed, 129 insertions, 0 deletions
diff --git a/BENCHMARKS.md b/BENCHMARKS.md
new file mode 100644
index 0000000..14a861c
--- /dev/null
+++ b/BENCHMARKS.md
@@ -0,0 +1,129 @@
1# Nostr Library Benchmarks
2
3This directory contains comprehensive benchmarks comparing three popular Go Nostr libraries:
4
5- **NWIO** (`northwest.io/nostr`) - This library
6- **NBD** (`github.com/nbd-wtf/go-nostr`) - Popular community library
7- **Fiat** (`fiatjaf.com/nostr`) - Original implementation by Fiatjaf
8
9## Benchmark Categories
10
11### Event Operations
12- **Unmarshal**: Parsing JSON into Event struct
13- **Marshal**: Serializing Event struct to JSON
14- **Serialize**: Canonical serialization for ID computation
15- **ComputeID**: Computing event ID hash
16- **Sign**: Signing events with private key
17- **Verify**: Verifying event signatures
18
19### Key Operations
20- **GenerateKey**: Generating new private keys
21
22### Filter Operations
23- **FilterMatch**: Simple filter matching (kind, author)
24- **FilterMatchComplex**: Complex filter matching (with tags, prefix matching)
25
26## Running Benchmarks
27
28**Important**: The comparison benchmarks require the `benchcmp` build tag to avoid polluting the module dependencies with competitor libraries.
29
30### Quick Start
31
32Run all benchmarks (automatically handles dependencies):
33```bash
34./run_benchmarks.sh
35```
36
37Or manually:
38```bash
39# First, get the comparison dependencies
40go get -tags=benchcmp -t ./...
41
42# Then run the benchmarks
43go test -tags=benchcmp -bench=. -benchmem -benchtime=1s
44```
45
46### Specific Benchmark Groups
47
48Event unmarshaling:
49```bash
50go test -tags=benchcmp -bench=BenchmarkEventUnmarshal -benchmem
51```
52
53Event signing:
54```bash
55go test -tags=benchcmp -bench=BenchmarkEventSign -benchmem
56```
57
58Event verification:
59```bash
60go test -tags=benchcmp -bench=BenchmarkEventVerify -benchmem
61```
62
63Filter matching:
64```bash
65go test -tags=benchcmp -bench=BenchmarkFilterMatch -benchmem
66```
67
68### Compare Single Library
69
70NWIO only:
71```bash
72go test -tags=benchcmp -bench='.*_NWIO' -benchmem
73```
74
75NBD only:
76```bash
77go test -tags=benchcmp -bench='.*_NBD' -benchmem
78```
79
80Fiat only:
81```bash
82go test -tags=benchcmp -bench='.*_Fiat' -benchmem
83```
84
85## Analyzing Results
86
87Use `benchstat` for statistical analysis:
88
89```bash
90# Install benchstat
91go install golang.org/x/perf/cmd/benchstat@latest
92
93# Run benchmarks multiple times and compare
94go test -tags=benchcmp -bench=. -benchmem -count=10 > results.txt
95benchstat results.txt
96```
97
98Compare two specific libraries:
99```bash
100go test -tags=benchcmp -bench='.*_NWIO' -benchmem -count=10 > nwio.txt
101go test -tags=benchcmp -bench='.*_NBD' -benchmem -count=10 > nbd.txt
102benchstat nwio.txt nbd.txt
103```
104
105## Understanding the Output
106
107Example output:
108```
109BenchmarkEventSign_NWIO-24 50000 35421 ns/op 1024 B/op 12 allocs/op
110```
111
112- `50000`: Number of iterations
113- `35421 ns/op`: Nanoseconds per operation (lower is better)
114- `1024 B/op`: Bytes allocated per operation (lower is better)
115- `12 allocs/op`: Memory allocations per operation (lower is better)
116
117## Performance Tips
118
1191. **Event Unmarshaling**: Critical for relay implementations
1202. **Event Signing**: Important for client implementations
1213. **Event Verification**: Critical for all implementations
1224. **Filter Matching**: Important for relay implementations with many subscriptions
123
124## Notes
125
126- All benchmarks use realistic event data
127- Benchmarks run with default Go test timeout
128- Results may vary based on hardware and system load
129- Use `-benchtime=5s` for more stable results on noisy systems