1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
|
# Benchmark Results Summary
Comparison of three Go Nostr libraries: **NWIO** (code.northwest.io/nostr), **NBD** (github.com/nbd-wtf/go-nostr), and **Fiat** (fiatjaf.com/nostr)
## Quick Performance Overview
### 🏆 Winners by Category
| Operation | Winner | Performance |
|-----------|--------|-------------|
| **Event Unmarshal** | NWIO/Fiat | ~2.5 µs (tied) |
| **Event Marshal** | NWIO | 1.79 µs (fastest, least memory) |
| **Event Serialize** | NBD | 129 ns (3x faster than NWIO) |
| **Compute ID** | Fiat | 276 ns (2x faster than NWIO) |
| **Generate Key** | NBD | 470 ns (80x faster!) |
| **Event Sign** | NBD/Fiat | ~59 µs (2x faster than NWIO) |
| **Event Verify** | NWIO | 99.7 µs (slightly faster) |
| **Filter Match** | NWIO | 7.1 ns (2x faster than Fiat) |
| **Filter Complex** | NWIO | 30.9 ns (fastest) |
## Detailed Results
### Event Unmarshaling (JSON → Event)
```
NWIO: 2,541 ns/op 888 B/op 17 allocs/op ⭐ FASTEST, LOW MEMORY
NBD: 2,832 ns/op 944 B/op 13 allocs/op
Fiat: 2,545 ns/op 752 B/op 10 allocs/op ⭐ LEAST MEMORY
```
**Analysis**: All three are very competitive. NWIO and Fiat are effectively tied. Fiat uses least memory.
### Event Marshaling (Event → JSON)
```
NWIO: 1,790 ns/op 1,010 B/op 3 allocs/op ⭐ FASTEST, LEAST ALLOCS
NBD: 1,819 ns/op 1,500 B/op 6 allocs/op
Fiat: 1,971 ns/op 2,254 B/op 13 allocs/op
```
**Analysis**: NWIO is fastest with minimal allocations. Significant memory advantage over competitors.
### Event Serialization (for ID computation)
```
NWIO: 391 ns/op 360 B/op 7 allocs/op
NBD: 129 ns/op 208 B/op 2 allocs/op ⭐ FASTEST, 3x faster
Fiat: 161 ns/op 400 B/op 3 allocs/op
```
**Analysis**: NBD dominates here with optimized serialization. NWIO has room for improvement.
### Event ID Computation
```
NWIO: 608 ns/op 488 B/op 9 allocs/op
NBD: 302 ns/op 336 B/op 4 allocs/op
Fiat: 276 ns/op 400 B/op 3 allocs/op ⭐ FASTEST
```
**Analysis**: NBD and Fiat are 2x faster. NWIO should optimize ID computation path.
### Key Generation
```
NWIO: 37,689 ns/op 208 B/op 4 allocs/op
NBD: 470 ns/op 369 B/op 8 allocs/op ⭐ FASTEST, 80x faster!
Fiat: 25,375 ns/op 272 B/op 5 allocs/op
```
**Analysis**: ⚠️ NWIO is significantly slower. NBD appears to use a different key generation strategy. This is the biggest performance gap.
### Event Signing
```
NWIO: 129,854 ns/op 2,363 B/op 42 allocs/op
NBD: 59,069 ns/op 2,112 B/op 35 allocs/op ⭐ TIED FASTEST
Fiat: 58,572 ns/op 1,760 B/op 29 allocs/op ⭐ LEAST MEMORY
```
**Analysis**: NBD and Fiat are 2x faster. NWIO has more allocations in signing path.
### Event Verification
```
NWIO: 99,744 ns/op 953 B/op 19 allocs/op ⭐ FASTEST
NBD: 105,995 ns/op 624 B/op 11 allocs/op ⭐ LEAST MEMORY
Fiat: 103,744 ns/op 640 B/op 9 allocs/op
```
**Analysis**: NWIO is slightly faster (6% faster than others). Very competitive across all three.
### Filter Matching (Simple)
```
NWIO: 7.1 ns/op 0 B/op 0 allocs/op ⭐ FASTEST, 2x faster
NBD: 10.8 ns/op 0 B/op 0 allocs/op
Fiat: 16.4 ns/op 0 B/op 0 allocs/op
```
**Analysis**: NWIO excels at filter matching! Zero allocations across all libraries.
### Filter Matching (Complex with Tags)
```
NWIO: 30.9 ns/op 0 B/op 0 allocs/op ⭐ FASTEST
NBD: 33.4 ns/op 0 B/op 0 allocs/op
Fiat: 42.6 ns/op 0 B/op 0 allocs/op
```
**Analysis**: NWIO maintains lead in complex filters. Important for relay implementations.
## Optimization Opportunities for NWIO
### High Priority 🔴
1. **Key Generation** - 80x slower than NBD
- Current: 37.7 µs
- Target: ~500 ns (similar to NBD)
- Impact: Critical for client applications
2. **Event Signing** - 2x slower than competitors
- Current: 130 µs
- Target: ~60 µs (match NBD/Fiat)
- Impact: High for client applications
### Medium Priority 🟡
3. **Event Serialization** - 3x slower than NBD
- Current: 391 ns
- Target: ~130 ns (match NBD)
- Impact: Used in ID computation
4. **ID Computation** - 2x slower than competitors
- Current: 608 ns
- Target: ~280 ns (match Fiat)
- Impact: Affects every event processing
## Current Strengths of NWIO ✅
1. **Filter Matching** - 2x faster than Fiat, fastest overall
2. **Event Marshaling** - Fastest with minimal allocations
3. **Event Verification** - Slightly faster than competitors
4. **Memory Efficiency** - Competitive or better in most operations
## Recommendations
### For Relay Implementations
- **NWIO excels**: Best filter matching performance
- All three are competitive for event parsing/verification
### For Client Implementations
- **NBD/Fiat preferred**: Much faster key generation and signing
- NWIO needs optimization in crypto operations
### Overall Assessment
- **NWIO**: Best for relay/filter-heavy workloads
- **NBD**: Most balanced, excellent crypto performance
- **Fiat**: Good all-around, lowest memory in some operations
## Running Your Own Benchmarks
```bash
# Run all benchmarks
./run_benchmarks.sh
# Compare specific operations
go test -bench=BenchmarkEventSign -benchmem comparison_bench_test.go
go test -bench=BenchmarkFilterMatch -benchmem comparison_bench_test.go
# Statistical analysis with benchstat
go test -bench=. -count=10 comparison_bench_test.go > results.txt
benchstat results.txt
```
|
