From 9d20d2281f4698024b7be67d1b86178b4e8e2484 Mon Sep 17 00:00:00 2001 From: Clawd Date: Mon, 16 Feb 2026 12:12:37 -0800 Subject: Clean up project structure - Add README.md - Move benchmark files to benchmarks/ - Move PLAN.md to .claude/ - Add .gitignore --- BENCHMARK_SUMMARY.md | 154 --------------------------------------------------- 1 file changed, 154 deletions(-) delete mode 100644 BENCHMARK_SUMMARY.md (limited to 'BENCHMARK_SUMMARY.md') diff --git a/BENCHMARK_SUMMARY.md b/BENCHMARK_SUMMARY.md deleted file mode 100644 index 2e087a3..0000000 --- a/BENCHMARK_SUMMARY.md +++ /dev/null @@ -1,154 +0,0 @@ -# Benchmark Results Summary - -Comparison of three Go Nostr libraries: **NWIO** (code.northwest.io/nostr), **NBD** (github.com/nbd-wtf/go-nostr), and **Fiat** (fiatjaf.com/nostr) - -## Quick Performance Overview - -### 🏆 Winners by Category - -| Operation | Winner | Performance | -|-----------|--------|-------------| -| **Event Unmarshal** | NWIO/Fiat | ~2.5 µs (tied) | -| **Event Marshal** | NWIO | 1.79 µs (fastest, least memory) | -| **Event Serialize** | NBD | 129 ns (3x faster than NWIO) | -| **Compute ID** | Fiat | 276 ns (2x faster than NWIO) | -| **Generate Key** | NBD | 470 ns (80x faster!) | -| **Event Sign** | NBD/Fiat | ~59 µs (2x faster than NWIO) | -| **Event Verify** | NWIO | 99.7 µs (slightly faster) | -| **Filter Match** | NWIO | 7.1 ns (2x faster than Fiat) | -| **Filter Complex** | NWIO | 30.9 ns (fastest) | - -## Detailed Results - -### Event Unmarshaling (JSON → Event) -``` -NWIO: 2,541 ns/op 888 B/op 17 allocs/op ⭐ FASTEST, LOW MEMORY -NBD: 2,832 ns/op 944 B/op 13 allocs/op -Fiat: 2,545 ns/op 752 B/op 10 allocs/op ⭐ LEAST MEMORY -``` -**Analysis**: All three are very competitive. NWIO and Fiat are effectively tied. Fiat uses least memory. - -### Event Marshaling (Event → JSON) -``` -NWIO: 1,790 ns/op 1,010 B/op 3 allocs/op ⭐ FASTEST, LEAST ALLOCS -NBD: 1,819 ns/op 1,500 B/op 6 allocs/op -Fiat: 1,971 ns/op 2,254 B/op 13 allocs/op -``` -**Analysis**: NWIO is fastest with minimal allocations. Significant memory advantage over competitors. - -### Event Serialization (for ID computation) -``` -NWIO: 391 ns/op 360 B/op 7 allocs/op -NBD: 129 ns/op 208 B/op 2 allocs/op ⭐ FASTEST, 3x faster -Fiat: 161 ns/op 400 B/op 3 allocs/op -``` -**Analysis**: NBD dominates here with optimized serialization. NWIO has room for improvement. - -### Event ID Computation -``` -NWIO: 608 ns/op 488 B/op 9 allocs/op -NBD: 302 ns/op 336 B/op 4 allocs/op -Fiat: 276 ns/op 400 B/op 3 allocs/op ⭐ FASTEST -``` -**Analysis**: NBD and Fiat are 2x faster. NWIO should optimize ID computation path. - -### Key Generation -``` -NWIO: 37,689 ns/op 208 B/op 4 allocs/op -NBD: 470 ns/op 369 B/op 8 allocs/op ⭐ FASTEST, 80x faster! -Fiat: 25,375 ns/op 272 B/op 5 allocs/op -``` -**Analysis**: ⚠️ NWIO is significantly slower. NBD appears to use a different key generation strategy. This is the biggest performance gap. - -### Event Signing -``` -NWIO: 129,854 ns/op 2,363 B/op 42 allocs/op -NBD: 59,069 ns/op 2,112 B/op 35 allocs/op ⭐ TIED FASTEST -Fiat: 58,572 ns/op 1,760 B/op 29 allocs/op ⭐ LEAST MEMORY -``` -**Analysis**: NBD and Fiat are 2x faster. NWIO has more allocations in signing path. - -### Event Verification -``` -NWIO: 99,744 ns/op 953 B/op 19 allocs/op ⭐ FASTEST -NBD: 105,995 ns/op 624 B/op 11 allocs/op ⭐ LEAST MEMORY -Fiat: 103,744 ns/op 640 B/op 9 allocs/op -``` -**Analysis**: NWIO is slightly faster (6% faster than others). Very competitive across all three. - -### Filter Matching (Simple) -``` -NWIO: 7.1 ns/op 0 B/op 0 allocs/op ⭐ FASTEST, 2x faster -NBD: 10.8 ns/op 0 B/op 0 allocs/op -Fiat: 16.4 ns/op 0 B/op 0 allocs/op -``` -**Analysis**: NWIO excels at filter matching! Zero allocations across all libraries. - -### Filter Matching (Complex with Tags) -``` -NWIO: 30.9 ns/op 0 B/op 0 allocs/op ⭐ FASTEST -NBD: 33.4 ns/op 0 B/op 0 allocs/op -Fiat: 42.6 ns/op 0 B/op 0 allocs/op -``` -**Analysis**: NWIO maintains lead in complex filters. Important for relay implementations. - -## Optimization Opportunities for NWIO - -### High Priority 🔴 -1. **Key Generation** - 80x slower than NBD - - Current: 37.7 µs - - Target: ~500 ns (similar to NBD) - - Impact: Critical for client applications - -2. **Event Signing** - 2x slower than competitors - - Current: 130 µs - - Target: ~60 µs (match NBD/Fiat) - - Impact: High for client applications - -### Medium Priority 🟡 -3. **Event Serialization** - 3x slower than NBD - - Current: 391 ns - - Target: ~130 ns (match NBD) - - Impact: Used in ID computation - -4. **ID Computation** - 2x slower than competitors - - Current: 608 ns - - Target: ~280 ns (match Fiat) - - Impact: Affects every event processing - -## Current Strengths of NWIO ✅ - -1. **Filter Matching** - 2x faster than Fiat, fastest overall -2. **Event Marshaling** - Fastest with minimal allocations -3. **Event Verification** - Slightly faster than competitors -4. **Memory Efficiency** - Competitive or better in most operations - -## Recommendations - -### For Relay Implementations -- **NWIO excels**: Best filter matching performance -- All three are competitive for event parsing/verification - -### For Client Implementations -- **NBD/Fiat preferred**: Much faster key generation and signing -- NWIO needs optimization in crypto operations - -### Overall Assessment -- **NWIO**: Best for relay/filter-heavy workloads -- **NBD**: Most balanced, excellent crypto performance -- **Fiat**: Good all-around, lowest memory in some operations - -## Running Your Own Benchmarks - -```bash -# Run all benchmarks -./run_benchmarks.sh - -# Compare specific operations -go test -bench=BenchmarkEventSign -benchmem comparison_bench_test.go -go test -bench=BenchmarkFilterMatch -benchmem comparison_bench_test.go - -# Statistical analysis with benchstat -go test -bench=. -count=10 comparison_bench_test.go > results.txt -benchstat results.txt -``` -- cgit v1.2.3