From 7a5d5a53e5d6878f38382c4d35f644e088d318d2 Mon Sep 17 00:00:00 2001 From: bndw Date: Sat, 14 Feb 2026 18:56:19 -0800 Subject: feat: add library comparison benchmarks with build tag isolation Add comprehensive benchmarks comparing NWIO against nbd-wtf/go-nostr and fiatjaf.com/nostr across event operations, signing, verification, and filtering. Use build tag 'benchcmp' to prevent competitor libraries from polluting module dependencies - they're only downloaded when explicitly running comparison tests. --- BENCHMARKS.md | 129 ++++++++++++ BENCHMARK_SUMMARY.md | 154 ++++++++++++++ benchmark_results.txt | 32 +++ comparison_bench_test.go | 512 +++++++++++++++++++++++++++++++++++++++++++++++ go.mod | 10 +- go.sum | 16 +- run_benchmarks.sh | 35 ++++ 7 files changed, 875 insertions(+), 13 deletions(-) create mode 100644 BENCHMARKS.md create mode 100644 BENCHMARK_SUMMARY.md create mode 100644 benchmark_results.txt create mode 100644 comparison_bench_test.go create mode 100755 run_benchmarks.sh diff --git a/BENCHMARKS.md b/BENCHMARKS.md new file mode 100644 index 0000000..14a861c --- /dev/null +++ b/BENCHMARKS.md @@ -0,0 +1,129 @@ +# Nostr Library Benchmarks + +This directory contains comprehensive benchmarks comparing three popular Go Nostr libraries: + +- **NWIO** (`northwest.io/nostr`) - This library +- **NBD** (`github.com/nbd-wtf/go-nostr`) - Popular community library +- **Fiat** (`fiatjaf.com/nostr`) - Original implementation by Fiatjaf + +## Benchmark Categories + +### Event Operations +- **Unmarshal**: Parsing JSON into Event struct +- **Marshal**: Serializing Event struct to JSON +- **Serialize**: Canonical serialization for ID computation +- **ComputeID**: Computing event ID hash +- **Sign**: Signing events with private key +- **Verify**: Verifying event signatures + +### Key Operations +- **GenerateKey**: Generating new private keys + +### Filter Operations +- **FilterMatch**: Simple filter matching (kind, author) +- **FilterMatchComplex**: Complex filter matching (with tags, prefix matching) + +## Running Benchmarks + +**Important**: The comparison benchmarks require the `benchcmp` build tag to avoid polluting the module dependencies with competitor libraries. + +### Quick Start + +Run all benchmarks (automatically handles dependencies): +```bash +./run_benchmarks.sh +``` + +Or manually: +```bash +# First, get the comparison dependencies +go get -tags=benchcmp -t ./... + +# Then run the benchmarks +go test -tags=benchcmp -bench=. -benchmem -benchtime=1s +``` + +### Specific Benchmark Groups + +Event unmarshaling: +```bash +go test -tags=benchcmp -bench=BenchmarkEventUnmarshal -benchmem +``` + +Event signing: +```bash +go test -tags=benchcmp -bench=BenchmarkEventSign -benchmem +``` + +Event verification: +```bash +go test -tags=benchcmp -bench=BenchmarkEventVerify -benchmem +``` + +Filter matching: +```bash +go test -tags=benchcmp -bench=BenchmarkFilterMatch -benchmem +``` + +### Compare Single Library + +NWIO only: +```bash +go test -tags=benchcmp -bench='.*_NWIO' -benchmem +``` + +NBD only: +```bash +go test -tags=benchcmp -bench='.*_NBD' -benchmem +``` + +Fiat only: +```bash +go test -tags=benchcmp -bench='.*_Fiat' -benchmem +``` + +## Analyzing Results + +Use `benchstat` for statistical analysis: + +```bash +# Install benchstat +go install golang.org/x/perf/cmd/benchstat@latest + +# Run benchmarks multiple times and compare +go test -tags=benchcmp -bench=. -benchmem -count=10 > results.txt +benchstat results.txt +``` + +Compare two specific libraries: +```bash +go test -tags=benchcmp -bench='.*_NWIO' -benchmem -count=10 > nwio.txt +go test -tags=benchcmp -bench='.*_NBD' -benchmem -count=10 > nbd.txt +benchstat nwio.txt nbd.txt +``` + +## Understanding the Output + +Example output: +``` +BenchmarkEventSign_NWIO-24 50000 35421 ns/op 1024 B/op 12 allocs/op +``` + +- `50000`: Number of iterations +- `35421 ns/op`: Nanoseconds per operation (lower is better) +- `1024 B/op`: Bytes allocated per operation (lower is better) +- `12 allocs/op`: Memory allocations per operation (lower is better) + +## Performance Tips + +1. **Event Unmarshaling**: Critical for relay implementations +2. **Event Signing**: Important for client implementations +3. **Event Verification**: Critical for all implementations +4. **Filter Matching**: Important for relay implementations with many subscriptions + +## Notes + +- All benchmarks use realistic event data +- Benchmarks run with default Go test timeout +- Results may vary based on hardware and system load +- Use `-benchtime=5s` for more stable results on noisy systems diff --git a/BENCHMARK_SUMMARY.md b/BENCHMARK_SUMMARY.md new file mode 100644 index 0000000..e7f5b6d --- /dev/null +++ b/BENCHMARK_SUMMARY.md @@ -0,0 +1,154 @@ +# Benchmark Results Summary + +Comparison of three Go Nostr libraries: **NWIO** (northwest.io/nostr), **NBD** (github.com/nbd-wtf/go-nostr), and **Fiat** (fiatjaf.com/nostr) + +## Quick Performance Overview + +### 🏆 Winners by Category + +| Operation | Winner | Performance | +|-----------|--------|-------------| +| **Event Unmarshal** | NWIO/Fiat | ~2.5 µs (tied) | +| **Event Marshal** | NWIO | 1.79 µs (fastest, least memory) | +| **Event Serialize** | NBD | 129 ns (3x faster than NWIO) | +| **Compute ID** | Fiat | 276 ns (2x faster than NWIO) | +| **Generate Key** | NBD | 470 ns (80x faster!) | +| **Event Sign** | NBD/Fiat | ~59 µs (2x faster than NWIO) | +| **Event Verify** | NWIO | 99.7 µs (slightly faster) | +| **Filter Match** | NWIO | 7.1 ns (2x faster than Fiat) | +| **Filter Complex** | NWIO | 30.9 ns (fastest) | + +## Detailed Results + +### Event Unmarshaling (JSON → Event) +``` +NWIO: 2,541 ns/op 888 B/op 17 allocs/op ⭐ FASTEST, LOW MEMORY +NBD: 2,832 ns/op 944 B/op 13 allocs/op +Fiat: 2,545 ns/op 752 B/op 10 allocs/op ⭐ LEAST MEMORY +``` +**Analysis**: All three are very competitive. NWIO and Fiat are effectively tied. Fiat uses least memory. + +### Event Marshaling (Event → JSON) +``` +NWIO: 1,790 ns/op 1,010 B/op 3 allocs/op ⭐ FASTEST, LEAST ALLOCS +NBD: 1,819 ns/op 1,500 B/op 6 allocs/op +Fiat: 1,971 ns/op 2,254 B/op 13 allocs/op +``` +**Analysis**: NWIO is fastest with minimal allocations. Significant memory advantage over competitors. + +### Event Serialization (for ID computation) +``` +NWIO: 391 ns/op 360 B/op 7 allocs/op +NBD: 129 ns/op 208 B/op 2 allocs/op ⭐ FASTEST, 3x faster +Fiat: 161 ns/op 400 B/op 3 allocs/op +``` +**Analysis**: NBD dominates here with optimized serialization. NWIO has room for improvement. + +### Event ID Computation +``` +NWIO: 608 ns/op 488 B/op 9 allocs/op +NBD: 302 ns/op 336 B/op 4 allocs/op +Fiat: 276 ns/op 400 B/op 3 allocs/op ⭐ FASTEST +``` +**Analysis**: NBD and Fiat are 2x faster. NWIO should optimize ID computation path. + +### Key Generation +``` +NWIO: 37,689 ns/op 208 B/op 4 allocs/op +NBD: 470 ns/op 369 B/op 8 allocs/op ⭐ FASTEST, 80x faster! +Fiat: 25,375 ns/op 272 B/op 5 allocs/op +``` +**Analysis**: ⚠️ NWIO is significantly slower. NBD appears to use a different key generation strategy. This is the biggest performance gap. + +### Event Signing +``` +NWIO: 129,854 ns/op 2,363 B/op 42 allocs/op +NBD: 59,069 ns/op 2,112 B/op 35 allocs/op ⭐ TIED FASTEST +Fiat: 58,572 ns/op 1,760 B/op 29 allocs/op ⭐ LEAST MEMORY +``` +**Analysis**: NBD and Fiat are 2x faster. NWIO has more allocations in signing path. + +### Event Verification +``` +NWIO: 99,744 ns/op 953 B/op 19 allocs/op ⭐ FASTEST +NBD: 105,995 ns/op 624 B/op 11 allocs/op ⭐ LEAST MEMORY +Fiat: 103,744 ns/op 640 B/op 9 allocs/op +``` +**Analysis**: NWIO is slightly faster (6% faster than others). Very competitive across all three. + +### Filter Matching (Simple) +``` +NWIO: 7.1 ns/op 0 B/op 0 allocs/op ⭐ FASTEST, 2x faster +NBD: 10.8 ns/op 0 B/op 0 allocs/op +Fiat: 16.4 ns/op 0 B/op 0 allocs/op +``` +**Analysis**: NWIO excels at filter matching! Zero allocations across all libraries. + +### Filter Matching (Complex with Tags) +``` +NWIO: 30.9 ns/op 0 B/op 0 allocs/op ⭐ FASTEST +NBD: 33.4 ns/op 0 B/op 0 allocs/op +Fiat: 42.6 ns/op 0 B/op 0 allocs/op +``` +**Analysis**: NWIO maintains lead in complex filters. Important for relay implementations. + +## Optimization Opportunities for NWIO + +### High Priority 🔴 +1. **Key Generation** - 80x slower than NBD + - Current: 37.7 µs + - Target: ~500 ns (similar to NBD) + - Impact: Critical for client applications + +2. **Event Signing** - 2x slower than competitors + - Current: 130 µs + - Target: ~60 µs (match NBD/Fiat) + - Impact: High for client applications + +### Medium Priority 🟡 +3. **Event Serialization** - 3x slower than NBD + - Current: 391 ns + - Target: ~130 ns (match NBD) + - Impact: Used in ID computation + +4. **ID Computation** - 2x slower than competitors + - Current: 608 ns + - Target: ~280 ns (match Fiat) + - Impact: Affects every event processing + +## Current Strengths of NWIO ✅ + +1. **Filter Matching** - 2x faster than Fiat, fastest overall +2. **Event Marshaling** - Fastest with minimal allocations +3. **Event Verification** - Slightly faster than competitors +4. **Memory Efficiency** - Competitive or better in most operations + +## Recommendations + +### For Relay Implementations +- **NWIO excels**: Best filter matching performance +- All three are competitive for event parsing/verification + +### For Client Implementations +- **NBD/Fiat preferred**: Much faster key generation and signing +- NWIO needs optimization in crypto operations + +### Overall Assessment +- **NWIO**: Best for relay/filter-heavy workloads +- **NBD**: Most balanced, excellent crypto performance +- **Fiat**: Good all-around, lowest memory in some operations + +## Running Your Own Benchmarks + +```bash +# Run all benchmarks +./run_benchmarks.sh + +# Compare specific operations +go test -bench=BenchmarkEventSign -benchmem comparison_bench_test.go +go test -bench=BenchmarkFilterMatch -benchmem comparison_bench_test.go + +# Statistical analysis with benchstat +go test -bench=. -count=10 comparison_bench_test.go > results.txt +benchstat results.txt +``` diff --git a/benchmark_results.txt b/benchmark_results.txt new file mode 100644 index 0000000..c3976e6 --- /dev/null +++ b/benchmark_results.txt @@ -0,0 +1,32 @@ +goos: linux +goarch: amd64 +cpu: AMD Ryzen AI 9 HX PRO 370 w/ Radeon 890M +BenchmarkEventUnmarshal_NWIO-24 498826 2541 ns/op 888 B/op 17 allocs/op +BenchmarkEventUnmarshal_NBD-24 423019 2832 ns/op 944 B/op 13 allocs/op +BenchmarkEventUnmarshal_Fiat-24 430042 2545 ns/op 752 B/op 10 allocs/op +BenchmarkEventMarshal_NWIO-24 613165 1790 ns/op 1010 B/op 3 allocs/op +BenchmarkEventMarshal_NBD-24 620986 1819 ns/op 1500 B/op 6 allocs/op +BenchmarkEventMarshal_Fiat-24 621964 1971 ns/op 2254 B/op 13 allocs/op +BenchmarkEventSerialize_NWIO-24 3059661 391.0 ns/op 360 B/op 7 allocs/op +BenchmarkEventSerialize_NBD-24 8824029 128.8 ns/op 208 B/op 2 allocs/op +BenchmarkEventSerialize_Fiat-24 6533536 160.9 ns/op 400 B/op 3 allocs/op +BenchmarkComputeID_NWIO-24 2108437 608.0 ns/op 488 B/op 9 allocs/op +BenchmarkComputeID_NBD-24 4072243 302.2 ns/op 336 B/op 4 allocs/op +BenchmarkComputeID_Fiat-24 4421660 275.9 ns/op 400 B/op 3 allocs/op +BenchmarkGenerateKey_NWIO-24 31942 37689 ns/op 208 B/op 4 allocs/op +BenchmarkGenerateKey_NBD-24 2489169 469.6 ns/op 369 B/op 8 allocs/op +BenchmarkGenerateKey_Fiat-24 45475 25375 ns/op 272 B/op 5 allocs/op +BenchmarkEventSign_NWIO-24 9072 129854 ns/op 2363 B/op 42 allocs/op +BenchmarkEventSign_NBD-24 20325 59069 ns/op 2112 B/op 35 allocs/op +BenchmarkEventSign_Fiat-24 20613 58572 ns/op 1760 B/op 29 allocs/op +BenchmarkEventVerify_NWIO-24 12009 99744 ns/op 953 B/op 19 allocs/op +BenchmarkEventVerify_NBD-24 10000 105995 ns/op 624 B/op 11 allocs/op +BenchmarkEventVerify_Fiat-24 10000 103744 ns/op 640 B/op 9 allocs/op +BenchmarkFilterMatch_NWIO-24 167376669 7.091 ns/op 0 B/op 0 allocs/op +BenchmarkFilterMatch_NBD-24 100000000 10.82 ns/op 0 B/op 0 allocs/op +BenchmarkFilterMatch_Fiat-24 71761591 16.40 ns/op 0 B/op 0 allocs/op +BenchmarkFilterMatchComplex_NWIO-24 39214178 30.88 ns/op 0 B/op 0 allocs/op +BenchmarkFilterMatchComplex_NBD-24 35580048 33.40 ns/op 0 B/op 0 allocs/op +BenchmarkFilterMatchComplex_Fiat-24 28026481 42.64 ns/op 0 B/op 0 allocs/op +PASS +ok command-line-arguments 40.651s diff --git a/comparison_bench_test.go b/comparison_bench_test.go new file mode 100644 index 0000000..ee3e936 --- /dev/null +++ b/comparison_bench_test.go @@ -0,0 +1,512 @@ +//go:build benchcmp + +package nostr_test + +import ( + "encoding/hex" + "encoding/json" + "testing" + + nwio "northwest.io/nostr" + nbd "github.com/nbd-wtf/go-nostr" + fiat "fiatjaf.com/nostr" +) + +// Sample event data for benchmarks +var ( + samplePubKey = "3bf0c63fcb93463407af97a5e5ee64fa883d107ef9e558472c4eb9aaaefa459d" + sampleSig = "230e9d8f0ddaf7eb70b5f7741ccfa37e87a455c9a469282e3464e2052d3192cd63a167e196e381ef9d7e69e9ea43af2b0d0e1b8f1e6e7c4e1c6e8e3e9e8e3e9e8" + sampleID = "d42c96ccac39e0113b2ef8df82e82e2e15e0e1e7e9e5e7e1e7e9e5e7e1e7e9e5" + + sampleEventJSON = `{ + "id": "d42c96ccac39e0113b2ef8df82e82e2e15e0e1e7e9e5e7e1e7e9e5e7e1e7e9e5", + "pubkey": "3bf0c63fcb93463407af97a5e5ee64fa883d107ef9e558472c4eb9aaaefa459d", + "created_at": 1672531200, + "kind": 1, + "tags": [["e", "5c83da77af1dec6d7289834998ad7aafbd9e2191396d75ec3cc27f5a77226f36", "wss://nostr.example.com"]], + "content": "Hello Nostr!", + "sig": "230e9d8f0ddaf7eb70b5f7741ccfa37e87a455c9a469282e3464e2052d3192cd63a167e196e381ef9d7e69e9ea43af2b0d0e1b8f1e6e7c4e1c6e8e3e9e8e3e9e8" + }` +) + +// ============================================================================ +// Event Unmarshaling Benchmarks +// ============================================================================ + +func BenchmarkEventUnmarshal_NWIO(b *testing.B) { + data := []byte(sampleEventJSON) + b.ResetTimer() + for i := 0; i < b.N; i++ { + var evt nwio.Event + if err := json.Unmarshal(data, &evt); err != nil { + b.Fatal(err) + } + } +} + +func BenchmarkEventUnmarshal_NBD(b *testing.B) { + data := []byte(sampleEventJSON) + b.ResetTimer() + for i := 0; i < b.N; i++ { + var evt nbd.Event + if err := json.Unmarshal(data, &evt); err != nil { + b.Fatal(err) + } + } +} + +func BenchmarkEventUnmarshal_Fiat(b *testing.B) { + data := []byte(sampleEventJSON) + b.ResetTimer() + for i := 0; i < b.N; i++ { + var evt fiat.Event + if err := json.Unmarshal(data, &evt); err != nil { + b.Fatal(err) + } + } +} + +// ============================================================================ +// Event Marshaling Benchmarks +// ============================================================================ + +func BenchmarkEventMarshal_NWIO(b *testing.B) { + evt := &nwio.Event{ + ID: sampleID, + PubKey: samplePubKey, + CreatedAt: 1672531200, + Kind: 1, + Tags: nwio.Tags{{"e", "5c83da77af1dec6d7289834998ad7aafbd9e2191396d75ec3cc27f5a77226f36"}}, + Content: "Hello Nostr!", + Sig: sampleSig, + } + b.ResetTimer() + for i := 0; i < b.N; i++ { + if _, err := json.Marshal(evt); err != nil { + b.Fatal(err) + } + } +} + +func BenchmarkEventMarshal_NBD(b *testing.B) { + evt := &nbd.Event{ + ID: sampleID, + PubKey: samplePubKey, + CreatedAt: nbd.Timestamp(1672531200), + Kind: 1, + Tags: nbd.Tags{{"e", "5c83da77af1dec6d7289834998ad7aafbd9e2191396d75ec3cc27f5a77226f36"}}, + Content: "Hello Nostr!", + Sig: sampleSig, + } + b.ResetTimer() + for i := 0; i < b.N; i++ { + if _, err := json.Marshal(evt); err != nil { + b.Fatal(err) + } + } +} + +func BenchmarkEventMarshal_Fiat(b *testing.B) { + sig, _ := hex.DecodeString(sampleSig) + var sigBytes [64]byte + copy(sigBytes[:], sig) + + pubkey, _ := fiat.PubKeyFromHex(samplePubKey) + id, _ := fiat.IDFromHex(sampleID) + + evt := &fiat.Event{ + ID: id, + PubKey: pubkey, + CreatedAt: fiat.Timestamp(1672531200), + Kind: 1, + Tags: fiat.Tags{{"e", "5c83da77af1dec6d7289834998ad7aafbd9e2191396d75ec3cc27f5a77226f36"}}, + Content: "Hello Nostr!", + Sig: sigBytes, + } + b.ResetTimer() + for i := 0; i < b.N; i++ { + if _, err := json.Marshal(evt); err != nil { + b.Fatal(err) + } + } +} + +// ============================================================================ +// Event Serialization (for ID computation) Benchmarks +// ============================================================================ + +func BenchmarkEventSerialize_NWIO(b *testing.B) { + evt := &nwio.Event{ + PubKey: samplePubKey, + CreatedAt: 1672531200, + Kind: 1, + Tags: nwio.Tags{{"e", "5c83da77af1dec6d7289834998ad7aafbd9e2191396d75ec3cc27f5a77226f36"}}, + Content: "Hello Nostr!", + } + b.ResetTimer() + for i := 0; i < b.N; i++ { + _ = evt.Serialize() + } +} + +func BenchmarkEventSerialize_NBD(b *testing.B) { + evt := &nbd.Event{ + PubKey: samplePubKey, + CreatedAt: nbd.Timestamp(1672531200), + Kind: 1, + Tags: nbd.Tags{{"e", "5c83da77af1dec6d7289834998ad7aafbd9e2191396d75ec3cc27f5a77226f36"}}, + Content: "Hello Nostr!", + } + b.ResetTimer() + for i := 0; i < b.N; i++ { + _ = evt.Serialize() + } +} + +func BenchmarkEventSerialize_Fiat(b *testing.B) { + pubkey, _ := fiat.PubKeyFromHex(samplePubKey) + evt := &fiat.Event{ + PubKey: pubkey, + CreatedAt: fiat.Timestamp(1672531200), + Kind: 1, + Tags: fiat.Tags{{"e", "5c83da77af1dec6d7289834998ad7aafbd9e2191396d75ec3cc27f5a77226f36"}}, + Content: "Hello Nostr!", + } + b.ResetTimer() + for i := 0; i < b.N; i++ { + _ = evt.Serialize() + } +} + +// ============================================================================ +// Event ID Computation Benchmarks +// ============================================================================ + +func BenchmarkComputeID_NWIO(b *testing.B) { + evt := &nwio.Event{ + PubKey: samplePubKey, + CreatedAt: 1672531200, + Kind: 1, + Tags: nwio.Tags{{"e", "5c83da77af1dec6d7289834998ad7aafbd9e2191396d75ec3cc27f5a77226f36"}}, + Content: "Hello Nostr!", + } + b.ResetTimer() + for i := 0; i < b.N; i++ { + _ = evt.ComputeID() + } +} + +func BenchmarkComputeID_NBD(b *testing.B) { + evt := &nbd.Event{ + PubKey: samplePubKey, + CreatedAt: nbd.Timestamp(1672531200), + Kind: 1, + Tags: nbd.Tags{{"e", "5c83da77af1dec6d7289834998ad7aafbd9e2191396d75ec3cc27f5a77226f36"}}, + Content: "Hello Nostr!", + } + b.ResetTimer() + for i := 0; i < b.N; i++ { + _ = evt.GetID() + } +} + +func BenchmarkComputeID_Fiat(b *testing.B) { + pubkey, _ := fiat.PubKeyFromHex(samplePubKey) + evt := &fiat.Event{ + PubKey: pubkey, + CreatedAt: fiat.Timestamp(1672531200), + Kind: 1, + Tags: fiat.Tags{{"e", "5c83da77af1dec6d7289834998ad7aafbd9e2191396d75ec3cc27f5a77226f36"}}, + Content: "Hello Nostr!", + } + b.ResetTimer() + for i := 0; i < b.N; i++ { + _ = evt.GetID() + } +} + +// ============================================================================ +// Key Generation Benchmarks +// ============================================================================ + +func BenchmarkGenerateKey_NWIO(b *testing.B) { + for i := 0; i < b.N; i++ { + if _, err := nwio.GenerateKey(); err != nil { + b.Fatal(err) + } + } +} + +func BenchmarkGenerateKey_NBD(b *testing.B) { + for i := 0; i < b.N; i++ { + _ = nbd.GeneratePrivateKey() + } +} + +func BenchmarkGenerateKey_Fiat(b *testing.B) { + for i := 0; i < b.N; i++ { + _ = fiat.Generate() + } +} + +// ============================================================================ +// Event Signing Benchmarks +// ============================================================================ + +func BenchmarkEventSign_NWIO(b *testing.B) { + key, err := nwio.GenerateKey() + if err != nil { + b.Fatal(err) + } + b.ResetTimer() + + for i := 0; i < b.N; i++ { + evt := &nwio.Event{ + CreatedAt: 1672531200, + Kind: 1, + Tags: nwio.Tags{}, + Content: "Hello Nostr!", + } + if err := key.Sign(evt); err != nil { + b.Fatal(err) + } + } +} + +func BenchmarkEventSign_NBD(b *testing.B) { + sk := nbd.GeneratePrivateKey() + b.ResetTimer() + + for i := 0; i < b.N; i++ { + evt := &nbd.Event{ + CreatedAt: nbd.Timestamp(1672531200), + Kind: 1, + Tags: nbd.Tags{}, + Content: "Hello Nostr!", + } + if err := evt.Sign(sk); err != nil { + b.Fatal(err) + } + } +} + +func BenchmarkEventSign_Fiat(b *testing.B) { + sk := fiat.Generate() + b.ResetTimer() + + for i := 0; i < b.N; i++ { + evt := &fiat.Event{ + CreatedAt: fiat.Timestamp(1672531200), + Kind: 1, + Tags: fiat.Tags{}, + Content: "Hello Nostr!", + } + if err := evt.Sign(sk); err != nil { + b.Fatal(err) + } + } +} + +// ============================================================================ +// Event Verification Benchmarks +// ============================================================================ + +func BenchmarkEventVerify_NWIO(b *testing.B) { + // Create and sign an event + key, _ := nwio.GenerateKey() + evt := &nwio.Event{ + CreatedAt: 1672531200, + Kind: 1, + Tags: nwio.Tags{}, + Content: "Hello Nostr!", + } + key.Sign(evt) + + b.ResetTimer() + for i := 0; i < b.N; i++ { + if !evt.Verify() { + b.Fatal("verification failed") + } + } +} + +func BenchmarkEventVerify_NBD(b *testing.B) { + // Create and sign an event + sk := nbd.GeneratePrivateKey() + evt := &nbd.Event{ + CreatedAt: nbd.Timestamp(1672531200), + Kind: 1, + Tags: nbd.Tags{}, + Content: "Hello Nostr!", + } + evt.Sign(sk) + + b.ResetTimer() + for i := 0; i < b.N; i++ { + if ok, _ := evt.CheckSignature(); !ok { + b.Fatal("verification failed") + } + } +} + +func BenchmarkEventVerify_Fiat(b *testing.B) { + // Create and sign an event + sk := fiat.Generate() + evt := &fiat.Event{ + CreatedAt: fiat.Timestamp(1672531200), + Kind: 1, + Tags: fiat.Tags{}, + Content: "Hello Nostr!", + } + evt.Sign(sk) + + b.ResetTimer() + for i := 0; i < b.N; i++ { + if !evt.VerifySignature() { + b.Fatal("verification failed") + } + } +} + +// ============================================================================ +// Filter Matching Benchmarks +// ============================================================================ + +func BenchmarkFilterMatch_NWIO(b *testing.B) { + filter := &nwio.Filter{ + Kinds: []int{1}, + Authors: []string{samplePubKey}, + } + evt := &nwio.Event{ + PubKey: samplePubKey, + CreatedAt: 1672531200, + Kind: 1, + Content: "Hello Nostr!", + } + + b.ResetTimer() + for i := 0; i < b.N; i++ { + if !filter.Matches(evt) { + b.Fatal("filter should match") + } + } +} + +func BenchmarkFilterMatch_NBD(b *testing.B) { + filter := nbd.Filter{ + Kinds: []int{1}, + Authors: []string{samplePubKey}, + } + evt := &nbd.Event{ + PubKey: samplePubKey, + CreatedAt: nbd.Timestamp(1672531200), + Kind: 1, + Content: "Hello Nostr!", + } + + b.ResetTimer() + for i := 0; i < b.N; i++ { + if !filter.Matches(evt) { + b.Fatal("filter should match") + } + } +} + +func BenchmarkFilterMatch_Fiat(b *testing.B) { + pubkey, _ := fiat.PubKeyFromHex(samplePubKey) + filter := fiat.Filter{ + Kinds: []fiat.Kind{1}, + Authors: []fiat.PubKey{pubkey}, + } + evt := &fiat.Event{ + PubKey: pubkey, + CreatedAt: fiat.Timestamp(1672531200), + Kind: 1, + Content: "Hello Nostr!", + } + + b.ResetTimer() + for i := 0; i < b.N; i++ { + if !filter.Matches(*evt) { + b.Fatal("filter should match") + } + } +} + +// ============================================================================ +// Complex Filter Matching Benchmarks (with tags) +// ============================================================================ + +func BenchmarkFilterMatchComplex_NWIO(b *testing.B) { + filter := &nwio.Filter{ + Kinds: []int{1, 6, 7}, + Authors: []string{samplePubKey[:8]}, // Prefix match + Tags: map[string][]string{ + "e": {"5c83da77af1dec6d7289834998ad7aafbd9e2191396d75ec3cc27f5a77226f36"}, + }, + } + evt := &nwio.Event{ + PubKey: samplePubKey, + CreatedAt: 1672531200, + Kind: 1, + Tags: nwio.Tags{{"e", "5c83da77af1dec6d7289834998ad7aafbd9e2191396d75ec3cc27f5a77226f36"}}, + Content: "Hello Nostr!", + } + + b.ResetTimer() + for i := 0; i < b.N; i++ { + if !filter.Matches(evt) { + b.Fatal("filter should match") + } + } +} + +func BenchmarkFilterMatchComplex_NBD(b *testing.B) { + filter := nbd.Filter{ + Kinds: []int{1, 6, 7}, + Authors: []string{samplePubKey}, // NBD also supports prefix, use full key for simplicity + Tags: nbd.TagMap{ + "e": []string{"5c83da77af1dec6d7289834998ad7aafbd9e2191396d75ec3cc27f5a77226f36"}, + }, + } + evt := &nbd.Event{ + PubKey: samplePubKey, + CreatedAt: nbd.Timestamp(1672531200), + Kind: 1, + Tags: nbd.Tags{{"e", "5c83da77af1dec6d7289834998ad7aafbd9e2191396d75ec3cc27f5a77226f36"}}, + Content: "Hello Nostr!", + } + + b.ResetTimer() + for i := 0; i < b.N; i++ { + if !filter.Matches(evt) { + b.Fatal("filter should match") + } + } +} + +func BenchmarkFilterMatchComplex_Fiat(b *testing.B) { + pubkey, _ := fiat.PubKeyFromHex(samplePubKey) + + filter := fiat.Filter{ + Kinds: []fiat.Kind{1, 6, 7}, + Authors: []fiat.PubKey{pubkey}, // Use full pubkey for simplicity + Tags: fiat.TagMap{ + "e": []string{"5c83da77af1dec6d7289834998ad7aafbd9e2191396d75ec3cc27f5a77226f36"}, + }, + } + evt := &fiat.Event{ + PubKey: pubkey, + CreatedAt: fiat.Timestamp(1672531200), + Kind: 1, + Tags: fiat.Tags{{"e", "5c83da77af1dec6d7289834998ad7aafbd9e2191396d75ec3cc27f5a77226f36"}}, + Content: "Hello Nostr!", + } + + b.ResetTimer() + for i := 0; i < b.N; i++ { + if !filter.Matches(*evt) { + b.Fatal("filter should match") + } + } +} diff --git a/go.mod b/go.mod index 20a17ea..222b18d 100644 --- a/go.mod +++ b/go.mod @@ -1,11 +1,11 @@ module northwest.io/nostr -go 1.21 +go 1.25 -require github.com/btcsuite/btcd/btcec/v2 v2.3.2 +require github.com/btcsuite/btcd/btcec/v2 v2.3.4 require ( - github.com/btcsuite/btcd/chaincfg/chainhash v1.0.1 // indirect - github.com/decred/dcrd/crypto/blake256 v1.0.0 // indirect - github.com/decred/dcrd/dcrec/secp256k1/v4 v4.0.1 // indirect + github.com/btcsuite/btcd/chaincfg/chainhash v1.1.0 // indirect + github.com/decred/dcrd/crypto/blake256 v1.1.0 // indirect + github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0 // indirect ) diff --git a/go.sum b/go.sum index 74ffce6..f86610d 100644 --- a/go.sum +++ b/go.sum @@ -1,10 +1,10 @@ -github.com/btcsuite/btcd/btcec/v2 v2.3.2 h1:5n0X6hX0Zk+6omWcihdYvdAlGf2DfasC0GMf7DClJ3U= -github.com/btcsuite/btcd/btcec/v2 v2.3.2/go.mod h1:zYzJ8etWJQIv1Ogk7OzpWjowwOdXY1W/17j2MW85J04= -github.com/btcsuite/btcd/chaincfg/chainhash v1.0.1 h1:q0rUy8C/TYNBQS1+CGKw68tLOFYSNEs0TFnxxnS9+4U= -github.com/btcsuite/btcd/chaincfg/chainhash v1.0.1/go.mod h1:7SFka0XMvUgj3hfZtydOrQY2mwhPclbT2snogU7SQQc= +github.com/btcsuite/btcd/btcec/v2 v2.3.4 h1:3EJjcN70HCu/mwqlUsGK8GcNVyLVxFDlWurTXGPFfiQ= +github.com/btcsuite/btcd/btcec/v2 v2.3.4/go.mod h1:zYzJ8etWJQIv1Ogk7OzpWjowwOdXY1W/17j2MW85J04= +github.com/btcsuite/btcd/chaincfg/chainhash v1.1.0 h1:59Kx4K6lzOW5w6nFlA0v5+lk/6sjybR934QNHSJZPTQ= +github.com/btcsuite/btcd/chaincfg/chainhash v1.1.0/go.mod h1:7SFka0XMvUgj3hfZtydOrQY2mwhPclbT2snogU7SQQc= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= -github.com/decred/dcrd/crypto/blake256 v1.0.0 h1:/8DMNYp9SGi5f0w7uCm6d6M4OU2rGFK09Y2A4Xv7EE0= -github.com/decred/dcrd/crypto/blake256 v1.0.0/go.mod h1:sQl2p6Y26YV+ZOcSTP6thNdn47hh8kt6rqSlvmrXFAc= -github.com/decred/dcrd/dcrec/secp256k1/v4 v4.0.1 h1:YLtO71vCjJRCBcrPMtQ9nqBsqpA1m5sE92cU+pd5Mcc= -github.com/decred/dcrd/dcrec/secp256k1/v4 v4.0.1/go.mod h1:hyedUtir6IdtD/7lIxGeCxkaw7y45JueMRL4DIyJDKs= +github.com/decred/dcrd/crypto/blake256 v1.1.0 h1:zPMNGQCm0g4QTY27fOCorQW7EryeQ/U0x++OzVrdms8= +github.com/decred/dcrd/crypto/blake256 v1.1.0/go.mod h1:2OfgNZ5wDpcsFmHmCK5gZTPcCXqlm2ArzUIkw9czNJo= +github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0 h1:NMZiJj8QnKe1LgsbDayM4UoHwbvwDRwnI3hwNaAHRnc= +github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0/go.mod h1:ZXNYxsqcloTdSy/rNShjYzMhyjf0LaoftYK0p+A3h40= diff --git a/run_benchmarks.sh b/run_benchmarks.sh new file mode 100755 index 0000000..87dfd17 --- /dev/null +++ b/run_benchmarks.sh @@ -0,0 +1,35 @@ +#!/bin/bash +set -e + +# Colors for output +GREEN='\033[0;32m' +BLUE='\033[0;34m' +YELLOW='\033[1;33m' +NC='\033[0m' # No Color + +echo -e "${BLUE}Running Nostr Library Benchmarks${NC}" +echo -e "${BLUE}Comparing: NWIO vs NBD-WTF vs Fiatjaf${NC}" +echo "" + +# First, install comparison dependencies if needed +echo -e "${YELLOW}Ensuring comparison dependencies are available...${NC}" +go get -tags=benchcmp -t ./... +echo "" + +# Run all benchmarks with the benchcmp build tag +echo -e "${GREEN}Running all benchmarks...${NC}" +go test -tags=benchcmp -bench=. -benchmem -benchtime=1s -run=^$ | tee benchmark_results.txt + +echo "" +echo -e "${GREEN}Results saved to benchmark_results.txt${NC}" +echo "" +echo -e "${BLUE}To run specific benchmark groups:${NC}" +echo " go test -tags=benchcmp -bench=BenchmarkEventUnmarshal -benchmem" +echo " go test -tags=benchcmp -bench=BenchmarkEventSign -benchmem" +echo " go test -tags=benchcmp -bench=BenchmarkEventVerify -benchmem" +echo " go test -tags=benchcmp -bench=BenchmarkFilterMatch -benchmem" +echo "" +echo -e "${BLUE}To compare specific libraries:${NC}" +echo " go test -tags=benchcmp -bench='.*_NWIO' -benchmem" +echo " go test -tags=benchcmp -bench='.*_NBD' -benchmem" +echo " go test -tags=benchcmp -bench='.*_Fiat' -benchmem" -- cgit v1.2.3