summaryrefslogtreecommitdiffstats
path: root/benchmarks/BENCHMARK_SUMMARY.md
blob: aa0d771f08bca5dd98e5577b797568db61b2baca (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
# Benchmark Results Summary

Comparison of three Go Nostr libraries: **NWIO** (code.northwest.io/nostr v0.3.0), **NBD** (github.com/nbd-wtf/go-nostr), and **Fiat** (fiatjaf.com/nostr)

## The Honest Truth

NWIO v0.3.0 uses a pure-Go secp256k1 implementation with zero external dependencies. This makes the crypto **dramatically slower** than libraries using btcec:

| Operation | NWIO | NBD | Fiat | NWIO Slowdown |
|-----------|------|-----|------|---------------|
| **Key Gen** | 22.6 ms | 2.6 µs | 49.7 µs | ~8500x slower |
| **Sign** | 49.4 ms | 122 µs | 121 µs | ~400x slower |
| **Verify** | 54.5 ms | 199 µs | 198 µs | ~274x slower |

That's not a typo. The pure `big.Int` implementation is brutal.

## Where NWIO Wins

Non-crypto operations are competitive or fastest:

| Operation | NWIO | NBD | Fiat | Winner |
|-----------|------|-----|------|--------|
| **Event Marshal** | 8.9 µs | 12.3 µs | 13.0 µs | NWIO ⭐ |
| **Event Unmarshal** | 10.4 µs | 11.0 µs | 8.5 µs | Fiat |
| **Filter Match** | 14.3 ns | 22.3 ns | 38.5 ns | NWIO ⭐ |
| **Filter Complex** | 66.4 ns | 74.2 ns | 92.8 ns | NWIO ⭐ |

## Detailed Results

```
goos: linux
goarch: amd64
cpu: Intel(R) Core(TM) i7-4770HQ CPU @ 2.20GHz

BenchmarkEventUnmarshal_NWIO-8        163561     10400 ns/op      888 B/op    17 allocs/op
BenchmarkEventUnmarshal_NBD-8          94100     10985 ns/op      944 B/op    13 allocs/op
BenchmarkEventUnmarshal_Fiat-8        128083      8498 ns/op      752 B/op    10 allocs/op

BenchmarkEventMarshal_NWIO-8          136460      8857 ns/op     1009 B/op     3 allocs/op
BenchmarkEventMarshal_NBD-8            87494     12326 ns/op     1497 B/op     6 allocs/op
BenchmarkEventMarshal_Fiat-8          203302     13049 ns/op     2250 B/op    13 allocs/op

BenchmarkEventSerialize_NWIO-8        451017      2967 ns/op      360 B/op     7 allocs/op
BenchmarkEventSerialize_NBD-8        1265192      1066 ns/op      208 B/op     2 allocs/op
BenchmarkEventSerialize_Fiat-8        907284      1378 ns/op      400 B/op     3 allocs/op

BenchmarkComputeID_NWIO-8             250470      4425 ns/op      488 B/op     9 allocs/op
BenchmarkComputeID_NBD-8              391370      3062 ns/op      336 B/op     4 allocs/op
BenchmarkComputeID_Fiat-8             339658      3677 ns/op      400 B/op     3 allocs/op

BenchmarkGenerateKey_NWIO-8               46  22596880 ns/op  1682613 B/op 27259 allocs/op
BenchmarkGenerateKey_NBD-8            857683      2643 ns/op      368 B/op     8 allocs/op
BenchmarkGenerateKey_Fiat-8            23874     49726 ns/op      272 B/op     5 allocs/op

BenchmarkEventSign_NWIO-8                 38  49364702 ns/op  3403147 B/op 55099 allocs/op
BenchmarkEventSign_NBD-8                9332    122518 ns/op     2112 B/op    35 allocs/op
BenchmarkEventSign_Fiat-8               8274    121756 ns/op     1760 B/op    29 allocs/op

BenchmarkEventVerify_NWIO-8               26  54485034 ns/op  3310792 B/op 53635 allocs/op
BenchmarkEventVerify_NBD-8              5815    199061 ns/op      624 B/op    11 allocs/op
BenchmarkEventVerify_Fiat-8             5856    198714 ns/op      640 B/op     9 allocs/op

BenchmarkFilterMatch_NWIO-8         81765290     14.34 ns/op        0 B/op     0 allocs/op
BenchmarkFilterMatch_NBD-8          53242167     22.26 ns/op        0 B/op     0 allocs/op
BenchmarkFilterMatch_Fiat-8         30670489     38.53 ns/op        0 B/op     0 allocs/op

BenchmarkFilterMatchComplex_NWIO-8  17972340     66.38 ns/op        0 B/op     0 allocs/op
BenchmarkFilterMatchComplex_NBD-8   14769445     74.21 ns/op        0 B/op     0 allocs/op
BenchmarkFilterMatchComplex_Fiat-8  12921300     92.83 ns/op        0 B/op     0 allocs/op
```

## Should You Use NWIO?

**For learning/reading code:** Yes. Zero dependencies, everything is auditable.

**For a side project:** Maybe. 50ms to sign an event is noticeable but tolerable if you're not signing constantly.

**For anything serious:** No. Use NBD or Fiat. The crypto performance gap is too large.

## Why So Slow?

The internal secp256k1 implementation uses Go's `math/big` for arbitrary-precision arithmetic. Every operation allocates, nothing is constant-time, and there's no assembly optimization. Production libraries like btcec use fixed-width limbs, stack allocation, and hand-tuned assembly.

This is the price of zero dependencies and readable code.