diff options
Diffstat (limited to 'BENCHMARKS.md')
| -rw-r--r-- | BENCHMARKS.md | 47 |
1 files changed, 24 insertions, 23 deletions
diff --git a/BENCHMARKS.md b/BENCHMARKS.md index 14a861c..1a671c1 100644 --- a/BENCHMARKS.md +++ b/BENCHMARKS.md | |||
| @@ -2,7 +2,7 @@ | |||
| 2 | 2 | ||
| 3 | This directory contains comprehensive benchmarks comparing three popular Go Nostr libraries: | 3 | This directory contains comprehensive benchmarks comparing three popular Go Nostr libraries: |
| 4 | 4 | ||
| 5 | - **NWIO** (`northwest.io/nostr`) - This library | 5 | - **NWIO** (`code.northwest.io/nostr`) - This library |
| 6 | - **NBD** (`github.com/nbd-wtf/go-nostr`) - Popular community library | 6 | - **NBD** (`github.com/nbd-wtf/go-nostr`) - Popular community library |
| 7 | - **Fiat** (`fiatjaf.com/nostr`) - Original implementation by Fiatjaf | 7 | - **Fiat** (`fiatjaf.com/nostr`) - Original implementation by Fiatjaf |
| 8 | 8 | ||
| @@ -25,61 +25,60 @@ This directory contains comprehensive benchmarks comparing three popular Go Nost | |||
| 25 | 25 | ||
| 26 | ## Running Benchmarks | 26 | ## Running Benchmarks |
| 27 | 27 | ||
| 28 | **Important**: The comparison benchmarks require the `benchcmp` build tag to avoid polluting the module dependencies with competitor libraries. | 28 | **Important**: The comparison benchmarks are in a separate module (`benchmarks/comparison/`) to avoid polluting the main module dependencies with competitor libraries. |
| 29 | 29 | ||
| 30 | ### Quick Start | 30 | ### Quick Start |
| 31 | 31 | ||
| 32 | Run all benchmarks (automatically handles dependencies): | 32 | Run all comparison benchmarks: |
| 33 | ```bash | 33 | ```bash |
| 34 | ./run_benchmarks.sh | 34 | cd benchmarks/comparison |
| 35 | ``` | 35 | go test -bench=. -benchmem -benchtime=1s |
| 36 | |||
| 37 | Or manually: | ||
| 38 | ```bash | ||
| 39 | # First, get the comparison dependencies | ||
| 40 | go get -tags=benchcmp -t ./... | ||
| 41 | |||
| 42 | # Then run the benchmarks | ||
| 43 | go test -tags=benchcmp -bench=. -benchmem -benchtime=1s | ||
| 44 | ``` | 36 | ``` |
| 45 | 37 | ||
| 46 | ### Specific Benchmark Groups | 38 | ### Specific Benchmark Groups |
| 47 | 39 | ||
| 48 | Event unmarshaling: | 40 | Event unmarshaling: |
| 49 | ```bash | 41 | ```bash |
| 50 | go test -tags=benchcmp -bench=BenchmarkEventUnmarshal -benchmem | 42 | cd benchmarks/comparison |
| 43 | go test -bench=BenchmarkEventUnmarshal -benchmem | ||
| 51 | ``` | 44 | ``` |
| 52 | 45 | ||
| 53 | Event signing: | 46 | Event signing: |
| 54 | ```bash | 47 | ```bash |
| 55 | go test -tags=benchcmp -bench=BenchmarkEventSign -benchmem | 48 | cd benchmarks/comparison |
| 49 | go test -bench=BenchmarkEventSign -benchmem | ||
| 56 | ``` | 50 | ``` |
| 57 | 51 | ||
| 58 | Event verification: | 52 | Event verification: |
| 59 | ```bash | 53 | ```bash |
| 60 | go test -tags=benchcmp -bench=BenchmarkEventVerify -benchmem | 54 | cd benchmarks/comparison |
| 55 | go test -bench=BenchmarkEventVerify -benchmem | ||
| 61 | ``` | 56 | ``` |
| 62 | 57 | ||
| 63 | Filter matching: | 58 | Filter matching: |
| 64 | ```bash | 59 | ```bash |
| 65 | go test -tags=benchcmp -bench=BenchmarkFilterMatch -benchmem | 60 | cd benchmarks/comparison |
| 61 | go test -bench=BenchmarkFilterMatch -benchmem | ||
| 66 | ``` | 62 | ``` |
| 67 | 63 | ||
| 68 | ### Compare Single Library | 64 | ### Compare Single Library |
| 69 | 65 | ||
| 70 | NWIO only: | 66 | NWIO only: |
| 71 | ```bash | 67 | ```bash |
| 72 | go test -tags=benchcmp -bench='.*_NWIO' -benchmem | 68 | cd benchmarks/comparison |
| 69 | go test -bench='.*_NWIO' -benchmem | ||
| 73 | ``` | 70 | ``` |
| 74 | 71 | ||
| 75 | NBD only: | 72 | NBD only: |
| 76 | ```bash | 73 | ```bash |
| 77 | go test -tags=benchcmp -bench='.*_NBD' -benchmem | 74 | cd benchmarks/comparison |
| 75 | go test -bench='.*_NBD' -benchmem | ||
| 78 | ``` | 76 | ``` |
| 79 | 77 | ||
| 80 | Fiat only: | 78 | Fiat only: |
| 81 | ```bash | 79 | ```bash |
| 82 | go test -tags=benchcmp -bench='.*_Fiat' -benchmem | 80 | cd benchmarks/comparison |
| 81 | go test -bench='.*_Fiat' -benchmem | ||
| 83 | ``` | 82 | ``` |
| 84 | 83 | ||
| 85 | ## Analyzing Results | 84 | ## Analyzing Results |
| @@ -91,14 +90,16 @@ Use `benchstat` for statistical analysis: | |||
| 91 | go install golang.org/x/perf/cmd/benchstat@latest | 90 | go install golang.org/x/perf/cmd/benchstat@latest |
| 92 | 91 | ||
| 93 | # Run benchmarks multiple times and compare | 92 | # Run benchmarks multiple times and compare |
| 94 | go test -tags=benchcmp -bench=. -benchmem -count=10 > results.txt | 93 | cd benchmarks/comparison |
| 94 | go test -bench=. -benchmem -count=10 > results.txt | ||
| 95 | benchstat results.txt | 95 | benchstat results.txt |
| 96 | ``` | 96 | ``` |
| 97 | 97 | ||
| 98 | Compare two specific libraries: | 98 | Compare two specific libraries: |
| 99 | ```bash | 99 | ```bash |
| 100 | go test -tags=benchcmp -bench='.*_NWIO' -benchmem -count=10 > nwio.txt | 100 | cd benchmarks/comparison |
| 101 | go test -tags=benchcmp -bench='.*_NBD' -benchmem -count=10 > nbd.txt | 101 | go test -bench='.*_NWIO' -benchmem -count=10 > nwio.txt |
| 102 | go test -bench='.*_NBD' -benchmem -count=10 > nbd.txt | ||
| 102 | benchstat nwio.txt nbd.txt | 103 | benchstat nwio.txt nbd.txt |
| 103 | ``` | 104 | ``` |
| 104 | 105 | ||
