353 lines
11 KiB
Text
353 lines
11 KiB
Text
=head1 NAME
|
|
|
|
FU::Benchmarks - A bunch of automated benchmark results.
|
|
|
|
=head1 DESCRIPTION
|
|
|
|
This file is automatically generated from 'bench.PL' in the L<FU> distribution.
|
|
These benchmarks compare performance of some FU functionality against similar
|
|
modules found on CPAN.
|
|
|
|
B<DISCLAIMER#1:> Obtaining accurate measurements is notoriously hard. Take the
|
|
numbers below with a few buckets of salt, any difference below 10% is most
|
|
likely noise.
|
|
|
|
B<DISCLAIMER#2:> Goodhart's law: "When a measure becomes a target, it ceases to
|
|
be a good measure". I've used these benchmarks to find and optimize hotspots in
|
|
FU, which in turn means these numbers may look better than they are in
|
|
real-world use.
|
|
|
|
B<DISCLAIMER#3:> Many of these benchmarks exists solely to test edge case
|
|
performance, these numbers are not representative for real-world use.
|
|
|
|
=head1 MODULE VERSIONS
|
|
|
|
The following module versions were used:
|
|
|
|
=over
|
|
|
|
=item L<Cpanel::JSON::XS> 4.39
|
|
|
|
=item L<DBD::Pg> 3.18.0
|
|
|
|
=item L<FU> 0.5
|
|
|
|
=item L<HTML::Tiny> 1.08
|
|
|
|
=item L<JSON::PP> 4.16
|
|
|
|
=item L<JSON::SIMD> 1.06
|
|
|
|
=item L<JSON::Tiny> 0.58
|
|
|
|
=item L<JSON::XS> 4.03
|
|
|
|
=item L<Pg::PQ> 0.15
|
|
|
|
=item L<TUWF::XML> 1.5
|
|
|
|
=item L<XML::Writer> 0.900
|
|
|
|
|
|
|
|
=back
|
|
|
|
=head1 BENCHMARKS
|
|
|
|
=head2 JSON Parsing & Formatting
|
|
|
|
These benchmarks run on large-ish arrays with repeated values. JSON encoding is
|
|
sufficiently fast that Perl function calling overhead tends to dominate for
|
|
smaller inputs, but I don't find that overhead very interesting.
|
|
|
|
Also worth noting that L<JSON::SIMD> formatting code is forked from
|
|
L<JSON::XS>, the SIMD parts are only used for parsing.
|
|
|
|
API object from L<JSON::XS> documentation.
|
|
|
|
Encode Canonical Decode
|
|
JSON::PP 5312/s 5119/s 1290/s
|
|
JSON::Tiny 7757/s - 3426/s
|
|
Cpanel::JSON::XS 114802/s 104141/s 107274/s
|
|
JSON::SIMD 130137/s 118948/s 115123/s
|
|
JSON::XS 128421/s 120243/s 117940/s
|
|
FU::Util 132890/s 111630/s 121124/s
|
|
|
|
Object (small)
|
|
|
|
Encode Canonical Decode
|
|
JSON::PP 907/s 829/s 202/s
|
|
JSON::Tiny 1224/s - 499/s
|
|
Cpanel::JSON::XS 45732/s 30862/s 20102/s
|
|
JSON::SIMD 49019/s 30699/s 23267/s
|
|
JSON::XS 49814/s 31326/s 25336/s
|
|
FU::Util 43853/s 26568/s 20426/s
|
|
|
|
Object (large)
|
|
|
|
Encode Canonical Decode
|
|
JSON::PP 927/s 747/s 104/s
|
|
JSON::Tiny 1108/s - 392/s
|
|
Cpanel::JSON::XS 30587/s 11875/s 15515/s
|
|
JSON::SIMD 24418/s 12388/s 22895/s
|
|
JSON::XS 23192/s 13174/s 23553/s
|
|
FU::Util 36455/s 11920/s 17370/s
|
|
|
|
Object (large, mixed unicode)
|
|
|
|
Encode Canonical Decode
|
|
JSON::PP 817/s 679/s 86/s
|
|
JSON::Tiny 1036/s - 402/s
|
|
Cpanel::JSON::XS 25333/s 1459/s 7480/s
|
|
JSON::SIMD 25031/s 1331/s 15997/s
|
|
JSON::XS 23580/s 1375/s 8526/s
|
|
FU::Util 33085/s 12639/s 9375/s
|
|
|
|
Small integers
|
|
|
|
Encode Decode
|
|
JSON::PP 113/s 29/s
|
|
JSON::Tiny 160/s 86/s
|
|
Cpanel::JSON::XS 7345/s 6151/s
|
|
JSON::SIMD 7963/s 4361/s
|
|
JSON::XS 7915/s 6058/s
|
|
FU::Util 7883/s 5671/s
|
|
|
|
Large integers
|
|
|
|
Encode Decode
|
|
JSON::PP 2176/s 329/s
|
|
JSON::Tiny 2999/s 1638/s
|
|
Cpanel::JSON::XS 32545/s 50162/s
|
|
JSON::SIMD 37201/s 51719/s
|
|
JSON::XS 36722/s 50110/s
|
|
FU::Util 110210/s 61006/s
|
|
|
|
ASCII strings
|
|
|
|
Encode Decode
|
|
JSON::PP 2934/s 336/s
|
|
JSON::Tiny 4126/s 1439/s
|
|
Cpanel::JSON::XS 116721/s 44560/s
|
|
JSON::SIMD 134711/s 50429/s
|
|
JSON::XS 135419/s 43976/s
|
|
FU::Util 164804/s 48163/s
|
|
|
|
Unicode strings
|
|
|
|
Encode Decode
|
|
JSON::PP 5113/s 253/s
|
|
JSON::Tiny 6603/s 2585/s
|
|
Cpanel::JSON::XS 97039/s 67669/s
|
|
JSON::SIMD 106928/s 102440/s
|
|
JSON::XS 105473/s 60558/s
|
|
FU::Util 187489/s 61121/s
|
|
|
|
String escaping (few)
|
|
|
|
Encode Decode
|
|
JSON::PP 4251/s 352/s
|
|
JSON::Tiny 4704/s 1869/s
|
|
Cpanel::JSON::XS 136755/s 118059/s
|
|
JSON::SIMD 158171/s 153692/s
|
|
JSON::XS 157261/s 97676/s
|
|
FU::Util 216443/s 96354/s
|
|
|
|
String escaping (many)
|
|
|
|
Encode Decode
|
|
JSON::PP 2224/s 366/s
|
|
JSON::Tiny 2884/s 984/s
|
|
Cpanel::JSON::XS 140220/s 107040/s
|
|
JSON::SIMD 152951/s 113242/s
|
|
JSON::XS 153471/s 106269/s
|
|
FU::Util 153081/s 100279/s
|
|
|
|
|
|
|
|
=head2 XML Writing
|
|
|
|
L<FU::XMLWriter> is the only XS-based XML DSL that I'm aware of, so all direct
|
|
competition is inherently slower by virtue of being pure perl. I'm sure some
|
|
templating modules will perform better, though.
|
|
|
|
HTML fragment
|
|
|
|
TUWF::XML 795/s
|
|
XML::Writer 833/s
|
|
HTML::Tiny 423/s
|
|
FU::XMLWriter 5396/s
|
|
|
|
|
|
|
|
=head2 PostgreSQL client
|
|
|
|
Fetching query results is highly unlikely to be a bottleneck in your code, this
|
|
benchmark is mainly here to verify that L<FU::Pg> is not introducing a
|
|
bottleneck where there shouldn't be one.
|
|
|
|
Fetch and bitwise-or 20k integers
|
|
|
|
Smallint Bigint
|
|
DBD::Pg 194/s 22/s
|
|
Pg::PQ 226/s 19/s
|
|
FU::Pg (bin) 239/s 23/s
|
|
FU::Pg (text) 222/s 21/s
|
|
|
|
|
|
|
|
=cut
|
|
|
|
# Cached data used by bench.PL. Same as the formatted tables above but easier to parse.
|
|
json/api Canonical Cpanel::JSON::XS 104141
|
|
json/api Canonical FU::Util 111630
|
|
json/api Canonical JSON::PP 5119
|
|
json/api Canonical JSON::SIMD 118948
|
|
json/api Canonical JSON::XS 120243
|
|
json/api Decode Cpanel::JSON::XS 107274
|
|
json/api Decode FU::Util 121124
|
|
json/api Decode JSON::PP 1290
|
|
json/api Decode JSON::SIMD 115123
|
|
json/api Decode JSON::Tiny 3426
|
|
json/api Decode JSON::XS 117940
|
|
json/api Encode Cpanel::JSON::XS 114802
|
|
json/api Encode FU::Util 132890
|
|
json/api Encode JSON::PP 5312
|
|
json/api Encode JSON::SIMD 130137
|
|
json/api Encode JSON::Tiny 7757
|
|
json/api Encode JSON::XS 128421
|
|
json/intl Decode Cpanel::JSON::XS 50162
|
|
json/intl Decode FU::Util 61006
|
|
json/intl Decode JSON::PP 329
|
|
json/intl Decode JSON::SIMD 51719
|
|
json/intl Decode JSON::Tiny 1638
|
|
json/intl Decode JSON::XS 50110
|
|
json/intl Encode Cpanel::JSON::XS 32545
|
|
json/intl Encode FU::Util 110210
|
|
json/intl Encode JSON::PP 2176
|
|
json/intl Encode JSON::SIMD 37201
|
|
json/intl Encode JSON::Tiny 2999
|
|
json/intl Encode JSON::XS 36722
|
|
json/ints Decode Cpanel::JSON::XS 6151
|
|
json/ints Decode FU::Util 5671
|
|
json/ints Decode JSON::PP 29
|
|
json/ints Decode JSON::SIMD 4361
|
|
json/ints Decode JSON::Tiny 86
|
|
json/ints Decode JSON::XS 6058
|
|
json/ints Encode Cpanel::JSON::XS 7345
|
|
json/ints Encode FU::Util 7883
|
|
json/ints Encode JSON::PP 113
|
|
json/ints Encode JSON::SIMD 7963
|
|
json/ints Encode JSON::Tiny 160
|
|
json/ints Encode JSON::XS 7915
|
|
json/objl Canonical Cpanel::JSON::XS 11875
|
|
json/objl Canonical FU::Util 11920
|
|
json/objl Canonical JSON::PP 747
|
|
json/objl Canonical JSON::SIMD 12388
|
|
json/objl Canonical JSON::XS 13174
|
|
json/objl Decode Cpanel::JSON::XS 15515
|
|
json/objl Decode FU::Util 17370
|
|
json/objl Decode JSON::PP 104
|
|
json/objl Decode JSON::SIMD 22895
|
|
json/objl Decode JSON::Tiny 392
|
|
json/objl Decode JSON::XS 23553
|
|
json/objl Encode Cpanel::JSON::XS 30587
|
|
json/objl Encode FU::Util 36455
|
|
json/objl Encode JSON::PP 927
|
|
json/objl Encode JSON::SIMD 24418
|
|
json/objl Encode JSON::Tiny 1108
|
|
json/objl Encode JSON::XS 23192
|
|
json/objs Canonical Cpanel::JSON::XS 30862
|
|
json/objs Canonical FU::Util 26568
|
|
json/objs Canonical JSON::PP 829
|
|
json/objs Canonical JSON::SIMD 30699
|
|
json/objs Canonical JSON::XS 31326
|
|
json/objs Decode Cpanel::JSON::XS 20102
|
|
json/objs Decode FU::Util 20426
|
|
json/objs Decode JSON::PP 202
|
|
json/objs Decode JSON::SIMD 23267
|
|
json/objs Decode JSON::Tiny 499
|
|
json/objs Decode JSON::XS 25336
|
|
json/objs Encode Cpanel::JSON::XS 45732
|
|
json/objs Encode FU::Util 43853
|
|
json/objs Encode JSON::PP 907
|
|
json/objs Encode JSON::SIMD 49019
|
|
json/objs Encode JSON::Tiny 1224
|
|
json/objs Encode JSON::XS 49814
|
|
json/obju Canonical Cpanel::JSON::XS 1459
|
|
json/obju Canonical FU::Util 12639
|
|
json/obju Canonical JSON::PP 679
|
|
json/obju Canonical JSON::SIMD 1331
|
|
json/obju Canonical JSON::XS 1375
|
|
json/obju Decode Cpanel::JSON::XS 7480
|
|
json/obju Decode FU::Util 9375
|
|
json/obju Decode JSON::PP 86
|
|
json/obju Decode JSON::SIMD 15997
|
|
json/obju Decode JSON::Tiny 402
|
|
json/obju Decode JSON::XS 8526
|
|
json/obju Encode Cpanel::JSON::XS 25333
|
|
json/obju Encode FU::Util 33085
|
|
json/obju Encode JSON::PP 817
|
|
json/obju Encode JSON::SIMD 25031
|
|
json/obju Encode JSON::Tiny 1036
|
|
json/obju Encode JSON::XS 23580
|
|
json/strel Decode Cpanel::JSON::XS 107040
|
|
json/strel Decode FU::Util 100279
|
|
json/strel Decode JSON::PP 366
|
|
json/strel Decode JSON::SIMD 113242
|
|
json/strel Decode JSON::Tiny 984
|
|
json/strel Decode JSON::XS 106269
|
|
json/strel Encode Cpanel::JSON::XS 140220
|
|
json/strel Encode FU::Util 153081
|
|
json/strel Encode JSON::PP 2224
|
|
json/strel Encode JSON::SIMD 152951
|
|
json/strel Encode JSON::Tiny 2884
|
|
json/strel Encode JSON::XS 153471
|
|
json/stres Decode Cpanel::JSON::XS 118059
|
|
json/stres Decode FU::Util 96354
|
|
json/stres Decode JSON::PP 352
|
|
json/stres Decode JSON::SIMD 153692
|
|
json/stres Decode JSON::Tiny 1869
|
|
json/stres Decode JSON::XS 97676
|
|
json/stres Encode Cpanel::JSON::XS 136755
|
|
json/stres Encode FU::Util 216443
|
|
json/stres Encode JSON::PP 4251
|
|
json/stres Encode JSON::SIMD 158171
|
|
json/stres Encode JSON::Tiny 4704
|
|
json/stres Encode JSON::XS 157261
|
|
json/strs Decode Cpanel::JSON::XS 44560
|
|
json/strs Decode FU::Util 48163
|
|
json/strs Decode JSON::PP 336
|
|
json/strs Decode JSON::SIMD 50429
|
|
json/strs Decode JSON::Tiny 1439
|
|
json/strs Decode JSON::XS 43976
|
|
json/strs Encode Cpanel::JSON::XS 116721
|
|
json/strs Encode FU::Util 164804
|
|
json/strs Encode JSON::PP 2934
|
|
json/strs Encode JSON::SIMD 134711
|
|
json/strs Encode JSON::Tiny 4126
|
|
json/strs Encode JSON::XS 135419
|
|
json/stru Decode Cpanel::JSON::XS 67669
|
|
json/stru Decode FU::Util 61121
|
|
json/stru Decode JSON::PP 253
|
|
json/stru Decode JSON::SIMD 102440
|
|
json/stru Decode JSON::Tiny 2585
|
|
json/stru Decode JSON::XS 60558
|
|
json/stru Encode Cpanel::JSON::XS 97039
|
|
json/stru Encode FU::Util 187489
|
|
json/stru Encode JSON::PP 5113
|
|
json/stru Encode JSON::SIMD 106928
|
|
json/stru Encode JSON::Tiny 6603
|
|
json/stru Encode JSON::XS 105473
|
|
pg/ints Bigint DBD::Pg 22
|
|
pg/ints Bigint FU::Pg (bin) 23
|
|
pg/ints Bigint FU::Pg (text) 21
|
|
pg/ints Bigint Pg::PQ 19
|
|
pg/ints Smallint DBD::Pg 194
|
|
pg/ints Smallint FU::Pg (bin) 239
|
|
pg/ints Smallint FU::Pg (text) 222
|
|
pg/ints Smallint Pg::PQ 226
|
|
xml/a Rate FU::XMLWriter 5396
|
|
xml/a Rate HTML::Tiny 423
|
|
xml/a Rate TUWF::XML 795
|
|
xml/a Rate XML::Writer 833
|