NAME

Bencher::Scenario::ListUtil - Benchmark List::Util (XS) vs PP implementation(s)

VERSION

This document describes version 0.05 of Bencher::Scenario::ListUtil (from Perl distribution Bencher-Scenario-ListUtil), released on 2017-01-25.

SYNOPSIS

To run benchmark with default option:

 % bencher -m ListUtil

To run module startup overhead benchmark:

 % bencher --module-startup -m ListUtil

For more options (dump scenario, list/include/exclude/add participants, list/include/exclude/add datasets, etc), see bencher or run bencher --help.

DESCRIPTION

EARLY VERSION, ONLY A FEW FUNCTIONS HAVE BEEN BENCHMARKED.

For max*/min*, in general the XS version are about 3x faster than PP.

Packaging a benchmark script as a Bencher scenario makes it convenient to include/exclude/add participants/datasets (either via CLI or Perl code), send the result to a central repository, among others . See Bencher and bencher (CLI) for more details.

BENCHMARKED MODULES

Version numbers shown below are the versions used when running the sample benchmark.

List::Util 1.45

PERLANCAR::List::Util::PP 0.04

BENCHMARK PARTICIPANTS

  • List::Util::max (perl_code) [numeric]

    Function call template:

     List::Util::max(@{<list>})
  • PERLANCAR::List::Util::PP::max (perl_code) [numeric]

    Function call template:

     PERLANCAR::List::Util::PP::max(@{<list>})
  • List::Util::maxstr (perl_code) [stringy]

    Function call template:

     List::Util::maxstr(@{<list>})
  • PERLANCAR::List::Util::PP::maxstr (perl_code) [stringy]

    Function call template:

     PERLANCAR::List::Util::PP::maxstr(@{<list>})
  • List::Util::min (perl_code) [numeric]

    Function call template:

     List::Util::min(@{<list>})
  • PERLANCAR::List::Util::PP::min (perl_code) [numeric]

    Function call template:

     PERLANCAR::List::Util::PP::min(@{<list>})
  • List::Util::minstr (perl_code) [stringy]

    Function call template:

     List::Util::minstr(@{<list>})
  • PERLANCAR::List::Util::PP::minstr (perl_code) [stringy]

    Function call template:

     PERLANCAR::List::Util::PP::minstr(@{<list>})

BENCHMARK DATASETS

  • num10

  • num100

  • num1000

  • str10

  • str100

  • str1000

SAMPLE BENCHMARK RESULTS

Run on: perl: v5.24.0, CPU: Intel(R) Core(TM) M-5Y71 CPU @ 1.20GHz (2 cores), OS: GNU/Linux LinuxMint version 17.3, OS kernel: Linux version 3.19.0-32-generic.

Benchmark with default options (bencher -m ListUtil):

 #table1#
 +-----------------------------------+---------+------------+------------+------------+---------+---------+
 | participant                       | dataset | rate (/s)  | time (μs)  | vs_slowest |  errors | samples |
 +-----------------------------------+---------+------------+------------+------------+---------+---------+
 | PERLANCAR::List::Util::PP::minstr | num1000 |    4838.76 | 206.664    |    1       |   0     |      28 |
 | PERLANCAR::List::Util::PP::maxstr | num1000 |    5370    | 186        |    1.11    | 5.3e-08 |      20 |
 | PERLANCAR::List::Util::PP::maxstr | str1000 |    7450    | 134        |    1.54    | 4.8e-08 |      25 |
 | PERLANCAR::List::Util::PP::minstr | str1000 |    8540    | 117        |    1.77    |   5e-08 |      23 |
 | List::Util::minstr                | num1000 |    9189.73 | 108.817    |    1.89919 | 3.1e-11 |      20 |
 | List::Util::maxstr                | num1000 |    9300    | 110        |    1.9     | 2.1e-07 |      20 |
 | PERLANCAR::List::Util::PP::max    | num1000 |   10000    |  98        |    2.1     | 1.1e-07 |      20 |
 | PERLANCAR::List::Util::PP::min    | num1000 |   10700    |  93.5      |    2.21    | 2.4e-08 |      24 |
 | List::Util::minstr                | str1000 |   17000    |  60        |    3.4     | 5.6e-07 |      20 |
 | List::Util::maxstr                | str1000 |   18000    |  55.6      |    3.72    | 2.2e-08 |      30 |
 | List::Util::min                   | num1000 |   42500    |  23.5      |    8.79    | 6.5e-09 |      21 |
 | List::Util::max                   | num1000 |   43000    |  23.3      |    8.89    | 6.7e-09 |      20 |
 | PERLANCAR::List::Util::PP::minstr | num100  |   52755.6  |  18.9553   |   10.9027  | 1.1e-11 |      20 |
 | PERLANCAR::List::Util::PP::maxstr | num100  |   57252.1  |  17.4666   |   11.832   | 1.2e-11 |      20 |
 | PERLANCAR::List::Util::PP::maxstr | str100  |   67000    |  15        |   14       | 7.2e-08 |      29 |
 | PERLANCAR::List::Util::PP::minstr | str100  |   74700    |  13.4      |   15.4     | 5.4e-09 |      31 |
 | PERLANCAR::List::Util::PP::max    | num100  |   99800    |  10        |   20.6     | 3.3e-09 |      20 |
 | PERLANCAR::List::Util::PP::min    | num100  |  110000    |   9.2      |   23       | 1.5e-08 |      26 |
 | List::Util::minstr                | num100  |  110990    |   9.0095   |   22.938   | 1.2e-11 |      20 |
 | List::Util::maxstr                | num100  |  111398    |   8.9768   |   23.0221  |   0     |      29 |
 | List::Util::maxstr                | str100  |  170000    |   5.9      |   35       | 3.9e-08 |      31 |
 | List::Util::minstr                | str100  |  181070    |   5.5226   |   37.422   | 1.1e-11 |      25 |
 | List::Util::max                   | num100  |  370300    |   2.7      |   76.53    | 1.2e-10 |      20 |
 | List::Util::min                   | num100  |  418000    |   2.39     |   86.4     | 8.3e-10 |      20 |
 | PERLANCAR::List::Util::PP::maxstr | num10   |  446000    |   2.24     |   92.2     | 8.2e-10 |      22 |
 | PERLANCAR::List::Util::PP::minstr | num10   |  453900    |   2.203    |   93.8     | 3.4e-11 |      20 |
 | PERLANCAR::List::Util::PP::minstr | str10   |  586600    |   1.705    |  121.2     | 3.5e-11 |      32 |
 | PERLANCAR::List::Util::PP::maxstr | str10   |  603000    |   1.66     |  125       | 7.8e-10 |      23 |
 | PERLANCAR::List::Util::PP::max    | num10   |  688500    |   1.452    |  142.3     | 3.4e-11 |      22 |
 | List::Util::minstr                | num10   |  798000    |   1.25     |  165       | 3.7e-10 |      25 |
 | PERLANCAR::List::Util::PP::min    | num10   |  890000    |   1.12     |  184       |   1e-09 |      29 |
 | List::Util::maxstr                | num10   |  897000    |   1.12     |  185       |   4e-10 |      22 |
 | List::Util::maxstr                | str10   | 1403770    |   0.712368 |  290.109   |   0     |      20 |
 | List::Util::minstr                | str10   | 1420000    |   0.704    |  294       | 9.1e-11 |      21 |
 | List::Util::max                   | num10   | 2300000    |   0.43     |  480       | 3.3e-09 |      20 |
 | List::Util::min                   | num10   | 2700000    |   0.38     |  550       | 8.3e-10 |      20 |
 +-----------------------------------+---------+------------+------------+------------+---------+---------+

Benchmark module startup overhead (bencher -m ListUtil --module-startup):

 #table2#
 +---------------------------+------------------------------+--------------------+----------------+-----------+------------------------+------------+---------+---------+
 | participant               | proc_private_dirty_size (kB) | proc_rss_size (MB) | proc_size (MB) | time (ms) | mod_overhead_time (ms) | vs_slowest |  errors | samples |
 +---------------------------+------------------------------+--------------------+----------------+-----------+------------------------+------------+---------+---------+
 | List::Util                | 1108                         | 4.4                | 16             |       9.6 |                    3.7 |        1   | 2.5e-05 |      21 |
 | PERLANCAR::List::Util::PP | 1032                         | 4.3                | 18             |       9.6 |                    3.7 |        1   |   2e-05 |      20 |
 | perl -e1 (baseline)       | 1032                         | 4.4                | 18             |       5.9 |                    0   |        1.6 | 2.8e-05 |      20 |
 +---------------------------+------------------------------+--------------------+----------------+-----------+------------------------+------------+---------+---------+

To display as an interactive HTML table on a browser, you can add option --format html+datatables.

HOMEPAGE

Please visit the project's homepage at https://metacpan.org/release/Bencher-Scenario-ListUtil.

SOURCE

Source repository is at https://github.com/perlancar/perl-Bencher-Scenario-ListUtil.

BUGS

Please report any bugs or feature requests on the bugtracker website https://rt.cpan.org/Public/Dist/Display.html?Name=Bencher-Scenario-ListUtil

When submitting a bug or request, please include a test-file or a patch to an existing test-file that illustrates the bug or desired feature.

AUTHOR

perlancar <perlancar@cpan.org>

COPYRIGHT AND LICENSE

This software is copyright (c) 2017 by perlancar@cpan.org.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.