NAME

Time::Progress::Stored - Report progress + store and retrieve the current status

DESCRIPTION

This module helps if you have a long running process which reports progress, but you need to actually display the progress to the user in a different process.

Typically this is a long running web request in the web server or in a job queue qorker, while the web browser periodically sends ajax requests to get updated on the current progress status to show in a progress bar.

Time::Progress::Stored stores the progress report as the worker performs its job, and retrieves it from elsewhere where the report is needed.

The report is a hashref with the following details:

    | id                | "progress-1454518121172"   |
    | max               | 262                        |
    | current           | 2                          |
    | activity          | "Importing Business Units" |
    | elapsed_seconds   | "1"                        |
    | elapsed_time      | " 0:01"                    |
    | finish_time       | "Wed Feb 3 16:50:54 2016"  |
    | percent           | 0.8                        |
    | percent_string    | " 0.8%"                    |
    | remaining_seconds | "130"                      |
    | remaining_time    | " 2:10"                    |
    | is_done           | 0                          |

SYNOPSIS

    ### In the worker process

    my $progess_id = ...; # Probably comes from the client
    my $max = @items * 2; # Let's say two passes over @items

    my $redis = Redis->new( ... );
    my $progress = Time::Progress::Stored->new({
        max         => $max,
        progress_id => $progress_id,
        storage     => Progress::Stored::Storage::Redis->new({ redis => $redis }),
    });

    # Do the work
    for my $item (@items) {
        # Do stuff
        $progress->advance("Doing the first thing");
    }
    for my $item (@items) {
        # Do other stuff
        $progress->advance("Doing other stuff with $item->{name}");
    }

    $progress->done();


    ### HTTP endpoint, e.g. a Mojo controller for /progress/:progress_id
    sub get {
        my $self = shift;
        my $progress_id = $self->param("progress_id");

        my $storage = Time::Progress::Stored::Storage::Redis->new({
            redis => $self->redis,
        });
        my $response_json = $storage->retrieve($progress_id)
            or return $self->render(status => 404, json => {});

        return $self->render(json => { progress => $response_json });
    }

Synopsis breakdown

Worker

As the long running piece of work comes in, start the progress reporting.

Create the $progress object with a storage to store the progress reports. Initialize it with the number of actions you want to perform.

For each time something is done, call $progress->advance(). This might update the actual report.

By default only about 100 steps will be reported (each percent progressed). You can increase this by specifying the attribute "report_every", e.g. to report every single time something happens, set it to 1 (this might slow things down though).

"advance" can optionally be called with an activity description string, indicating what's happening right now. This can be large scale steps, or include details about this specific action being performed.

At the end, call $progress->done, just to make sure the reporting knows it has reached the end. This will set is_done: 1 in the report.

HTTP endpoint

The client should send requests to the endpoint periodically to get the current progress report. The request contains the progress id, and it's simply fetched and returned as a JSON payload.

See above for the contents of the payload.

Client with long running web request

This is for when the long running process is running as one web request.

A web page would typically provide a progress id while kickking off the long running process.

The client then sends ajax requests to the HTTP endpoint (using the id) every second to get updates on the progress. It does this until the long running progress has responded.

If you look at the report payload you'll see there is a variety of information to provide user feedback, e.g. percent done, time left, what's going on etc.

Client with asynchronous worker process

This is for when the long running process is running as a background process, e.g. in a job queue.

A web page would typically kick off the long running process. Instead of the client passing in a progress id, the $progress object defaults to a unique id (or it can just make one up), and the HTTP endpoint can report $progress->progress_id back to the client.

The client then sends ajax requests to the HTTP endpoint (using the id) every second to get updates on the progress. It does this until the progress report "is_done" is set to 1.

Storage backends

Time::Progress::Stored can use various backends to store the status.

Current backends are Redis and Memory. They are tiny and it would be simple to write another backend for e.g. a file, or using one of the Cache or CHI modules. Patches more than welcome.

Pitfalls

If you're testing with a single threaded test server like Morbo, the web browser's Ajax requests won't be processed until the long running process has finished, so you'll never see the progress until it's all done.

ATTRIBUTES

max

The number of times you're planning to call ->advance().

storage

A Storage object to store reports in.

Default: Time::Progress::Stored::Storage::Memory, but that's probably only useful for testing.

progress_id

The id to identify this progress report with. Either set by the client, or defaulted to a unique URL-param safe value you can return to the client.

report_every

Number of calls to ->advance() for every time a new progress report is generated and stored.

Default: a value that results in a report for every percent of progress.

current

The current iteration. You could in theory set this yourself, but it's recommended to call ->advance().

current_activity

Same as ->current, but for the current activity.

METHODS

advance($activity?)

Advance the progress one iteration. Optionally set the current $activity.

done()

Mark the progress as being completed, and set the is_done key in the report.

DEVELOPMENT

Author

Johan Lindstrom, <johanl [AT] cpan.org>

Contributors

Source code

https://github.com/jplindstrom/p5-Time-Progress-Stored

Bug reports

Please report any bugs or feature requests on GitHub:

https://github.com/jplindstrom/p5-Time-Progress-Stored/issues.

Caveats

COPYRIGHT & LICENSE

Copyright 2016- Broadbean Technologies, All Rights Reserved.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

ACKNOWLEDGEMENTS

Thanks to Broadbean for providing time to open source this during one of the regular Hack-days.

This module uses the excellent Time::Progress module, which you should be using if you just need a progress bar in a command line script.