-
-
20 Oct 2021 09:35:18 UTC
- Distribution: Search-Elasticsearch-Client-6_0
- Module version: 7.715
- Source (raw)
- Browse (raw)
- Changes
- Homepage
- How to Contribute
- Repository
- Issues
- Testers (111 / 0 / 0)
- Kwalitee
Bus factor: 1- % Coverage
- License: apache_2_0
- Activity
24 month- Tools
- Download (66.43KB)
- MetaCPAN Explorer
- Permissions
- Subscribe to distribution
- Permalinks
- This version
- Latest version
and 1 contributors-
Enrico Zimuel
- Dependencies
- Devel::GlobalDestruction
- Moo
- Moo::Role
- Search::Elasticsearch
- Search::Elasticsearch::Role::API
- Search::Elasticsearch::Role::Client::Direct
- Search::Elasticsearch::Role::Is_Sync
- Search::Elasticsearch::Util
- Try::Tiny
- namespace::clean
- strict
- warnings
- Reverse dependencies
- CPAN Testers List
- Dependency graph
- NAME
- VERSION
- SYNOPSIS
- DESCRIPTION
- CREATING A NEW INSTANCE
- FLUSHING THE BUFFER
- CREATE, INDEX, UPDATE, DELETE
- AUTHOR
- COPYRIGHT AND LICENSE
NAME
Search::Elasticsearch::Client::6_0::Bulk - A helper module for the Bulk API
VERSION
version 7.715
SYNOPSIS
use Search::Elasticsearch; my $es = Search::Elasticsearch->new; my $bulk = $es->bulk_helper( index => 'my_index', type => 'my_type' ); # Index docs: $bulk->index({ id => 1, source => { foo => 'bar' }}); $bulk->add_action( index => { id => 1, source => { foo=> 'bar' }}); # Create docs: $bulk->create({ id => 1, source => { foo => 'bar' }}); $bulk->add_action( create => { id => 1, source => { foo=> 'bar' }}); $bulk->create_docs({ foo => 'bar' }) # Delete docs: $bulk->delete({ id => 1}); $bulk->add_action( delete => { id => 1 }); $bulk->delete_ids(1,2,3) # Update docs: $bulk->update({ id => 1, script => '...' }); $bulk->add_action( update => { id => 1, script => '...' }); # Manual flush $bulk->flush;
DESCRIPTION
This module provides a wrapper for the "bulk()" in Search::Elasticsearch::Client::6_0::Direct method which makes it easier to run multiple create, index, update or delete actions in a single request.
The Search::Elasticsearch::Client::6_0::Bulk module acts as a queue, buffering up actions until it reaches a maximum count of actions, or a maximum size of JSON request body, at which point it issues a
bulk()
request.Once you have finished adding actions, call "flush()" to force the final
bulk()
request on the items left in the queue.This class does Search::Elasticsearch::Client::6_0::Role::Bulk and Search::Elasticsearch::Role::Is_Sync.
CREATING A NEW INSTANCE
new()
my $bulk = $es->bulk_helper( index => 'default_index', # optional type => 'default_type', # optional %other_bulk_params # optional max_count => 1_000, # optional max_size => 1_000_000, # optional max_time => 6, # optional verbose => 0 | 1, # optional on_success => sub {...}, # optional on_error => sub {...}, # optional on_conflict => sub {...}, # optional );
The
new()
method returns a new$bulk
object. You must pass your Search::Elasticsearch client as thees
argument.The
index
andtype
parameters provide default values forindex
andtype
, which can be overridden in each action. You can also pass any other values which are accepted by the bulk() method.See "flush()" for more information about the other parameters.
FLUSHING THE BUFFER
flush()
$result = $bulk->flush;
The
flush()
method sends all buffered actions to Elasticsearch using a bulk() request.Auto-flushing
An automatic "flush()" is triggered whenever the
max_count
,max_size
, ormax_time
threshold is breached. This causes all actions in the buffer to be sent to Elasticsearch.max_count
The maximum number of actions to allow before triggering a "flush()". This can be disabled by setting
max_count
to0
. Defaults to1,000
.max_size
The maximum size of JSON request body to allow before triggering a "flush()". This can be disabled by setting
max_size
to0
. Defaults to1_000,000
bytes.max_time
The maximum number of seconds to wait before triggering a flush. Defaults to
0
seconds, which means that it is disabled. Note: This timeout is only triggered when new items are added to the queue, not in the background.
Errors when flushing
There are two types of error which can be thrown when "flush()" is called, either manually or automatically.
Temporary Elasticsearch errors
A
Cxn
error like aNoNodes
error which indicates that your cluster is down. These errors do not clear the buffer, as they can be retried later on.Action errors
Individual actions may fail. For instance, a
create
action will fail if a document with the sameindex
,type
andid
already exists. These action errors are reported via callbacks.
Using callbacks
By default, any Action errors (see above) cause warnings to be written to
STDERR
. However, you can use theon_error
,on_conflict
andon_success
callbacks for more fine-grained control.All callbacks receive the following arguments:
$action
-
The name of the action, ie
index
,create
,update
ordelete
. $response
-
The response that Elasticsearch returned for this action.
$i
-
The index of the action, ie the first action in the flush request will have
$i
set to0
, the second will have$i
set to1
etc.
on_success
my $bulk = $es->bulk_helper( on_success => sub { my ($action,$response,$i) = @_; # do something }, );
The
on_success
callback is called for every action that has a successful response.on_conflict
my $bulk = $es->bulk_helper( on_conflict => sub { my ($action,$response,$i,$version) = @_; # do something }, );
The
on_conflict
callback is called for actions that have triggered aConflict
error, eg trying tocreate
a document which already exists. The$version
argument will contain the version number of the document currently stored in Elasticsearch (if found).on_error
my $bulk = $es->bulk_helper( on_error => sub { my ($action,$response,$i) = @_; # do something }, );
The
on_error
callback is called for any error (unless theon_conflict
) callback has already been called).Disabling callbacks and autoflush
If you want to be in control of flushing, and you just want to receive the raw response that Elasticsearch sends instead of using callbacks, then you can do so as follows:
my $bulk = $es->bulk_helper( max_count => 0, max_size => 0, on_error => undef ); $bulk->add_actions(....); $response = $bulk->flush;
CREATE, INDEX, UPDATE, DELETE
add_action()
$bulk->add_action( create => { ...params... }, index => { ...params... }, update => { ...params... }, delete => { ...params... } );
The
add_action()
method allows you to add multiplecreate
,index
,update
anddelete
actions to the queue. The first value is the action type, and the second value is the parameters that describe that action. See the individual helper methods below for details.Note: Parameters like
index
ortype
can be specified asindex
or as_index
, so the following two lines are equivalent:index => { index => 'index', type => 'type', id => 1, source => {...}}, index => { _index => 'index', _type => 'type', _id => 1, source => {...}},
Note: The
index
andtype
parameters can be specified in the params for any action, but if not specified, will default to theindex
andtype
values specified in "new()". These are required parameters: they must be specified either in "new()" or in every action.create()
$bulk->create( { index => 'custom_index', source => { doc body }}, { type => 'custom_type', id => 1, source => { doc body }}, ... );
The
create()
helper method allows you to add multiplecreate
actions. It accepts the same parameters as "create()" in Search::Elasticsearch::Client::6_0::Direct except that the document body should be passed as thesource
or_source
parameter, instead of asbody
.create_docs()
$bulk->create_docs( { doc body }, { doc body }, ... );
The
create_docs()
helper is a shorter form of "create()" which can be used when you are using the defaultindex
andtype
as set in "new()" and you are not specifying a customid
per document. In this case, you can just pass the individual document bodies.index()
$bulk->index( { index => 'custom_index', source => { doc body }}, { type => 'custom_type', id => 1, source => { doc body }}, ... );
The
index()
helper method allows you to add multipleindex
actions. It accepts the same parameters as "index()" in Search::Elasticsearch::Client::6_0::Direct except that the document body should be passed as thesource
or_source
parameter, instead of asbody
.delete()
$bulk->delete( { index => 'custom_index', id => 1}, { type => 'custom_type', id => 2}, ... );
The
delete()
helper method allows you to add multipledelete
actions. It accepts the same parameters as "delete()" in Search::Elasticsearch::Client::6_0::Direct.delete_ids()
$bulk->delete_ids(1,2,3...)
The
delete_ids()
helper method can be used when all of the documents you want to delete have the defaultindex
andtype
as set in "new()". In this case, all you have to do is to pass in a list of IDs.update()
$bulk->update( { id => 1, doc => { partial doc }, doc_as_upsert => 1 }, { id => 2, script => { script } upsert => { upsert doc } }, ... );
The
update()
helper method allows you to add multipleupdate
actions. It accepts the same parameters as "update()" in Search::Elasticsearch::Client::6_0::Direct. An update can either use a partial doc which gets merged with an existing doc (example 1 above), or can use ascript
to update an existing doc (example 2 above). More information onscript
can be found here: "update()" in Search::Elasticsearch::Client::6_0::Direct.AUTHOR
Enrico Zimuel <enrico.zimuel@elastic.co>
COPYRIGHT AND LICENSE
This software is Copyright (c) 2021 by Elasticsearch BV.
This is free software, licensed under:
The Apache License, Version 2.0, January 2004
Module Install Instructions
To install Search::Elasticsearch::Client::6_0, copy and paste the appropriate command in to your terminal.
cpanm Search::Elasticsearch::Client::6_0
perl -MCPAN -e shell install Search::Elasticsearch::Client::6_0
For more information on module installation, please visit the detailed CPAN module installation guide.