# PODNAME: Neo4j::Driver::Net
# ABSTRACT: Explains the design of the networking modules

__END__

=pod

=encoding UTF-8

=head1 NAME

Neo4j::Driver::Net - Explains the design of the networking modules

=head1 VERSION

version 0.31

=head1 OVERVIEW

Each L<Neo4j::Driver::Session> has exactly one network controller
instance that is used by the session and all of its transactions
to communicate with the Neo4j server. This document discusses the
features and known limitations of the network controllers.

B<Unless you're planning to develop custom network adapters
for the driver, you probably don't need to read this document.>

The controllers don't communicate with the server directly. Instead,
they use another module that has responsibility for the actual
network transmissions. For HTTP connections, that other module can
be customised via L<Neo4j::Driver::Plugin/"http_adapter_factory">.
For Bolt, see L</"EXTENSIONS"> below.

Network responses received from the server will be parsed for Neo4j
statement results by the appropriate result handler for the response
format used by the server. A custom networking module can also
provide custom response parsers, for example implemented in XS code.

Please note that the division of labour between sessions or
transactions on the one hand and networking controllers on the other
hand is an internal implementation detail of the driver and as such
is B<subject to unannounced change.> While some of those details are
explained in this document, this is done only to help contributors
and users of I<public> APIs better understand the driver's design.
See L<Neo4j:::Driver::Plugin/"USE OF INTERNAL APIS"> for more
information on this topic.

=head1 SYNPOSIS

 $helper = Neo4j::Driver::Net::HTTP->new({
   net_module => 'Local::Neo4jUserAgentHTTP',
 });
 
 $helper->_set_database($db_name);
 $helper->{active_tx} = ...;
 @results = $helper->_run($tx, @statements);
 
 # Parsing a JSON result
 die unless $helper->{http_agent}->http_header->{success};
 $json_coder = $helper->{http_agent}->json_coder;
 $json_coder->decode( $helper->{http_agent}->fetch_all );

B<WARNING:> Some of these calls are private APIs.
See L<Neo4j:::Driver::Plugin/"USE OF INTERNAL APIS">.

=head1 FEATURES

The networking controllers primarily deal with the following tasks:

=over

=item * Establish a database connection.

=item * Provide L<Neo4j::Driver::ServerInfo>.

=item * Handle certain generic protocol requirements
(such as HTTP content negotiation).

=item * Sync state between server transactions and
driver transaction objects.

=item * Control the translation of Cypher statements
to network transmissions, and of network transmissions
to statement results.

=back

HTTP connections use proactive content negotiation
(L<RFC 7231|https://tools.ietf.org/html/rfc7231#section-3.4.1>)
to obtain a suitable response from the Neo4j server. The driver
supports both Jolt and JSON as result formats. There is also
a fallback result handler, which is used to parse error
messages out of C<text/*> responses. The HTTP result handlers
are individually queried for the media types they support.
This information is cached by the networking controller.

All result handlers inherit a common interface from
L<Neo4j::Driver::Result>. They provide methods to initialise and
bless result data records as L<Neo4j::Driver::Record> objects.
Result handlers are also responsible that all values returned
from Neo4j are provided to users in the format that is documented
for L<Neo4j::Driver::Record/"get">. For backwards compatibility,
a lot of the internal data structures currently match the format
of Neo4j JSON responses.

The first HTTP connection to a Neo4j server is always made to the
L<Discovery API|https://neo4j.com/docs/http-api/4.2/discovery/>,
which is used to obtain L<Neo4j::Driver::ServerInfo> and the
transaction endpoint URI template. These are the only GET
requests made by the driver.
Because of a known issue with Neo4j, the C<Accept> request
header field needs to be varied by HTTP request method
(L<#12644|https://github.com/neo4j/neo4j/issues/12644>).

With HTTP being a stateless protocol, Neo4j supports multiple
concurrent transactions by using a different URL for each one
in a REST-like fashion. For requests made to such explicit
transaction endpoints, the Neo4j
L<Transactional HTTP API|https://neo4j.com/docs/http-api/4.2/actions/>
always provides transaction status information in the response.
Transactions that remain open include an expiration time. The
networking controller parses and stores this timestamp and uses it to
track which transactions are still open and which have timed out.
The origination C<Date> field is used to synchronise the clocks
of the driver and the Neo4j server
(L<RFC 7231|https://tools.ietf.org/html/rfc7231#section-7.1.1.2>).

Bolt, on the other hand, currently only supports a single open
transaction per connection. While a Bolt connection can be viewed
as a simple state machine in the backend Bolt library (see
L<Bolt Protocol Server State Spec|https://7687.org/bolt/bolt-protocol-server-state-specification-4.html#appendix---bolt-message-state-transitions>),
L<Neo4j::Bolt> currently doesn't allow users to directly observe
state changes, so it is currently somewhat difficult to determine
the Bolt connection state. The driver attempts to infer it based
on the behaviour of L<Neo4j::Bolt>, and mostly gets it right, but
there may be some as-yet-unknown issues. Bug reports are welcome.

One key difference between HTTP and Bolt is the handling of
transaction state in case of Neo4j errors. According to
L<Neo4j Status Codes|https://neo4j.com/docs/status-codes/4.2/>,
the effect of errors is always a transaction rollback. On HTTP,
these rollbacks take place immediately. On Bolt, however, the
transaction is typically only marked as uncommittable on the Neo4j
server, but the Bolt connection is not actually put into the
C<FAILED> state immediately. To try and work around this difference
between HTTP and Bolt, this driver's Bolt network controller always
attempts an explicit transaction rollback if faced with any error
condition. Again, this approach mostly gets it right, but there
may be some remaining issues, particularly when network errors and
server errors happen simultaneously.

=head1 COMPATIBILITY

L<Neo4j::Driver> S<version 0.21> is compatible with
S<L<Neo4j::Bolt> 0.01> or later.

When using HTTP, S<L<Neo4j::Driver> 0.21> supports determining the
version of any Neo4j server via L<Neo4j::Driver::Session/"server">.
This even works on S<Neo4j 1.x>, but running statements on
S<Neo4j 1.x> will fail, because it lacks the transactional API.

S<L<Neo4j::Driver> 0.21> is compatible with Neo4j S<versions 2.x>,
S<3.x, and 4.x>. It supports HTTP responses in the
L<formats|https://neo4j.com/docs/http-api/4.2/actions/result-format/>
JSON and Jolt (both strict mode and sparse mode).

For Bolt as well as HTTP, future versions of the driver will tend
to implement new requirements in order to stay compatible with
newer versions of Neo4j. Support for old Neo4j or library versions
is likely to only be dropped with major updates to the driver
(such as S<0.x to 1.x>).

=head1 BUGS AND LIMITATIONS

As described above, there may be cases in which the state of a
Bolt connection is not determined correctly, leading to unexpected
failures. If such bugs do exist, they are expected to mostly happen
in rare edge cases, but please be sure to report any problems.

Clock synchronisation using the HTTP C<Date> header does not take
into account network delays. In case of high network latency, the
driver may treat transactions as open even though they have already
expired on the server. To address this, you could either increase
the transaction idle timeout in F<neo4j.conf> or manipulate the
return value of C<date_header()> in a custom network adapter.

The metadata in HTTP JSON responses is often insufficient to fully
describe the response data. In particular:

=over

=item * Path metadata doesn't include node labels or relationship
type (L<#12613|https://github.com/neo4j/neo4j/issues/12613>).

=item * Records with fields that are maps or lists have unparsable
metadata (L<#12306|https://github.com/neo4j/neo4j/issues/12306>).

=item * Byte arrays are coded as lists of integers in JSON results.

=back

As of S<Neo4j 4.2>, the Jolt documentation for byte arrays
doesn't match the implementation
(L<#12660|https://github.com/neo4j/neo4j/issues/12660>). Future
Neo4j versions might fix the implementation to match the docs.

Neo4j spatial and temporal types are not currently implemented
in all response format parsers.

=head1 EXTENSIONS

The C<net_module> config option available in driver S<versions 0.21>
S<through 0.30> has been replaced with an experimental plug-in API;
see L<Neo4j::Driver::Deprecations/"Custom networking modules">.

=head2 Custom Bolt networking modules

By default, Bolt networking uses the amazing XS module
L<Neo4j::Bolt> by Mark A. Jensen (MAJENSEN), which in turn uses the
S<C library> L<libneo4j-client|https://neo4j-client.net/>
to actually connect to the Neo4j server. Updates and improvements
are quite possibly best made directly in those libraries, so that
not only L<Neo4j::Driver>, but also other users benefit from them.

There is currently no public API for custom Bolt network adapters;
see L<Neo4j::Driver::Plugin/"Network adapter API for Bolt">.
However, the driver's test suite still uses the former C<net_module>
option internally, so it will remain available for the time being.

 use Local::MyBolt;
 $driver->config(uri => 'bolt://...');
 $driver->{net_module} = 'Local::MyBolt';  # private API

The module name provided will be used in place of
L<Neo4j::Bolt> and will have to match its API I<exactly.>
Note that there is no support for this approach. For details,
see L<Neo4j:::Driver::Plugin/"USE OF INTERNAL APIS">.

=head2 Custom HTTP networking modules

L<Neo4j::Driver> includes a default HTTP network adapter.
The included adapter may change in future. As of S<version 0.21>,
the default adapter uses L<LWP> directly. Earlier versions used
L<REST::Client>.

The driver's test suite still uses the former C<net_module>
option internally. Modules to be used as C<net_module> must
implement the API for an HTTP adapter; see
L<Neo4j:::Driver::Plugin/"Network adapter API for HTTP">.
Additionally, they must implement the following method:

=over

=item new

 sub new {
   my ($class, $driver) = @_;
   ...
 }

Initialises the object. May or may not establish a network
connection. May access C<$driver> config options using the
method L<Neo4j::Driver/"config"> only.

=back

=head2 Custom result handlers

This section has been replaced by
L<Neo4j:::Driver::Plugin/"Result handler API">.

=head1 USE OF INTERNAL APIS

This section has been replaced by
L<Neo4j:::Driver::Plugin/"USE OF INTERNAL APIS">.

=head1 AUTHOR

Arne Johannessen <ajnn@cpan.org>

=head1 COPYRIGHT AND LICENSE

This software is Copyright (c) 2016-2022 by Arne Johannessen.

This is free software, licensed under:

  The Artistic License 2.0 (GPL Compatible)

=cut