• NAME
• VERSION
• SYNOPSIS
• DESCRIPTION
• METHODS
  • decode_utf8
  • is_non_unicode
  • to_string, '""' operator
  • str_compare, 'cmp' operator
  • str_concat, '.' operator
  • add_json_filter
  • TO_JSON
  • FROM_JSON
• AUTHOR
• COPYRIGHT AND LICENSE

NAME

DataStore::CAS::FS::InvalidUTF8 - Wrapper to represent non-utf8 data in a unicode context

VERSION

version 0.011000

SYNOPSIS

  my $j= JSON->new()->convert_blessed;
  DataStore::CAS::FS::InvalidUTF8->add_json_filter($j);
  my $x= DataStore::CAS::FS::InvalidUTF8->decode_utf8("\x{FF}");
  my $json= $j->encode($x);
  my $x2= "".$j->decode($json);
  is( $x, $x2 );
  ok( !utf8::is_utf8($x2) );

DESCRIPTION

Much like using 'i' (or j) as the square root of -1, InvalidUTF8 allows a value which should have been utf-8, but isn't, to exist alongside the others.

Combining InvalidUTF8 parts to make a valid utf-8 string will automatically decode the utf-8 into the resulting unicode string.

Comparing InvalidUTF8 with a regular perl string will first convert the string to a UTF-8 representation, and then do a byte-wise comparison.

InvalidUTF8 can also safely pass through JSON, if the filter is added to the JSON decoder, and "allow_blessed" is set on the encoder.

METHODS

decode_utf8

  $string_or_ref= $class->decode_utf8( $byte_str )

If the $byte_str is valid UTF-8, this method returns the decoded perl unicode string. If not, it returns the string wrapped in an instance of InvalidUTF8.

is_non_unicode

This method returns true, and can be used in tests like

  if ($_->can('is_non_unicode')) { ... }

as a way of detecting InvalidUTF8 objects by API rather than class hierarchy.

to_string, '""' operator

Returns the original string.

str_compare, 'cmp' operator

Converts peer to utf-8 bytes, then compares the bytes.

str_concat, '.' operator

Converts the peer to utf-8 bytes, concatenates the bytes, and then re-evaluates whether the result needs to be wrapped in an instance of InvalidUTF8.

add_json_filter

  $json_instance= $class->add_json_filter($json_instance);

Applies a filter to the JSON object so that when it encounters

  { "*InvalidUTF8*": "$string" }

it will inflate the string using the FROM_JSON method.

TO_JSON

Called by the JSON module when convert_blessed is enabled. Returns

  { "*InvalidUTF8*" => $original_str }

which can be converted back to a InvalidUTF8 object during decode_json, if the filter is applied.

FROM_JSON

Pass this function to JSON's filter_json_single_key_object with a key of *InvalidUTF8* to restore the objects that were serialized. It takes care of calling utf8::downgrade to undo the JSON module's unicode conversion.

AUTHOR

Michael Conrad <mconrad@intellitree.com>

COPYRIGHT AND LICENSE

This software is copyright (c) 2013 by Michael Conrad, and IntelliTree Solutions llc.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.