Catmandu 1.20

On May 21th 2019, Nicolas Steenlant (our main developer and guru of Catmandu) released version 1.20 of our Catmandu toolkit with some very interesting new features. The main addition is a brand new way how Catmandu Fix-es can be implemented using the new Catmandu::Path implementation. This coding by Nicolas will make it much easier and straightforward to implement any kind of fixes in Perl.

In the previous versions of Catmandu there were only two options to create new fixes:

  1. Create a Perl package in the Catmandu::Fix namespace which implements a fix method. This was very easy: update the $data hash you got as first argument, return the updated $data and you were done. Then disadvantage was that accessing fields in a deeply nested record was tricky and slow to code.
  2. Create a Perl package in the Catmandu::Fix namespace which implemented emit functions. These were functions that generate Perl code on the fly. Using emit functions it was easier to get fast access to deeply nested data. But, to create Fix packages was pretty complex.

In Catmandu 1.20 there is now support for a third and easy way to create new Fixes using the Catmandu::Fix::Builder and Catmandu::Fix::Path class. Let me give an simple example of a skeleton Fix that does nothing:

package Catmandu::Fix::rot13;

use Catmandu::Sane;
use Moo;
use Catmandu::Util::Path qw(as_path);
use Catmandu::Fix::Has;

with 'Catmandu::Fix::Builder';

has path => (fix_arg => 1);

sub _build_fixer {
   my ($self) = @_;
   sub {
     my $data = $_[0];
     # ..do some magic here ...
     $data;
   }
}

1;

In the code above we start implementing a rot13(path) Fix that should read a string on a JSON path and encrypt it using the ROT13 algorithm. This Fix is only the skeleton which doesn’t do anything. What we have is:

    • We import the as_path method be able to easily access data on JSON paths/
    • We import Catmandu::Fix::Has to be able to use has path constructs to read in arguments for our Fix.
    • We import Catmandu::Fix::Builder to use the new Catmandu 1.20 builder class provides a _build_fixermethod.
    • The builder is nothing more than a closure that reads the data, does some action on the data and return the data.

We can use this skeleton builder to implement our ROT13 algorithm. Add these lines instead of the # do some magic part:

# On the path update the string value...
as_path($self->path)->updater(
   if_string => sub {
       my $value = shift;
       $value =~ tr{N-ZA-Mn-za-m}{A-Za-z};
       $value;
   },
)->($data);

The as_path method receives a JSON path string an creates an object which you can use to manipulate data on that path. One can update the values found with the updater method, or read data at that path with the getter method or create a new path with the creator method. In our example, we update the string found at the JSON path using if_string condition. The updaterhas many conditions:

  • if_string needs a closure what should happen when a string is found on the JSON path.
  • if_array_ref needs a closure what should happen when an array is found on the JSON path.
  • if_hash_refneeds a closure what should happen when a hash is found on the JSON path.

In our case we are only interested in transforming strings using our rot13(path) fix. The ROT13 algorithm is very easy and only switched the order of some characters. When we execute this fix on some sample data we get this result:


$ catmandu -I lib convert Null to YAML --fix 'add_field(demo,hello);rot13v2(demo)'
---
demo: uryyb
...

In this case the Fix can be written much shorter when we know that every Catmandu::Path method return a closure (hint: look at the ->($data) in the code. The complete Fix can look like:

package Catmandu::Fix::rot13;

use Catmandu::Sane;
use Moo;
use Catmandu::Util::Path qw(as_path);
use Catmandu::Fix::Has;

with 'Catmandu::Fix::Builder';

has path  => (fix_arg => 1);

sub _build_fixer {
    my ($self) = @_;
    # On the path update the string value...
    as_path($self->path)->updater(
      if_string => sub {
         my $value = shift;
         $value =~ tr{N-ZA-Mn-za-m}{A-Za-z};
         $value;
      },
   );
}

1;

This is as easy as it can get to manipulate deeply nested data with your own Perl tools. All the code is in Perl, there is no limit on the number of external CPAN packages one can include in these Builder fixes.

We can’t wait what Catmandu extensions you will create.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s