carlos@blog ~/notes>

\o/

Running Perl Workers on Iron.IO

Despite not being listed as an option on their documentation, it is possible to run perl on their IronWorker service.

TL;DR (or Show Me The Code): working example, full sources.

Quick start

IronWorkerNG (their upload/management tool)

We will need their management tool to upload code.

gem install iron_worker_ng

The credentials

IronWorkerNG reads the credentials from a file named iron.json.
To get it, we need to login to https://hud.iron.io/dashboard, click on the key-icon and download iron.json to the root directory of the project.

Resolving dependencies on CPAN modules

Iron.IO doesn’t come with any perl module other than what is in core, so we need to bring in all our dependencies.

For that we will use Carton

$ cpanm -nq Carton   # install carton

add our dependencies to cpanfile

$ cat cpanfile
requires 'Data::Printer';
requires 'File::Slurp';
requires 'Mojolicious';

and install them to a ./local/ directory

$ carton install

Seting up the worker

The configuration for the worker is defined in a <service>.worker file, so we create a simple one that tells it to run our perl script and to upload our dependencies (generated by Carton, above) together with the code.

$ cat iron-pl.worker
runtime 'perl'
exec 'iron-pl.pl'
dir 'local/lib/perl5'

This will upload our code together with the ./local/lib/perl5 directory (as ./perl5 on their system)

For more information, check the .worker documentation.

Writing the code!

This is what will be run when you schedule some tasks.
We can go crazy on that one, but here I’d like to highlight just a few things:

#!/usr/bin/env perl
use 5.010; use utf8; use strict; use warnings;
(...)

use lib 'perl5';

We need to load the dependencies from the local directory, thus the use lib above.

(...)
my %args = @ARGV;

They pass arguments as ('-d', '/task/', '-e', 'production', '-id', '5277ca9ed16f93360109d0aa', '-payload', '/task/task_payload.json') so we can load them as a Hash and access it as $args{-d} :–)

The data passed from the scheduled task to the worker comes in a .json file, so we need to parse that

my $payload = Mojo::JSON->new->decode( read_file($args{-payload}) );

For more info (and executable code), check the full sources on github.

Deploy

With all pieces ready, we can upload the code to their platform using their management tool.

$ iron_worker upload iron-pl.worker
    ------> Creating client
            Project 'ironing-board' with id='5277a2b987a3b90005000044'
    ------> Creating code package
            Found workerfile with path='iron-pl.worker'
            Detected exec with path='iron-pl.pl' and args='{}'
            Merging dir with path='local/lib/perl5' and dest=''
            Code package name is 'iron-pl'
    ------> Uploading code package 'iron-pl'
            Code package uploaded with id='5277af07c7abc62bd5098755' and revision='4'
            Check 'https://hud.iron.io/tq/projects/5277a2b987a3b90005000044/code/5277af07c7abc62bd5098755' for more info

Add tasks to the queue

$ iron_worker queue iron-pl -p '{"tags":["iron.io","perl"]}'
    ------> Creating client
            Project 'ironing-board' with id='5277a2b987a3b90005000044'
    ------> Queueing task
            Code package 'iron-pl' queued with id='5277ca9ed16f93360109d0aa'
            Check 'https://hud.iron.io/tq/projects/5277a2b987a3b90005000044/jobs/5277ca9ed16f93360109d0aa' for more info

Check results

We can either go straight to the web interface using the url listed above or use their management tool:

    $ iron_worker log 5277ca9ed16f93360109d0aa
    ------> Creating client
            Project 'ironing-board' with id='5277a2b987a3b90005000044'
    ------> Getting log for task with id='5277ca9ed16f93360109d0aa'
    Iron-Play v0.0.1
    2013-11-04T16:26:10
    Environment: {
        HOME              "/task",
        LANG              "en_US.UTF-8",
        LD_LIBRARY_PATH   ".:./lib:./__debs__/usr/lib:./__debs__/usr/lib/x86_64-linux-gnu:./__debs__/lib:./__debs__/lib/x86_64-linux-gnu",
        LOGNAME           "nobody",
        MAIL              "/var/mail/nobody",
        OLDPWD            "/task",
        PATH              ".:./bin:./__debs__/usr/bin:./__debs__/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games",
        PWD               "/task",
        SHELL             "/bin/sh",
        SUDO_COMMAND      "/usr/bin/ruby run.rb --sleep 60 -e production -n 1 -j /mnt/iron-jail",
        SUDO_GID          1000,
        SUDO_UID          1000,
        SUDO_USER         "ubuntu",
        TERM              "unknown",
        USER              "nobody",
        USERNAME          "root"
    }
    Arguments: {
        -d         "/task/",
        -e         "production",
        -id        "5277ca9ed16f93360109d0aa",
        -payload   "/task/task_payload.json"
    }
    Payload (/task/task_payload.json): \ {
        tags   [
            [0] "iron.io",
            [1] "perl"
        ]
    }

    Top users for tag 'iron.io':
    AnaelFavre: 417
    thousandsofthem: 446
    Travis R: 6935

    Top users for tag 'perl':
    ikegami: 106098
    mpapec: 10324
    TLP: 40585

    Done.

But, but..

Of course, this example is not very useful, manually scheduling tasks and getting results from calling iron_worker log <id> is not really something interesting :–)

To be useful it would need to be automatically triggered by some event and produce usable results (updating a databse, notifying another service, etc..) but I assume you already know this if you’re searching for how to run perl on IronWorker.

So, go crazy and drop me a hello on the comments bellow if this post was somehow helpful.

Cheers! \o/

Comments