Write asynchronously to file in perl

Posted by Stefhen on Stack Overflow See other posts from Stack Overflow or by Stefhen
Published on 2010-04-12T00:32:01Z Indexed on 2010/04/12 0:43 UTC
Read the original article Hit count: 498

Filed under:
|
|

Basically I would like to:

  1. Read a large amount of data from the network into an array into memory.
  2. Asynchronously write this array data, running it thru bzip2 before it hits the disk.

repeat..

Is this possible? If this is possible, I know that I will have to somehow read the next pass of data into a different array as the AIO docs say that this array must not be altered before the async write is complete. I would like to background all of my writes to disk in order as the bzip2 pass is going to take much longer than the network read.

Is this doable? Below is a simple example of what I think is needed, but this just reads a file into array @a for testing.

use warnings;
use strict;
use EV;
use IO::AIO;
use Compress::Bzip2;
use FileHandle;
use Fcntl;


my @a;

print "loading to array...\n";
while(<>) {
  $a[$. - 1] = $_;
}
print "array loaded...\n";


my $aio_w = EV::io IO::AIO::poll_fileno, EV::WRITE, \&IO::AIO::poll_cb;


aio_open "./out", O_WRONLY || O_NONBLOCK, 0, sub {
  my $fh = shift or die "error while opening: $!\n";

  aio_write $fh, undef, undef, $a, -1, sub {
    $_[0] > 0 or die "error: $!\n";
    EV::unloop;
  };
};

EV::loop EV::LOOP_NONBLOCK;

© Stack Overflow or respective owner

Related posts about perl

Related posts about asynchronous