Write asynchronously to file in perl
- by Stefhen
Basically I would like to:
Read a large amount of data from the network into an array into memory.
Asynchronously write this array data, running it thru bzip2 before it hits the disk.
repeat..
Is this possible? If this is possible, I know that I will have to somehow read the next pass of data into a different array as the AIO docs say that this array must not be altered before the async write is complete. I would like to background all of my writes to disk in order as the bzip2 pass is going to take much longer than the network read.
Is this doable? Below is a simple example of what I think is needed, but this just reads a file into array @a for testing.
use warnings;
use strict;
use EV;
use IO::AIO;
use Compress::Bzip2;
use FileHandle;
use Fcntl;
my @a;
print "loading to array...\n";
while(<>) {
$a[$. - 1] = $_;
}
print "array loaded...\n";
my $aio_w = EV::io IO::AIO::poll_fileno, EV::WRITE, \&IO::AIO::poll_cb;
aio_open "./out", O_WRONLY || O_NONBLOCK, 0, sub {
my $fh = shift or die "error while opening: $!\n";
aio_write $fh, undef, undef, $a, -1, sub {
$_[0] > 0 or die "error: $!\n";
EV::unloop;
};
};
EV::loop EV::LOOP_NONBLOCK;
