Read header data from files on remote server

Posted by rejeep on Stack Overflow See other posts from Stack Overflow or by rejeep
Published on 2009-03-12T20:29:37Z Indexed on 2010/04/22 12:03 UTC
Read the original article Hit count: 142

Filed under:
|
|
|
|

Hi!

I'm working on a project right now where I need to read header data from files on remote servers. I'm talking about many and large files so I cant read whole files, but just the header data I need.

The only solution I have is to mount the remote server with fuse and then read the header from the files as if they where on my local computer. I've tried it and it works. But it has some drawbacks. Specially with FTP:

  • Really slow (FTP is compared to SSH with curlftpfs). From same server, with SSH 90 files was read in 18 seconds. And with FTP 10 files in 39 seconds.
  • Not dependable. Sometimes the mountpoint will not be unmounted.
  • If the server is active and a passive mounting is done. That mountpoint and the parent folder gets locked in about 3 minutes.
  • Does timeout, even when there's data transfer going (guess this is the FTP-protocol and not curlftpfs).

Fuse is a solution, but I don't like it very much because I don't feel that I can trust it. So my question is basically if there's any other solutions to the problem. Language is preferably Ruby, but any other will work if Ruby does not support the solution.

Thanks!

© Stack Overflow or respective owner

Related posts about fuse

Related posts about ftp