I'm looking toward building a largish ZFS Pool (150TB+), and I'd like to hear people experiences about data loss scenarios due to failed hardware, in particular, distinguishing between instances where just some data is lost vs. the whole filesystem (of if there even is such a distinction in ZFS).

For example: let's say a vdev is lost due to a failure like an external drive enclosure losing power, or a controller card failing. From what I've read the pool should go into a faulted mode, but if the vdev is returned the pool should recover? or not? or if the vdev is partially damaged, does one lose the whole pool, some files, etc.?

What happens if a ZIL device fails? Or just one of several ZILs?

Truly any and all anecdotes or hypothetical scenarios backed by deep technical knowledge are appreciated!

Thanks!

Update:

We're doing this on the cheap since we are a small business (9 people or so) but we generate a fair amount of imaging data.

The data is mostly smallish files, by my count about 500k files per TB.

The data is important but not uber-critical. We are planning to use the ZFS pool to mirror 48TB "live" data array (in use for 3 years or so), and use the the rest of the storage for 'archived' data.

The pool will be shared using NFS.

The rack is supposedly on a building backup generator line, and we have two APC UPSes capable of powering the rack at full load for 5 mins or so.