How would I go about writing my own stream manipulator class?

Basically what I'm trying to wrap my head around is storing the reference to the underlying stream in the writer. For example, when writing to a memory stream using a StreamWriter, when a Write() is made, the underlying memory stream is written to.

Can I store the reference to an underlying stream without using pointers or unsafe code? Even if it was just a string I wanted to "write" to.

Really this has little to do with stream writers, and I'm just wondering how I could store references in a class. The StreamWriter was the best example I could come up with for this.