Buffered StreamWriter for writing over a long period of time

Question

I've got a program that will be running over a long duration (hours), and regularly writing output to a text file.

I'm looking to use a TextWriter implementation to write to the file, and I am concerned that keeping the file locked open during the entire length of operations may be problematic.

First question: Will there be performance problems (or other kinds) for keeping a stream open to a file for an extended duration?

If so, will a StreamWriter (opened with File Name constructor) manage opening and closing the file on a regular buffered basis for me, or will it hold the file open for the duration of its existence?

Lastly, is there a built in option for handling these more long-duration writes? Or will I need a custom Writer/Stream implementation?

Answer 1

Personally I would use one of the File.Appendxxx routines, which open the file, append the data and then close it again.

If I'm writing at such a rate that the cost of all this opening and closing is too high, then I add some kind of memory-based queue and flush it periodically.

If you're doing text-based logging, you might look at one of the umpteen logging frameworks for .NET, which can do this sort of stuff for you, along with things like rotating filenames, etc.

StreamWriter / FileStream, etc, will generally hold the file open until you dispose them.

Answer 2

Having a file opened indefinitely for it to be written to is not advised because this can cause issues when you need to back-up the file, edit the file, read the file, or even if the system crashes while it is in the process of writing to the file inevitably corrupting the data.

Will's answer does provide a solution of opening and appending the file when needed. The point here is if the file is not being written to 24/7 then there is no reason why you shouldn't close it.

To directly answer your question in the sense of the file needing to be open 24/7. I would use asynchronous methods to write to the file using tasks. This way you can then invoke another task to write a back-up of the file, say everyday. The back-up file will allow you to view the information of the production file.