The task is to use the set of files for several threads appending to the "last" file (buffering), and another thread sequentially reading what was appended when it thinks appropriate and mark the read records as processed (unbuffering). If nothing to unbuffer then both buffering and unbuffering point to end of last file. The operations are relatively simple and fast, therefore ok to use mutexes to control race conditions.
Right now I implemented this by opening/closing buffering files when there's a need to buffer or unbuffer, with mutexes, therefore if threads try to access the same file it will never happen concurrently - only one file is open at any given time.
But I have got the architectural problem - these files are located on the removable storage device, and if I remove (eject) this device the application does not notice it (most of the times because it accesses the files relatively rarely) and keeps writing/reading data into the location previously shadowed by the mounted storage device - while I expect it to notice the genuine file/volume removal and perform buffering reinitialization.
One of the possible ways I see is to once open buffering files for first (buffering/appending) and second (unbuffering/updating) purposes and keep FDs persistent, but there could be issues when this will happen to be the same file (therefore two FDs/FPs pointing to the same physical file). However it will cause write to removed volume raising the error and it will be the trigger for buffering reinitialization.
I am having a feeling like I am trying to reinvent the bicycle, or there's better solution to the volume removal noticing issue.
Will be glad for suggestions. Thanks!
Update: Have considered a number of ways to solve issue starting from notifying application on storage hotplug event (which is a workaround and not solution) ending with designing my own database or file system (which would be ineffective use of time). Right now I have found that I can obtain dev_t and inot_t from the fstat call, comparing to original location file was initialized with. If both match - device is the same as it was before, if something differs - system config has changed and I must reinitialize the buffering subsystem. Still must test to see if it satisfies the needs.
For implementing buffering to a file with volume removal notice and reinitialization:
Buffering to File:
Use a buffer to accumulate data before writing to the file. Flush the buffer when it reaches a certain size. Volume Removal Notice:
Periodically check the file size. If it decreases unexpectedly, assume volume removal. Reinitialization:
Implement logic (e.g., close and reopen the file) in response to volume removal to handle reinitialization.