Reason for (win) x64-calling convention restrictions in epilogues

Question

I'm trying to understand the actual reasoning behind the restrictions that the x64 calling convention has on epilogues, at least on Windows? Quoting from MSDN:

The epilog code must follow a strict set of rules for the unwind code to reliably unwind through exceptions and interrupts. These rules reduce the amount of unwind data required, because no extra data is needed to describe each epilog. Instead, the unwind code can determine that an epilog is being executed by scanning forward through a code stream to identify an epilog.

I get what they are saying, but I don't understand the practical implications, at least for exceptions. Given the limited set of instructions that a valid epilogue can contain, I don't see what type of exceptions could even occur in the first place. If we look at the possible instructions in the epilogue, with their most complex example:

   lea      RSP, -128[R13]
    ; epilogue proper starts here
    add      RSP, fixed-allocation-size
    pop      R13
    pop      R14
    pop      R15
    ret

We can't really do anything that would result in an exception being thrown. pop could result in stack-underflow, but if that's happening, we don't have a valid return-address left, and thus no valid unwind could occur. Or if RSP points to some invalid memory address, where we would once again not really have a valid target for ret. So we would, at most, invoke the top-level SEH handler for the particular function that the epilogue belongs to, in such a scenario.

Is the reason really just for interrupts (which I have no knowledge or understand of)? Or am I missing how an x64-compliant epilogue could trigger an exception? If the reasoning was just to not have exceptions in epilogues, they wouldn't need the whole machinery of scanning the code-stream to determine if an exception occurs within an epilogue.

As for why I'm asking this: I'm doing some trickery for my native codes stack-switching, that includes generating technically invalid epilogues (and prologs, to an extent). I'm trying to make sure that I don't run into unexpected side-effects, as long as I make sure not to allow recoverable exceptions to occur there (for a normal user-mode application).