Variable-Depth Markov Models

What would a Markov model act like if we allowed variable depth instead of fixing it? First, if you imagine the MM going top-down from initial to final states, it might make sense to group states that always appear in the same context — horizontal generalization into a state-type. Allowing variable depth could be implemented by vertical generalization into state-chain-types. In text processing/generation, this might appear as the creation of single states from idiomatic expressions.