Recent advances in Bayesian approaches to operational modal analysis (OMA) have motivated the need to explore a range of possible prior distributions over the parameters of the state space representation of linear time-invariant dynamic systems. Choosing prior structures that can guarantee physically meaningful solutions in the context of structural dynamics, a property not necessarily guaranteed in many state space model-based system identification methods (e.g. stochastic subspace identification), is of particular interest. In this work, a range of prior model definitions for state space models are explored, inclduing those which embed and enforce existing knowledge of the physics, with the view that physically meaningful posterior estimates could be obtained. Four potential priors for the state matrix are shown, with samples of these priors visualised on the elements of the state matrix and the corresponding modal properties. The ability of these priors to satisfy the condition of providing physically meaningful estimates is then discussed. Finally, considerations or challenges when selecting such priors in the context of Bayesian inference are identified.