Automated Operational Modal Analysis is recurrent for the modal identification of civil structures. However, its use for rotating machines is hampered by non-linearities, time-invariance, and overall superimposing harmonic excitation. In particular, harmonic excitation can either mask or disturb the identification of structural modes with natural frequencies close to the harmonics. Sometimes, harmonic frequencies can also be identified as modes with low damping, disturbing the modal identification of the system. The literature has already proposed harmonic removal methods, but most have only been tested on a few simple rotating systems. The idea behind this study is to evaluate a pre-processing method for harmonic removal in a complex rotating system, composed of a rotor supported by magnetic bearings and influenced by gas seals whose rotation is given by an electric motor. Data was acquired with different rotation speeds and the harmonic removal method was applied to generate clean signals. To evaluate the method's performance, AOMA was applied to the original and clean signals. Analysis reveals that the harmonic removal method was capable of both removing the harmonics and enabling the identification of physical modes previously masked by them. The modal identification was also compared to references from a mathematical model of the test rig and Experimental Modal Analysis tests. Results show that the combination of the harmonic removal method and AOMA enabled the identification of many physical modes of interest, especially the rotor mode that defines the stability margin of the system, that is, the operating point at which the system becomes unstable. Differences between the identified modal parameters and the references exist, which is expected, especially in the identification of damping ratios. These presented results are another contribution to support condition monitoring of complex rotating systems via Operational Modal Analysis.