Correct, AQ will try to keep the best estimation, but if the GPS information is totally unreliable it will fall back to altitude hold (if you're in pos hold). So, absolute position is no longer possible and the AQ will start to drift.
All other modes are only available if the gpx fix is reliable so they also will fall back to the basics..
To be honest, I have never tested that, it is hard to do in real life. A teammember is working on an 'hardware in the loop' simulation where this kind of tests will be possible without even leaving the bench..