Jess42 wrote:How am I subject to possible removal?
You are subject to possible removal in exactly the same way as in other family member whose right to reside depends on their spouse being a 'qualified person'. If your husband were not a qualified person, and you arrived in the UK together, you yourself would have no right to be in the UK once you had completed your initial 3 months. The Home Office has not yet admitted that your husband is a qualified person.
There is some doubt as to whether the HO has the right to retain your passport. One poster reported that her solicitor would challenge the retention, which certainly looks like an abuse of powers, but I have seen no further report. The key point to the argument would be that your removal is actually unlikely (the HO has no reason to believe your husband is not a qualified person or that your marriage is a marriage of convenience), and therefore the passport should not be retained.
It ought to be possible to receive your passport from the Home Office as you leave the UK. I am not sure how easy that is to arrange. You would get back to the UK border using your US passport, and then re-enter on the basis that your husband was a qualified person residing in the UK. Some documentary evidence would be required. (Husbands are a handy part of a substitute for a visa.)