The burden of suspicion falls upon the people doing the judging. Not the ones who are being judged: single or in non-marital relationships.
Your point that married people are seen as less of a spouse thief is flawed and illogical, as cheaters are in most cases, ironically, married people itself. Your point (or rather the people in them whose point you're tryin to make) implies that married people are angels who have fallen from heaven totally incapable of adultery or infidelity.
This paints single/partnered people in poor light and directly implies that all single people are criminals by default unless proven otherwise in any new residential community or town, which is extremely rude, condescending and downright ridiculous. Stop victim blaming!
(and yes, my rant is not to you, but the people on behalf of whose point you are making)