No one can really say for certain how jagex catches gold selling accounts, it isnt something publicly released for obvious reasons, I doubt many of the jmods know either.
As mentioned above, gold buyers generally get off its the sellers that usually catch the bans. I obviously cant say for certain as well but I suspect that if theres a large amount of gold flowing through your account to and from many different ips from different locations around the world, your account is likely to get flagged and will be monitored closely.
Gold buyers usually recieve a 2week ban if caught rwting. If caught again it will escalate to a perm.
Gold sellers recieve a straight perm ban is caught.
I suspect jagex also might flag the ips of the accounts caught for rwt. If another account happens to connect with that ip and has large gold flow to and fro it will almost certainly recieve a perm straight. So always use different ips from your main when buying/selling.
To reduce your ban rate a little, buy in increments (20m a time/across different accounts). Use different ip from your main and dont openly admit to rwting in game.