If a man doesn't want more children, then instead of having sex with his wife, he has sex with a prostitute, so any children aren't his economic problem. The family stays fixed in size, but the wife has a somewhat sexless life. That situation sounds a great deal like Victorian London, where the women were often sex starved and the number of brothels was astonishing.
This process also keeps down birthrates in general. A woman can only get pregnant once every nine months, so all the extra sex that she has with all those partners usually produces no children. By becoming a generic sink, the prostitute ensures that most sexual activity produces no offspring. Thus, prostitution should decrease the overall birth rate.
Does that really hold up? Is this already a theory? I don't know, but I do find the idea fascinating. It really helps to explain certain social behavior, and why this behavior was tolerated. If a woman doesn't want more offspring, but she also doesn't want a drain on her family's resources because her husband has a mistress, then tolerating a far smaller economic drain via sex workers becomes a prudent strategy. It effectively create a social firewall against any incurred genetic obligation.
Once effective birth control comes into play, this strategy dissipates because it allows also more sex inside monogamy, thus safely reducing family size, while also allowing women to move into non-monogamy.
Note that a similar sort of strategy works on the upper economic levels. A ruler wants male children, but he doesn't want too many children as this will cause problems, so he has some children by his proper wife (who are heirs to the money and titles), but he also begets children on other women, his mistresses, who have no claim to his lands or money. In this respect, marriage acts as a resource firewall. Presumably, the economic drain of a mistress is less than the economic strife of too many heirs or too many dowries.