You can make fun all you want but Polari has a point
It's always the poor underclass with 5 kids, while the educated upper class has 2 tops.
Mandark brought up the Flynn Effect, which is apparently about how average intelligence worldwide is always increasing.
But you also have to consider that in first-world and in developing nations, you're starting to have to learn more before you can get a desirable job. Gradually, things like the level of schooling you've completed and especially practical skills like math and literacy will universally rise as jobs that are mostly labor-intensive will start being automated.
I'm not including people who can't stand to ever work/use birth control and just don't give a damn about providing for their kids but that's probably a negligible minority. The movie Polari is talking about assumes they'll eventually outbreed everyone else.