A classic example of an infinite regress is when asked the question 'Where did the universe come from?' one answers with 'a god created the universe', because then it must be asked 'who/what then created god?', and then 'who/what created the thing that created god?' ad infinitum.
Theists have been arguing for a long time that their respective gods were 'uncaused' or were 'without cause'. They use this as an attempt to avoid the infinite regress. Thomas Aquinas is one of the most prominent figures in antiquity that argued for this position, he put forward the 'Argument from Contingency', which suggests that something must exist to explain the existence of the universe. It basically comes straight back to the 'Prime Mover' or 'Uncaused Cause' of Aristotle. This argument fails before it even reaches the starting blocks, because it is not based on any evidence at all. This entire argument is just speculation about beings which have never had any evidence to even mildly verify their mere existence. How can I take an argument seriously if it seems only to be rooted in the wild imaginations of Theists?
Our knowledge about the origins of the universe is growing all the time, and with every new discovery we make, we are pointed towards the idea that nothing created the universe. But even if we were no closer to solving the problem today than we were 20 years ago, there would still be no reason to make the god assumption. The god hypothesis is stale, no evidence for the Prime Mover or Uncaused Cause has been discovered since the first incarnations of the argument, which date all the way back to Aristotle. Several millennia should have been ample time. It's time to let go of our antiquated fantasies.
"In many cultures it is customary to answer that God created the universe out of nothing. But this is mere temporizing. If we wish courageously to pursue the question, we must, of course ask next where God comes from? And if we decide this to be unanswerable, why not save a step and conclude that the universe has always existed?"
-Carl Sagan, Cosmos, page 257
See Carl Sagan himself say it in this video.
Here is a lecture given by Lawrence Krauss called 'A Universe from Nothing', which he explains some mechanisms by which the universe itself could be uncaused. Unlike the theistic speculations, these are actually based on observation, and testable science.