How can the Universe be infinite if it was all concentrated into a point at the Big Bang?

The Universe was not concentrated into a point at the time of the Big Bang. But the observable Universe was concentrated into a point. The distinction between the whole Universe and the part of it that we can see is important. In the figure below, two views of the Universe are shown: on the left for 1 Gyr after the Big Bang, and on the right the current Universe 13 Gyr after the Big Bang (assuming that the Hubble constant is H_o = 50 km/sec/Mpc and the Universe has the critical density.)

The size of the box in each view is 78 billion light years. The green circle on the the right is the part of the Universe that we can currently see. In the view on the left, this same part of the Universe is shown by the green circle, but now the green circle is a tiny fraction of the 78 billion light year box, and the box is an infinitesimal fraction of the whole Universe. If we go to smaller and smaller times since the Big Bang, the green circle shrinks to a point, but the 78 billion light year box is always full, and it is always an infinitesimal fraction of the infinite Universe.

Note that the black dots represent galaxies, and the galaxies do not expand even though the separation between galaxies grows with time.