Who are the Gods to which the narrator in "By the Waters of Babylon" refers? Why would he consider them Gods?

The "gods" that John refers to are the inhabitants of the "Place of the Gods."  As the story progresses, the reader learns that the Place of the Gods is actually the former New York City.  Specifically Manhattan.  That would make the gods the former New Yorkers.  I'm quite certain that some current New Yorkers would agree with being referred to as gods.  

John believes that the people must have been gods, because he cannot fathom how else they would have built the enormous buildings that are in the Place of the Gods.  He assumes that the strangely hard roads must have been built by gods, and for sure only gods could have built bridges large enough to span the Hudson.  

In the end though, John has a dream or vision that shows him that the "gods" were normal people like himself with great knowledge and technology.