Your corpus is the internet. Words on the internet are for the most part not randomly placed next to each other. The neural network created by this has implicitly created reasoning model.
Much like saying an ant hive exhibits intelligence.
What is intelligence? The ability to acquire and apply knowledge and skills. It's all relative. Not as intelligent as a human but more intelligent than a plant.
Reasoning, an easier thing to prove, we can literally go ask bing chat to determine something and it will follow a logical thought process to answer your question (this is reasoning). They've confirmed it was running GPT4.
Humans are very irrational but are still very good at this when they want to be but not always. A limiting factor for GPT4 is probably computing space/power.
I will try another analogy. What if we have a parrot with exceptional memory, which can not only repeat things it heard some time ago, but to continue words it hearing now. I come to the cage and say: "Cogito?" and parrot continue "Ergo sum!". Is parrot intelligent and able to reason, even if I do not know about Descartes?