I made this post a while ago about how there seems to be something I refer to as "language thinking" which is different from visual intelligence:
https://saidit.net/s/nonmorons/comments/9md9/theory_languagethinking_vs_visualthinking/Someone who is a language thinker will be able to say things that are logically consistent but will frequently base them on bad assumptions. ChatGPT operates purely on language, so if my theory is correct that language thinking is something orthogonal to visual thinking then the end point of GPT will be something that says things that are logically consistent but are pure bullshit. Like a salesman. We are already seeing this in ChatGPT. So as ChatGPT develops will it become more salesman-like, or will advanced training of the language models eventually result in a deeper and more visual way kind of artificial intelligence?
The former case will be very good because it will just result in the salesmen/bullshitters being put out of work. But if the latter case happens then all of humanity will be doomed. In humans I don't see a strong correlation between being a good language thinker and truly intelligent, so my bets are on the former.
What is true intelligence if not language processing? If they are not the same thing, and intelligence is something greater than language processing, then intelligence can never be expressed in terms of language, so the question cannot be answered. And if it cannot be expressed then it cannot be programmed. There is a saying that the creation cannot be greater than the creator - maybe this is an example of this.