And on the apparently contradictory “AGI” development, I’ll add the Big Tech kingpens like Altman/Zuck/Musk are hyping that while doing the exact opposite. They say that, turn around, cut fundamental research, and focus more on scaling plain transformers up or even making AI ‘products’ like Zuck is. And the models they deploy are extremely conservative, architecturally.
Going on public statements from all three, I think their technical knowledge is pretty poor and they may be drinking some of their own kool-aid. They might believe scaling up transformers infinitely will somehow birth more generalist ‘in the sky’ intelligence, when in reality it’s fundamentally a tool like you describe, no matter how big it is.
This is spot on.
And on the apparently contradictory “AGI” development, I’ll add the Big Tech kingpens like Altman/Zuck/Musk are hyping that while doing the exact opposite. They say that, turn around, cut fundamental research, and focus more on scaling plain transformers up or even making AI ‘products’ like Zuck is. And the models they deploy are extremely conservative, architecturally.
Going on public statements from all three, I think their technical knowledge is pretty poor and they may be drinking some of their own kool-aid. They might believe scaling up transformers infinitely will somehow birth more generalist ‘in the sky’ intelligence, when in reality it’s fundamentally a tool like you describe, no matter how big it is.