benni@lemmy.world to Lemmy Shitpost@lemmy.world · 2 个月前In heatlemmy.worldimagemessage-square136fedilinkarrow-up11.24K
arrow-up11.24KimageIn heatlemmy.worldbenni@lemmy.world to Lemmy Shitpost@lemmy.world · 2 个月前message-square136fedilink
minus-square_stranger_@lemmy.worldlinkfedilinkarrow-up4·2 个月前It’s not even words, it “thinks” in “word parts” called tokens.
minus-squareSwedneck@discuss.tchncs.delinkfedilinkarrow-up1·27 天前and those tokens? just numbers, indexes. LLMs have no concept of language or words or anything, it’s literally just a statistical calculator where the numbers encode some combination of letter(s)
It’s not even words, it “thinks” in “word parts” called tokens.
and those tokens? just numbers, indexes. LLMs have no concept of language or words or anything, it’s literally just a statistical calculator where the numbers encode some combination of letter(s)