Glossary Term
Token
Category: technical
In Simple Terms
A small chunk of text that a language model processes.
Think of It Like...
Think of it like Lego pieces that build a sentence.
Real World Example
A sentence might be split into 10 to 15 tokens.
Related Terms
Keep exploring
Ready for more? Try a lesson or read the latest guides.