-: We are going to talk about tokens and token limits.
Let's roll.
Let's begin with what a token is.
Large language models, such as ChatGPT,
process text using tokens,
which are common sequences of characters found in a text.
These models learn to understand
relationships between tokens
and then predicting the next one in a sequence.
For example, we have here the sentence,
today is a beautiful sunny day.
That consists of six words,
31 characters, and seven tokens.
You can see with the color shades
how each one of these seven tokens is composed,
and you can take any piece of text you have
and count the amount of tokens in this URL
and see how it might be tokenized.
So what are token limits?
The maximum number of tokens that a model
or API can process,
both input tokens, the prompts,
and the output tokens,
the results are considered to measure it.
For example, GPT-4 Turbo has a context window
of 128,000 tokens,
at least when this class was recorded,
around 500 book pages.
we used the max_tokens parameter in the API
to limit the output tokens.
Why would we do this?
For example, to get conciseness
and readability in our results,
to control the API costs,
to save computational power,
and to ensure the results are below certain length.
Now that we made introductions to tokens,
let's check out some examples on our code.