estimateTokens method

int estimateTokens(
  1. String text
)

Estimates the number of tokens in the given text.

This is an approximation — different providers tokenize differently. Default implementation uses a rough ~kApproxCharsPerToken chars per token estimate.

Implementation

int estimateTokens(String text) {
  return (text.length / kApproxCharsPerToken).ceil();
}