LLMs are notoriously bad at following "Write exactly 100 words." How would you design a workflow to strictly enforce a character or word limit?

Question

Accepted Answer

You cannot trust an LLM to count. To enforce limits, you should: (A) Use a "summarization" loop that truncates if the length is exceeded, or (B) Use Logit Bias to heavily penalize the model if it tries to generate tokens beyond a certain point (though this is difficult to implement).

LLMs are notoriously bad at following "Write exactly 100 words." How would you design a workflow to strictly enforce a character or word limit?

Practice Your Response

Similar Questions in Generative AI & LLMs