Token
A token is the basic unit a language model reads and writes. It can be a whole word, part of a word, or even a single punctuation mark. Models do not see raw text; they see a sequence of token IDs produced by a tokenizer. Different models use different tokenizers, so the same sentence may cost a different number of tokens on GPT-4o than on Claude or Gemini. As a rough guide, one token is about four English characters or three quarters of a word. A 500-word article might be 700 to 900 tokens, depending on the tokenizer. Tokens matter for two reasons. First, pricing is per token, so longer prompts and outputs cost more. Second, models have a context window measured in tokens; if your input exceeds that limit, the model cannot process it. Tools like the VePrompts tokenizer show you exactly how a specific model splits your text.
Published 2026-06-12
Related terms
Explore the glossary
Find definitions for AI, LLM, MCP, RAG, agent, and prompt engineering terms.
Browse all termsRelated Resources
Prompt Engineering
GlossaryPrompt engineering is the practice of crafting inputs to a language model so it produces better outputs without changing the model's weights. It covers word choice, structure, examples, constraints, and the order in which information appears. A well-engineered prompt can turn a mediocre response into a precise, actionable one. Effective prompts are usually clear, specific, and formatted. They state the task, define the audience, set the output format, and include any constraints. Adding examples, known as few-shot prompting, helps the model understand patterns that are hard to describe in words. Breaking complex tasks into steps, called chain-of-thought prompting, improves reasoning and arithmetic. Prompt engineering is iterative. You write a prompt, test it on diverse inputs, measure the results, and refine. Tools like the VePrompts Prompt Optimizer can surface issues such as ambiguity, missing constraints, or conflicting instructions. Good prompt engineering is often the fastest way to improve an AI feature before investing in fine-tuning or custom infrastructure.
DeepSeek Coder Architect
PromptLeverage DeepSeek Coder for complex software architecture, code generation, and technical problem-solving with advanced reasoning.
3D Printing Optimizer
SkillOptimize 3D models for additive manufacturing considering orientation, supports, infill, and material properties.
Firecrawl
MCP ServerOfficial Firecrawl MCP Server - Adds powerful web scraping and search to Cursor, Claude and any other LLM clients.
Prompt
GlossaryThe input text given to a language model to elicit a desired response.