Short answer
Natural language processing (NLP) is the AI/ML discipline of enabling computers to analyze, understand, and generate human language.
Why it matters
Article
What it is
Natural language processing sits at the intersection of computer science, linguistics, and machine learning. Its goal is to make human language computationally usable: classify documents, extract entities, answer questions, translate between languages, summarize text, and power conversational interfaces. Language is ambiguous, context-dependent, and culturally grounded, so NLP systems must handle uncertainty rather than expect perfectly formal input.
Plain-English explanation
People communicate with words that change meaning based on situation. “Bank” can mean a riverside or a financial institution. NLP systems try to infer useful structure and meaning from text or speech transcripts so software can act—route a support ticket, highlight a contract clause, or draft a reply.
Older NLP pipelines used hand-crafted features and linguistic rules. Modern NLP often relies on embeddings and transformers that learn contextual representations from large corpora, then adapt to tasks with fine-tuning or prompting. Hybrid systems still appear in production when rules provide precision for critical patterns.
How it works
Core stages can include tokenization, representation (bags of words, embeddings), modeling (classifiers, sequence labelers, encoder-decoders, LLMs), and evaluation with task metrics (F1 for extraction, BLEU/COMET for translation, exact match/F1 for QA, plus human judgment). Pretraining on unlabeled text teaches general language structure; downstream objectives specialize behavior.
Speech interfaces add automatic speech recognition and sometimes text-to-speech. Multilingual NLP addresses cross-language transfer and tokenization challenges. Responsible NLP also measures toxicity, privacy leakage, and disparate performance across dialects and demographic language varieties.
Why it matters
Language is how organizations store knowledge and serve customers. NLP automates large-scale reading and writing tasks, improves accessibility, and enables search beyond keywords. Errors can misinform or exclude communities, so evaluation must include real linguistic diversity—not only benchmark averages.
Common misconceptions
- “NLP is solved because chatbots are fluent.” Fluency is not robust understanding. Domain jargon, low-resource languages, and adversarial phrasing still cause failures.
- “Removing stopwords is always required.” Classical tips do not universally apply to modern neural models; preprocessing should match the model and task.
- “Translation metrics alone prove quality.” Automatic scores help compare systems but miss many meaning errors; human evaluation remains important.
Further learning
Stanford CS224N is a standard academic path into modern NLP. Pair it with topics on large language models, generative AI, and glossary terms such as embedding and token.
Encyclopedia connections
This topic sits in the llms branch of the Artificial-Intelligence.info encyclopedia. Use related guides on training vs inference, responsible AI, and hands-on tooling pages to go deeper without treating any single article as complete expertise.
Examples
Named entity recognition in news text
Semantic search over policies
FAQ
Is NLP only about chatbots?
What is the role of embeddings in NLP?
Related glossary terms
- Embedding — A numeric vector representation of an item—such as a token, sentence, image, or document—used to capture similarity and…
- Large language model — A large neural network, typically a transformer, trained on massive text corpora to predict tokens and perform language…
- Multimodal — AI systems that process or generate more than one data type—such as text, images, audio, or video—within a unified…
- Natural language processing — The AI subfield focused on analyzing, understanding, and generating human language for tasks like translation, search…
- RAG (retrieval-augmented generation) — A pattern that retrieves relevant documents and conditions a generative model on that evidence to improve grounded…
- Token — A chunk of text (often a word piece) used as a basic unit for language model training, context limits, and generation.
- Transformer — A neural network architecture that uses attention mechanisms to model relationships among tokens in a sequence…
Sources
-
CS224N: Natural Language Processing with Deep Learning
Stanford UniversityFlagship university course for modern neural NLP.
-
Google AI Blog — language research communications
GoogleEducational posts on language models and NLP applications.