Question 1

What is chain-of-thought prompting and when does it help?

Accepted Answer

Chain-of-thought (CoT) prompting instructs the model to write out intermediate reasoning steps before producing a final answer, which improves accuracy on multi-step arithmetic, logic puzzles, and compositional questions. It is most impactful on models with at least ~10B parameters and on tasks where the answer space is large enough that guessing is hard.

Question 2

What is a context window in an LLM and why does its size matter?

Accepted Answer

The context window is the maximum number of tokens an LLM can attend to in a single forward pass — both the input prompt and the model's own generated output count toward this limit. Its size determines how much prior text influences each prediction, sets a hard ceiling on document length and conversation history, and drives memory and compute costs that scale quadratically with sequence length under standard attention.

Question 3

Why is cosine similarity preferred over Euclidean distance for comparing text vectors?

Accepted Answer

Cosine similarity measures the angle between two vectors, making it invariant to vector magnitude — so a short document and a long document on the same topic score high regardless of length differences. Euclidean distance conflates directional difference with scale difference, which is misleading for sparse or length-varying text.

Question 4

How does an LLM generate text — what is next-token prediction and autoregression?

Accepted Answer

An LLM generates text one token at a time by computing a probability distribution over its entire vocabulary for the next token, sampling from that distribution, appending the result, and repeating — a process called autoregression. Each new token is conditioned on all previously generated tokens, so the output at step N is only as good as the choices made at steps 1 through N-1.

Question 5

What are n-grams and when should you use them in NLP?

Accepted Answer

An n-gram is a contiguous sequence of n tokens from text — bigrams capture two-word phrases, trigrams capture three. They add local word-order context to bag-of-words models, improving tasks like language modelling, spell-checking, and text classification where short phrases are discriminative.

Question 6

What prompt engineering techniques should every LLM practitioner know?

Accepted Answer

The core toolkit is: system prompts (role and constraints), few-shot examples (format and tone anchoring), chain-of-thought (step-by-step reasoning), and output constraints (JSON schema, stop sequences). Combining these predictably closes the gap between a capable base model and a production-ready feature.

Question 7

What is sequence padding and why is it necessary for batch training?

Accepted Answer

Padding adds dummy tokens to shorter sequences so all examples in a batch share the same length, which is required for tensor operations. Attention masks tell the model to ignore padded positions, preventing them from contributing to loss or attention scores.

Question 8

What is the difference between stemming and lemmatization?

Accepted Answer

Stemming strips suffixes using heuristic rules and may produce non-words, while lemmatization uses a vocabulary and morphological analysis to return the canonical dictionary form. Lemmatization is slower but always produces valid words, making it preferable when interpretability matters.

Question 9

What are stop words and when should you remove them?

Accepted Answer

Stop words are high-frequency function words — 'the', 'is', 'at', 'which' — that typically carry little discriminative content for tasks like classification or retrieval. Removing them reduces vocabulary size and noise, but for tasks like sentiment analysis or question answering, some function words can be semantically important and should be kept.

Question 10

How do you reliably get structured outputs (JSON, typed objects) from an LLM?

Accepted Answer

Modern APIs offer constrained decoding — the model's token sampling is restricted to only produce tokens that are valid continuations of a JSON schema. Combined with Pydantic validation in application code, this eliminates the JSON-parsing errors that plagued earlier prompt-only approaches. When constrained decoding is unavailable, few-shot examples plus output parsing with retry is the fallback.

Question 11

What is TF-IDF and how does it improve on raw bag-of-words counts?

Accepted Answer

TF-IDF weights each term by how often it appears in a document (TF) scaled down by how common it is across the whole corpus (IDF), so words that are frequent everywhere — like 'the' — get low scores while distinctive terms get high scores. This makes document vectors more informative than raw counts for retrieval and classification.

Question 12

What are tokens in an LLM and why is API pricing per token rather than per word or character?

Accepted Answer

A token is the smallest unit a language model processes — typically a word, sub-word fragment, or punctuation mark produced by a byte-pair encoding (BPE) or similar algorithm. Pricing is per token because each token requires one forward-pass position in the attention matrix, directly driving compute and memory cost regardless of whether it maps to a full word or a single letter.

Question 13

What do 'parameters' mean in a language model and what do they actually store?

Accepted Answer

Parameters are the learnable floating-point numbers — weights and biases — that define a neural network's behaviour. In a transformer LLM, they are distributed across token embedding matrices, multi-head attention projection matrices (Q, K, V, O), and feed-forward network layers. They encode compressed statistical associations between tokens learned during training, not explicit facts or rules.

Question 14

What is a vector database and how does it enable semantic retrieval?

Accepted Answer

A vector database stores dense numerical embeddings alongside their source documents and uses approximate nearest-neighbor (ANN) algorithms to find the most semantically similar entries for a query vector in milliseconds. Unlike a keyword index, similarity is measured in geometric space so synonyms and paraphrases match naturally. Common choices include Pinecone, Weaviate, Qdrant, and pgvector for Postgres.

Question 15

What is Retrieval-Augmented Generation (RAG) and why is it used?

Accepted Answer

RAG couples a retrieval step — fetching relevant documents from an external store — with a generative model so the LLM can answer questions about knowledge it was never trained on. It solves the stale-knowledge and hallucination problems without retraining. The pattern is preferred when the knowledge base changes frequently or contains proprietary data.

Question 16

What is tokenization in NLP and why does it matter?

Accepted Answer

Tokenization splits raw text into discrete units — words, subwords, or characters — that a model can process numerically. The strategy chosen controls vocabulary size, out-of-vocabulary rate, and how well the model handles rare or morphologically complex words.

Question 17

Why do dense word embeddings outperform one-hot vectors?

Accepted Answer

One-hot vectors are high-dimensional, sparse, and treat all words as equidistant — they carry zero semantic information. Dense embeddings place similar words close together in a low-dimensional space, enabling models to generalize from seen words to unseen but related ones.

Question 18

What is the difference between encoder models like BERT and decoder models like GPT?

Accepted Answer

BERT is an encoder-only transformer that reads all tokens bidirectionally and is trained with masked language modelling — ideal for tasks requiring a rich contextual representation of an entire sequence, like classification or NER. GPT is a decoder-only transformer that attends only to previous tokens via a causal mask and is trained with next-token prediction — ideal for text generation. Encoder-decoder models like T5 combine both for tasks that map one sequence to another.

Question 19

How does Byte-Pair Encoding (BPE) tokenization work?

Accepted Answer

BPE starts with a character-level vocabulary and iteratively merges the most frequent adjacent pair of symbols until a target vocabulary size is reached. The resulting subword units handle rare and unseen words gracefully without any out-of-vocabulary tokens.

Question 20

What chunking strategies exist for RAG and how do you choose between them?

Accepted Answer

Chunking splits source documents into retrievable units before embedding. The right strategy depends on document structure, query style, and the model's context window. Fixed-size chunks are simple but break mid-sentence; semantic or structural chunking preserves coherence; hierarchical chunking enables parent-document retrieval for richer context.

Question 21

How do you evaluate the quality of an LLM or RAG system?

Accepted Answer

Evaluation splits into retrieval quality (did we fetch the right chunks?) and generation quality (did the model use them correctly?). Key metrics are context precision/recall for retrieval and faithfulness plus answer relevance for generation. Frameworks like RAGAS automate LLM-as-judge scoring; human annotation anchors the ground truth.

Question 22

When should you use prompt engineering versus fine-tuning to adapt an LLM?

Accepted Answer

Prompt engineering is the right starting point when the task can be described in natural language, the required knowledge already exists in the base model, and iteration speed matters — no training required. Fine-tuning is warranted when you need consistent output format at scale, domain-specific style that prompts cannot reliably impose, or when latency and token costs from long system prompts are prohibitive.

Question 23

How does GloVe differ from Word2Vec in learning word embeddings?

Accepted Answer

GloVe (Global Vectors) builds a global co-occurrence matrix over the entire corpus and then factorizes it, directly encoding how often pairs of words co-occur. Word2Vec uses local context windows and a prediction objective, never explicitly seeing the global statistics. GloVe tends to capture linear substructures slightly better while Word2Vec handles rare words better with negative sampling.

Question 24

What are out-of-vocabulary (OOV) words and how do modern NLP systems handle them?

Accepted Answer

OOV words are tokens unseen during vocabulary construction that a model cannot look up in its embedding table. Classical word-level models replace them with a generic UNK token, losing all information, while subword tokenizers (BPE, WordPiece) eliminate OOV entirely by decomposing any word into known subunits.

Question 25

What is the difference between pretraining, fine-tuning, instruction-tuning, and RLHF?

Accepted Answer

Pretraining teaches a model general language structure by predicting tokens across a massive corpus; fine-tuning adapts the pretrained weights to a narrower task or domain using supervised data; instruction-tuning is supervised fine-tuning specifically on (instruction, response) pairs so the model follows directives; RLHF further aligns the model to human preferences by training a reward model on ranked responses and using it as a signal for policy optimisation with PPO or a similar algorithm.

Question 26

What is prompt injection and how do you defend against it?

Accepted Answer

Prompt injection is an attack in which malicious text in retrieved documents or user input overrides the application's system instructions, redirecting the model to perform unintended actions. Defenses layer input/output validation, privilege separation, and tool-call confirmation — no single fix is sufficient.

Question 27

What techniques reduce LLM cost and latency in production?

Accepted Answer

Cost scales with input plus output tokens; latency scales with output tokens and model size. The highest-leverage levers are: model routing (use a small model when the task is simple), prompt caching (reuse expensive prefix computation), output length control, and batching. Together these can cut spend 60–90% without quality regression.

Question 28

What is hybrid search and when should you use semantic vs keyword retrieval?

Accepted Answer

Keyword search (BM25) excels at exact term matching — product codes, proper nouns, rare abbreviations. Semantic search (dense embeddings) captures meaning and handles paraphrases. Hybrid search runs both in parallel and merges scores with Reciprocal Rank Fusion, giving the best of both worlds for most production RAG systems.

Question 29

What is the difference between temperature, top-k, and top-p sampling in LLMs?

Accepted Answer

Temperature rescales the logits before softmax — lower values sharpen the distribution toward the most likely token, higher values flatten it. Top-k restricts sampling to the k highest-probability tokens; top-p (nucleus sampling) restricts it to the smallest set of tokens whose cumulative probability reaches p. In practice top-p adapts the candidate pool dynamically while top-k uses a fixed count.

Question 30

How do function/tool calling and LLM agents work at a high level?

Accepted Answer

Tool calling extends the LLM's output space to include structured function invocations. The model emits a JSON object naming a tool and its arguments; the runtime executes the tool and feeds the result back as a new message. An agent is a loop that repeats this cycle — observe, think, act — until the task is complete or a stopping condition is met.

Question 31

What causes LLM hallucinations and how can they be reduced?

Accepted Answer

Hallucinations occur because an LLM is trained to produce plausible next tokens, not verified facts — it has no internal truth-checking mechanism, only statistical patterns. Common causes include rare or conflicting training data, overconfident decoding, and prompts that lead the model to extrapolate beyond what it learned. Mitigation strategies include retrieval-augmented generation, grounding responses to retrieved sources, lowering temperature, and calibrated refusal training.

Question 32

How does Word2Vec work, and what is the difference between Skip-gram and CBOW?

Accepted Answer

Word2Vec trains a shallow neural network to predict context from a target word (Skip-gram) or a target word from its context (CBOW), learning dense vector representations as a by-product. Skip-gram works better for rare words; CBOW is faster and suits large corpora.

Question 33

Why are smaller language models (SLMs) sometimes preferable to larger ones?

Accepted Answer

Smaller models win on latency, inference cost, on-device deployment, and fine-tuning feasibility. When trained on high-quality, curated data and aligned for a narrow task, a 7B–13B model can match or exceed a general-purpose 70B+ model on that specific workload while using a fraction of the compute budget.

Question 34

What is the key difference between Word2Vec embeddings and BERT's contextual embeddings?

Accepted Answer

Word2Vec assigns a single static vector to each word token regardless of surrounding words, so polysemous words like 'bank' always get the same representation. BERT generates a different vector for each token depending on its full sentential context, allowing it to disambiguate meaning on the fly.

Question 35

What is the KV cache in a transformer and why does it matter for inference?

Accepted Answer

The KV cache stores the key and value tensors computed during previous forward passes so they do not need to be recomputed for every new token during autoregressive generation. Without it, generating each token would require a full forward pass over the entire context from scratch, making inference cost grow quadratically with sequence length rather than linearly.

Question 36

When should you use RAG vs fine-tuning vs a long-context model?

Accepted Answer

RAG is the default for dynamic, proprietary, or frequently updated knowledge. Fine-tuning is correct when you need to change the model's behavior, format, or domain-specific reasoning style — not just its knowledge. Long-context models are appropriate when your entire knowledge base fits in a single context window and latency is acceptable.

Question 37

How does RLHF work and what problem does it solve?

Accepted Answer

RLHF (Reinforcement Learning from Human Feedback) aligns a language model's outputs to human preferences by training a reward model on ranked human comparisons, then using that reward signal to fine-tune the policy with reinforcement learning. It solves the gap between a model that is good at next-token prediction and a model that is genuinely helpful, harmless, and honest.