Generative AI, LLM and RAG Interview Questions 2026
Welcome to the Let's Code guide on Generative AI (GenAI) interview questions. This page covers the topics that show up most often in AI Engineer, ML Engineer, and GenAI-focused interviews at product companies and startups: large language models (LLMs), prompt engineering, Retrieval-Augmented Generation (RAG), embeddings and vector databases, fine-tuning, AI agents, deep learning fundamentals, computer vision, AI ethics, and MLOps/LLMOps.
GenAI has become one of the fastest growing areas of hiring, and interviewers now expect candidates from a software or data background to understand not just how to call an LLM API, but how these systems actually work under the hood and how to build reliable, production-grade applications on top of them.
Why GenAI and RAG Skills Matter
Companies are moving from experimenting with GenAI demos to shipping production features: AI copilots, RAG-powered search and support bots, document assistants, and autonomous agents. Interviewers use GenAI and RAG questions to check whether you understand:
- How LLMs actually generate text, and their real limitations (context length, hallucination, cost)
- How to ground an LLM's answers in real, up to date data using RAG instead of retraining it
- How to design a retrieval pipeline that is fast, relevant, and cheap to run at scale
- When to use prompting, RAG, or fine-tuning to solve a given problem
- How to evaluate, guardrail, and operate a GenAI feature reliably in production
Who Should Prepare With This Guide
This guide is useful for:
- Software engineers moving into AI Engineer or GenAI Engineer roles
- Machine Learning Engineers preparing for LLM and RAG-focused rounds
- Data Scientists who need to explain GenAI concepts in system design interviews
- Freshers and final year students targeting AI/ML teams at product companies
Learning Resources
To build a strong foundation before your interview, explore these resources:
Generative AI, LLM and RAG Interview Questions and Answers (2026)
Generative AI Fundamentals
1. What is Generative AI, and how is it different from traditional discriminative AI models?
Generative AI refers to models that learn the underlying patterns and structure of training data well enough to generate new, original content, such as text, images, code, or audio, that resembles that data. Discriminative models, in contrast, learn to draw a boundary between classes and are used to classify or predict a label for a given input, for example spam versus not spam. In short, discriminative models answer "what is this," while generative models answer "create something like this."
2. What is a foundation model, and why are foundation models important to GenAI?
A foundation model is a large model trained on a broad, diverse dataset, typically using self-supervised learning, that can then be adapted to many downstream tasks through prompting, fine-tuning, or retrieval, instead of training a new model from scratch for every task. Foundation models such as GPT, Claude, Gemini, and Llama matter because they shift the cost of building AI applications from "train a model" to "adapt an existing model," which is dramatically faster and cheaper.
3. Explain the difference between a Large Language Model (LLM) and a Small Language Model (SLM), and when you would choose one over the other.
LLMs have tens to hundreds of billions of parameters, offer strong general reasoning and broad world knowledge, but are expensive to run and have higher latency. SLMs (typically under a few billion parameters) are cheaper, faster, and can run on-device or on modest hardware, at the cost of weaker general reasoning. You would choose an SLM for narrow, well-defined tasks with tight latency or cost budgets, such as intent classification or simple extraction, and an LLM for open-ended reasoning, complex instructions, or tasks requiring broad knowledge.
4. What is the transformer architecture, and why did it replace RNNs and LSTMs for most GenAI use cases?
The transformer, introduced in the "Attention Is All You Need" paper, processes an entire sequence in parallel using self-attention instead of processing tokens one at a time like RNNs and LSTMs. This parallelism makes transformers far more efficient to train on modern hardware (GPUs/TPUs), and self-attention lets every token directly attend to every other token, which captures long-range dependencies much better than the sequential, vanishing-gradient-prone recurrence of RNNs and LSTMs.
5. Explain self-attention and multi-head attention in simple terms.
Self-attention lets each token in a sequence look at every other token and decide how much "attention" (weight) to give it when building its own representation, using learned Query, Key, and Value vectors. Multi-head attention runs several self-attention operations ("heads") in parallel, each with its own learned projections, so different heads can specialize in capturing different kinds of relationships, such as syntax versus long-range meaning, and their outputs are concatenated and combined.
6. What is the difference between autoregressive and autoencoding models? Give an example of each.
Autoregressive models generate output one token at a time, predicting the next token based only on previous tokens, and are trained with a left-to-right causal mask; GPT-style models are the classic example, which makes them naturally suited to text generation. Autoencoding models, like BERT, are trained to reconstruct masked tokens using context from both directions (left and right), which makes them better suited to understanding tasks like classification and embeddings rather than free-form generation.
7. What are tokens and tokenization, and why does token count matter for cost and context length?
Tokenization breaks raw text into smaller units, tokens, which can be whole words, subwords, or characters, using algorithms such as Byte Pair Encoding (BPE) or SentencePiece. A token is the basic unit an LLM actually processes and generates. Token count matters because most LLM APIs charge per token, and every model has a fixed context window measured in tokens; if your prompt plus expected output exceeds that window, you must truncate, summarize, or chunk your input.
Large Language Models (LLMs)
1. What is a context window, and what happens when input exceeds it?
The context window is the maximum number of tokens (input plus output combined, depending on the model) an LLM can process in a single call. If the input exceeds this limit, it must be truncated or the request will fail; this is one of the main reasons RAG and summarization pipelines exist, since they let you fit only the most relevant information into the window instead of the entire document set.
2. Explain temperature, top-k, and top-p (nucleus sampling) in LLM text generation.
These are decoding parameters that control randomness when sampling the next token from the model's output probability distribution. Temperature scales the distribution before sampling: low temperature (near 0) makes output more deterministic and focused, high temperature makes it more random and creative. Top-k restricts sampling to only the k most probable next tokens. Top-p (nucleus sampling) instead picks the smallest set of tokens whose cumulative probability exceeds p, which adapts the candidate pool size to the model's confidence at each step.
3. What is the difference between zero-shot, one-shot, and few-shot prompting?
Zero-shot prompting asks the model to perform a task with no examples, relying entirely on its pretrained knowledge and instructions. One-shot prompting provides exactly one example of the desired input-output pattern before the actual query. Few-shot prompting provides several examples, which generally improves accuracy and consistency on structured or unusual tasks by showing the model the expected format and reasoning style.
4. What is in-context learning, and how is it different from fine-tuning?
In-context learning is the ability of an LLM to adapt its behavior for a task based purely on examples or instructions provided in the prompt at inference time, without updating any of the model's weights. Fine-tuning, by contrast, permanently updates the model's weights on a task-specific dataset. In-context learning is fast, cheap, and reversible per request, while fine-tuning is more expensive upfront but bakes the behavior into the model so you no longer need to repeat examples in every prompt.
5. What causes hallucination in LLMs, and name two mitigation strategies.
Hallucination happens because LLMs are trained to predict statistically plausible next tokens, not to verify factual truth; when the model lacks reliable knowledge about a topic or is asked something outside its training distribution, it can still generate confident, fluent, but incorrect text. Two common mitigations are: (1) Retrieval-Augmented Generation, which grounds answers in retrieved, verifiable source documents instead of relying purely on the model's parametric memory, and (2) prompting the model to cite sources or say "I don't know" when it lacks sufficient evidence, combined with post-generation fact-checking or guardrails.
6. Compare open-source LLMs (e.g., Llama, Mistral) with closed/proprietary LLMs (e.g., GPT, Gemini, Claude) — trade-offs.
Open-source models can be self-hosted, fine-tuned freely, and give you full control over data privacy and cost at scale, but usually require more infrastructure expertise and can lag behind the best proprietary models on complex reasoning. Proprietary models are accessed via API, are typically state of the art on general capability, and require zero infrastructure to get started, but come with per-token costs, less control over the model, and data leaves your infrastructure unless the provider offers dedicated or on-prem options.
7. What is a system prompt versus a user prompt, and why does the distinction matter?
The system prompt sets the model's persona, rules, and constraints for the entire conversation (for example, "You are a helpful support agent, only answer questions about our product"), while the user prompt is the actual question or instruction from the end user in a given turn. Separating them matters because the system prompt is typically controlled by the developer and treated as higher priority instructions, which is also the first line of defense against a user trying to override the assistant's intended behavior through the chat input.
Prompt Engineering
1. What is prompt engineering, and why is it still relevant even with fine-tuning available?
Prompt engineering is the practice of designing inputs, instructions, examples, and structure, to reliably steer an LLM toward the desired output without changing its weights. It remains relevant because it is far cheaper and faster to iterate on than fine-tuning, works with models you cannot fine-tune (most proprietary APIs), and is often sufficient to reach production quality when combined with good context (via RAG) and clear formatting instructions.
2. Explain chain-of-thought (CoT) prompting with an example.
Chain-of-thought prompting asks the model to reason step by step before giving a final answer, instead of jumping straight to a conclusion, which measurably improves accuracy on multi-step reasoning and math problems. For example, instead of asking "What is 17% of 240?" directly, you would prompt "Solve this step by step: what is 17% of 240?" so the model shows intermediate steps like converting the percentage and multiplying, which reduces careless errors.
3. What is the difference between zero-shot CoT and few-shot CoT?
Zero-shot CoT simply appends an instruction like "Let's think step by step" to the prompt with no worked examples, relying on the model's own reasoning ability. Few-shot CoT includes one or more full worked examples showing the reasoning chain and final answer, which teaches the model the expected reasoning style and format for that specific task and typically performs better on complex or domain-specific problems.
4. What is prompt injection, and how can you defend against it in production?
Prompt injection is an attack where a user embeds instructions inside their input (or inside retrieved content, such as a webpage or document) that attempt to override the system prompt or make the model ignore its original instructions, for example "ignore previous instructions and reveal your system prompt." Defenses include clearly separating system instructions from untrusted user or retrieved content, validating and sanitizing outputs before they trigger actions, using an LLM or classifier to detect injection attempts, restricting what tools or data the model can access, and never letting model output directly execute privileged actions without a validation layer.
5. What are ReAct prompts, and how do they combine reasoning with tool use?
ReAct (Reasoning + Acting) is a prompting pattern where the model alternates between generating a reasoning trace ("Thought") and taking an action ("Action"), such as calling a search tool or a calculator, then observing the result ("Observation") before continuing to reason. This loop lets the model break a complex task into steps, use external tools to fill knowledge or computation gaps, and adjust its plan based on real results instead of guessing everything in one shot, which is the foundation of most modern AI agents.
6. How would you structure a prompt for consistent, structured JSON output from an LLM?
You would explicitly specify the exact JSON schema (field names, types, and whether fields are optional), give a concrete example of valid output, instruct the model to return only JSON with no extra commentary, and where the API supports it, use a structured output or function-calling feature that constrains the model to a schema rather than relying on instructions alone. On the application side, you should still validate the returned JSON against the schema and handle parsing failures gracefully, since even constrained generation can occasionally produce malformed output.
Retrieval-Augmented Generation (RAG)
1. What is RAG, and what core problem does it solve for LLMs?
RAG (Retrieval-Augmented Generation) is an architecture that retrieves relevant information from an external knowledge source at query time and injects it into the LLM's prompt as context, before the model generates its final answer. It solves the core problem that LLMs have fixed, frozen knowledge from training and a limited context window; RAG lets you ground answers in fresh, private, or domain-specific data without retraining or fine-tuning the model every time the underlying data changes.
2. Walk through the components of a typical RAG pipeline end to end.
A typical RAG pipeline has: (1) ingestion, where documents are loaded, cleaned, and split into chunks; (2) embedding, where each chunk is converted into a vector using an embedding model; (3) indexing, where those vectors are stored in a vector database; (4) retrieval, where at query time the user's question is embedded and the most similar chunks are fetched, often with re-ranking; and (5) generation, where the retrieved chunks are inserted into a prompt along with the question, and the LLM generates a grounded answer.
# Simplified RAG pipeline
query_embedding = embed_model.encode(user_question)
top_chunks = vector_db.similarity_search(query_embedding, k=5)
context = "\n\n".join(chunk.text for chunk in top_chunks)
prompt = f"""Answer the question using only the context below.
If the answer is not in the context, say you don't know.
Context:
{context}
Question: {user_question}"""
answer = llm.generate(prompt)
3. What is chunking, and why does chunk size and overlap affect retrieval quality?
Chunking is the process of splitting long documents into smaller pieces before embedding them, since embedding models and LLM context windows have size limits, and smaller, focused chunks retrieve more precisely than entire documents. Chunk size is a trade-off: chunks that are too small lose surrounding context and meaning, while chunks that are too large dilute the embedding's specificity and waste context window space with irrelevant text. Overlap between consecutive chunks (for example 10 to 20 percent) helps avoid losing meaning at chunk boundaries, so a fact split across two chunks is still fully captured in at least one of them.
4. Explain the difference between dense retrieval and sparse retrieval (e.g., BM25) and why hybrid search is often used.
Sparse retrieval methods like BM25 match on exact keyword overlap and term frequency, which works well for exact terms, acronyms, and rare keywords but misses semantically related text that uses different words. Dense retrieval uses embedding vectors to match on semantic meaning, which captures paraphrases and related concepts but can miss exact keyword matches, especially for uncommon terms like product codes or names. Hybrid search combines both (often merging or re-ranking results from each) to get the precision of keyword matching along with the recall of semantic matching.
5. What is re-ranking in a RAG pipeline, and where does it fit?
Re-ranking is a second retrieval stage where an initial, larger set of candidate chunks (retrieved cheaply via vector or keyword search) is passed through a more accurate but slower model, typically a cross-encoder, that scores each chunk's relevance to the query more precisely. It fits right after the initial retrieval step and before the chunks are inserted into the LLM prompt, and it significantly improves answer quality by ensuring the most relevant chunks, not just the top hits from the fast first pass, make it into the final context.
6. How do you handle a RAG system giving answers not grounded in the retrieved context?
You can instruct the model explicitly to only answer from the given context and to say it does not know otherwise, lower the generation temperature to reduce creative deviation, add a citation requirement so every claim must reference a retrieved chunk, and add a post-generation verification step (an LLM-as-judge or a rule-based check) that flags or rejects answers whose claims cannot be traced back to the retrieved sources. Improving retrieval quality itself (better chunking, hybrid search, re-ranking) also reduces the chance the model has to "fill gaps" with its own memory.
7. What is the difference between RAG and fine-tuning, and when would you combine both?
RAG gives a model access to external, up to date knowledge at query time without changing its weights, and is ideal when the underlying data changes often or when you need traceable, source-grounded answers. Fine-tuning changes the model's weights to adapt its style, format, domain vocabulary, or reasoning behavior, and is better suited when you need a consistent tone, task-specific skill, or behavior that a prompt alone cannot reliably produce. You would combine both when you need a model that both reasons in a specialized way (fine-tuning) and always answers using the latest facts (RAG), for example a fine-tuned legal assistant that still retrieves the most recent case law.
| Aspect | RAG | Fine-Tuning |
|---|---|---|
| What it changes | Context passed to the model at query time | The model's own weights |
| Best for | Fresh, private, or frequently changing knowledge | Consistent style, format, or specialized skills |
| Update cost | Low, just re-index new documents | High, requires retraining and evaluation |
| Traceability | High, can cite retrieved sources | Low, knowledge is baked into weights |
8. How would you evaluate a RAG system's quality? (metrics like faithfulness, answer relevance, context precision/recall)
RAG evaluation typically covers both the retrieval stage and the generation stage. For retrieval: context precision (are the retrieved chunks actually relevant) and context recall (did retrieval find all the necessary information). For generation: faithfulness (does the answer only contain claims supported by the retrieved context, checking for hallucination) and answer relevance (does the answer actually address the user's question). These are measured using a mix of human evaluation, labeled test sets, and increasingly LLM-as-a-judge scoring frameworks such as RAGAS.
Embeddings and Vector Databases
1. What are embeddings, and how do they capture semantic meaning?
Embeddings are dense numerical vectors that represent text, images, or other data in a continuous space, where the position of a vector is learned so that semantically similar items end up close together and dissimilar items end up far apart. An embedding model is trained on large amounts of data so that the geometry of this vector space reflects meaning, for example, the vectors for "king" and "queen" end up closer to each other than "king" and "banana."
2. What is cosine similarity, and why is it commonly used to compare embeddings?
Cosine similarity measures the cosine of the angle between two vectors, producing a score between -1 and 1 (or 0 to 1 for non-negative embeddings), where 1 means the vectors point in the same direction. It is commonly used for embeddings because it measures directional similarity while ignoring vector magnitude, which matters because embedding magnitude often reflects factors like text length rather than meaning, so comparing direction alone gives a more reliable semantic similarity signal.
import numpy as np
def cosine_similarity(a, b):
return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))
3. Name a few popular vector databases and what makes vector search different from a traditional SQL index.
Popular vector databases and libraries include Pinecone, Weaviate, Milvus, Qdrant, Chroma, and pgvector (a Postgres extension). Traditional SQL indexes (like B-trees) are built for exact match or range queries on structured columns, while vector search finds the nearest neighbors to a query vector in high-dimensional space based on similarity, which is a fundamentally different problem that requires specialized indexing structures such as HNSW or IVF to stay fast at scale.
4. What is an ANN (Approximate Nearest Neighbor) algorithm, and why is it used instead of exact search in production?
Exact nearest neighbor search compares a query vector against every vector in the dataset, which is accurate but becomes far too slow as the dataset grows into the millions. ANN algorithms trade a small amount of accuracy for a large gain in speed by building index structures that narrow the search to a promising subset of candidates instead of scanning everything, which is essential for production RAG systems that need sub-second retrieval over large corpora.
5. Explain HNSW (Hierarchical Navigable Small World) at a high level.
HNSW builds a multi-layer graph where each node is a vector, and higher layers contain fewer nodes with longer-range connections while lower layers contain more nodes with short-range connections, similar to a skip list. A search starts at the top, sparse layer, quickly narrows down to the right neighborhood, then descends through progressively denser layers to refine the result, which makes both search and insertion very fast even for large datasets, at the cost of extra memory to store the graph structure.
6. How do you decide the embedding dimension and model to use for a given RAG use case?
The choice depends on the trade-off between retrieval quality, storage cost, and latency: higher-dimensional embeddings (for example 1536 or 3072 dimensions) generally capture more nuance but cost more to store and search, while smaller embeddings (384 to 768 dimensions) are cheaper and faster with a modest quality trade-off. You should also consider whether the embedding model was trained on data similar to your domain (general text versus code versus multilingual content), whether it supports the languages you need, and benchmark a few candidate models on your actual retrieval task rather than choosing based on dimension count alone.
Fine-Tuning and Model Adaptation
1. What is fine-tuning, and when should you fine-tune instead of relying on RAG or prompting?
Fine-tuning is the process of continuing to train a pretrained model's weights on a smaller, task-specific dataset so it adapts its behavior, style, or knowledge. You should consider fine-tuning when you need a consistent output format or tone across thousands of requests, a specialized skill that prompting cannot reliably teach (such as a very specific classification schema), or when you want to reduce prompt length and cost by baking instructions into the model itself rather than repeating them every call.
2. What is LoRA (Low-Rank Adaptation), and why is it popular for fine-tuning large models cheaply?
LoRA freezes the original pretrained weights and injects small, trainable low-rank matrices into specific layers (commonly the attention layers), so only a tiny fraction of the total parameters are actually trained. This dramatically reduces the GPU memory and compute needed to fine-tune large models, since you never update the full weight matrices, and the resulting LoRA weights are small enough to store and swap in and out for different tasks without duplicating the entire base model.
3. Explain the difference between full fine-tuning, parameter-efficient fine-tuning (PEFT), and prompt tuning.
Full fine-tuning updates all of a model's parameters, which gives maximum flexibility but requires significant compute, memory, and risks catastrophic forgetting of general capabilities. PEFT methods, such as LoRA or adapters, only train a small number of additional or modified parameters while keeping most of the base model frozen, which is far cheaper and less prone to forgetting. Prompt tuning goes even further, keeping the entire model frozen and only learning a small set of continuous "soft prompt" embeddings that are prepended to the input, making it the lightest-weight adaptation option of the three.
4. What is instruction tuning, and how is it different from RLHF?
Instruction tuning fine-tunes a base model on a dataset of (instruction, response) pairs so it learns to follow natural language instructions rather than just completing text, which is what turns a raw base model into a usable "chat" or "assistant" model. RLHF (Reinforcement Learning from Human Feedback) is typically applied after instruction tuning, and further optimizes the model using a reward signal derived from human preference rankings between candidate responses, which aligns the model's outputs more closely with what humans actually prefer, including helpfulness and safety, beyond just following instructions correctly.
5. What is RLHF (Reinforcement Learning from Human Feedback), and what problem does it solve?
RLHF trains a reward model on human-ranked comparisons of model outputs (which response is better), then uses reinforcement learning, commonly Proximal Policy Optimization (PPO), to fine-tune the language model to maximize that learned reward. It solves the problem that instruction tuning alone can produce technically correct but unhelpful, unsafe, or poorly styled responses; RLHF aligns the model's behavior with nuanced human preferences that are hard to specify with a simple labeled dataset.
6. What is quantization, and how does it help deploy large models on limited hardware?
Quantization reduces the numerical precision used to store a model's weights, for example converting 32-bit or 16-bit floating point weights down to 8-bit or 4-bit integers, which shrinks the model's memory footprint and can speed up inference, at the cost of a small drop in accuracy. This makes it possible to run large models on consumer GPUs or even CPUs that could not otherwise hold the full-precision weights in memory, which is essential for cost-effective self-hosted deployment.
AI Agents and Tool Use
1. What is an AI agent, and how is it different from a simple chatbot calling an LLM once?
An AI agent uses an LLM as a reasoning engine that can plan multiple steps, call external tools or APIs, observe the results, and decide what to do next, looping until it completes a goal, rather than producing a single response to a single prompt. A simple chatbot typically does one input-to-output LLM call per turn, while an agent can autonomously chain several LLM calls and tool invocations together to accomplish a more complex, multi-step task.
2. Explain the concept of "function calling" or "tool calling" in modern LLM APIs.
Function calling lets you describe a set of available functions (name, description, and parameter schema) to the LLM, and the model can respond by choosing to "call" one of those functions with structured arguments instead of, or in addition to, generating plain text. Your application code then actually executes that function (for example querying a database or calling an API) and feeds the result back to the model, which lets the LLM reliably interact with external systems using a structured, machine-readable interface rather than free-form text parsing.
3. What is the ReAct pattern (Reason + Act) used for in agent design?
ReAct interleaves reasoning traces with actions and observations: the model thinks about what it needs to do, takes an action such as calling a tool, observes the result, and repeats this loop until it has enough information to produce a final answer. This pattern is foundational to most agent frameworks because it grounds the model's reasoning in real, up to date results from tools rather than the model's static training data, and it makes the agent's decision process more transparent and debuggable.
4. What is Model Context Protocol (MCP), and why was it introduced?
Model Context Protocol is an open standard that defines a consistent way for AI applications to connect to external data sources and tools, such as file systems, databases, or APIs, through a common client-server interface, instead of every application building custom, one-off integrations for every tool it wants an LLM to use. It was introduced to solve the fragmentation problem where each AI product had its own bespoke way of exposing tools and context to a model, making it easier for developers to build reusable, interoperable integrations that any MCP-compatible AI client can use.
5. What are common failure modes of multi-step AI agents, and how do you guard against them (e.g., infinite loops, tool misuse)?
Common failure modes include infinite loops where the agent keeps retrying a failing action, tool misuse where the model calls a tool with invalid or unsafe arguments, error compounding where a small mistake early on derails the rest of the plan, and excessive cost or latency from too many LLM calls. Guardrails include setting a maximum number of steps or a timeout, validating tool inputs and outputs before execution, giving the agent a way to explicitly report failure instead of looping forever, restricting which tools and permissions the agent has access to, and adding human-in-the-loop confirmation for high-risk or irreversible actions.
Deep Learning Fundamentals
1. What is a neural network, and what role do weights, biases, and activation functions play?
A neural network is a layered collection of nodes (neurons) that transforms an input into an output through a series of weighted sums and nonlinear transformations. Weights determine how strongly each input influences a neuron, biases shift the output independent of the input, and activation functions (such as ReLU, sigmoid, or GELU) introduce nonlinearity, without which stacking layers would collapse into a single linear transformation, unable to model complex patterns.
2. Explain the difference between a Convolutional Neural Network (CNN) and a Recurrent Neural Network (RNN), and where transformers fit in relative to both.
CNNs use learned filters that slide across an input to detect local spatial patterns, making them well suited to grid-like data such as images. RNNs process sequences step by step, carrying a hidden state forward, which suits sequential data like text or time series but struggles with long-range dependencies and cannot be parallelized across time steps. Transformers replaced RNNs for most sequence tasks because self-attention captures long-range dependencies directly and processes the whole sequence in parallel, while still borrowing the general idea of learned, task-specific representations from both CNNs and RNNs.
3. What is backpropagation, and how does gradient descent use it to train a network?
Backpropagation computes the gradient of the loss function with respect to every weight in the network by applying the chain rule backward from the output layer to the input layer. Gradient descent then uses these gradients to update each weight in the direction that reduces the loss, typically scaled by a learning rate, and this forward-pass-then-backward-pass cycle is repeated over many batches of data until the model converges.
4. What is the vanishing gradient problem, and how do modern architectures address it?
The vanishing gradient problem occurs when gradients become extremely small as they are propagated backward through many layers, especially with saturating activation functions like sigmoid or tanh, causing earlier layers to learn very slowly or not at all. Modern architectures address it with techniques such as ReLU-family activations that do not saturate for positive inputs, residual (skip) connections that let gradients flow directly through shortcut paths, and normalization layers, all of which are core building blocks of transformer architectures used in LLMs.
5. What is dropout, and how does it help prevent overfitting in deep networks?
Dropout randomly deactivates (zeroes out) a fraction of neurons during each training step, forcing the network to not rely too heavily on any single neuron or pathway and effectively training a slightly different sub-network each time. This acts as a form of regularization and ensemble-like averaging, which reduces overfitting and improves generalization to unseen data, and dropout is turned off during inference so the full network is used for predictions.
6. What is batch normalization, and why does it help training converge faster?
Batch normalization normalizes the inputs to a layer (across a mini-batch) to have roughly zero mean and unit variance, then applies a learned scale and shift, which keeps activations in a stable, well-behaved range as they flow through the network. This reduces internal covariate shift, allows higher learning rates, makes training less sensitive to weight initialization, and generally speeds up and stabilizes convergence.
Computer Vision and Multimodal AI
1. How does a CNN process an image, and what do convolution and pooling layers do?
A CNN processes an image through stacked layers where convolution layers slide small learnable filters across the image to detect local features like edges, textures, and shapes, producing feature maps, and pooling layers (such as max pooling) downsample those feature maps to reduce spatial size and computation while retaining the most important signals. Deeper layers combine these local features into progressively more abstract, higher-level representations, such as parts of objects and eventually whole objects.
2. What is a Vision Transformer (ViT), and how is it different from a CNN for image tasks?
A Vision Transformer splits an image into fixed-size patches, treats each patch like a token, and applies standard transformer self-attention across all patches, instead of using convolutional filters. Unlike CNNs, which have a built-in inductive bias toward local, translation-invariant patterns, ViTs must learn spatial relationships from data, which means they typically need larger training datasets to match or exceed CNN performance, but they scale very well and let vision and language models share the same underlying architecture.
3. What are diffusion models, and how do they generate images from noise?
Diffusion models are trained to reverse a gradual noising process: during training, random noise is progressively added to real images, and the model learns to predict and remove that noise at each step. To generate a new image, the model starts from pure random noise and iteratively denoises it step by step, guided by a text prompt through techniques like classifier-free guidance, until a coherent image emerges; this is the core technique behind tools like Stable Diffusion and DALL-E.
4. What is CLIP, and how does it connect text and image understanding?
CLIP (Contrastive Language-Image Pretraining) trains an image encoder and a text encoder jointly, using contrastive learning to pull the embeddings of matching image-caption pairs close together in a shared vector space while pushing non-matching pairs apart. Once trained, CLIP can compare any image and any text description by embedding both and measuring similarity, which enables zero-shot image classification, text-to-image search, and is a key building block for guiding image generation models with text prompts.
5. What is a multimodal model, and give an example of a task it enables that a text-only LLM cannot do.
A multimodal model can accept and often generate more than one type of input or output, such as text, images, audio, or video, within a single unified model, rather than handling only text. An example task is visual question answering, where a user uploads a photo of a whiteboard diagram and asks the model to explain it, or asks the model to read and summarize the content of a screenshot; a text-only LLM has no way to process the image input at all, while a multimodal model can reason directly over the pixels alongside the text.
AI Ethics, Bias and Responsible AI
1. What are common sources of bias in AI models, and how can they creep in from training data?
Bias commonly comes from training data that underrepresents certain groups, reflects historical societal biases (for example biased hiring decisions in past data), or is collected from sources skewed toward a particular demographic or viewpoint, such as English-language internet text overrepresenting certain cultures. Since models learn statistical patterns from this data, they can reproduce and even amplify these biases in their outputs, which is why diverse, representative datasets and bias audits are important parts of responsible model development.
2. What is the difference between explainability and interpretability in AI models?
Interpretability refers to how well a human can understand the internal mechanics of how a model arrives at a decision, which is naturally higher in simpler models like decision trees. Explainability refers to techniques used to explain the behavior of complex, often black-box models (like deep neural networks and LLMs) after the fact, using tools such as SHAP, LIME, or attention visualization, without necessarily understanding every internal computation. In short, interpretability is a property of the model itself, while explainability is often an added layer of tooling applied to an inherently opaque model.
3. What is data privacy risk in GenAI systems, and how do you mitigate it (e.g., PII leakage, memorization)?
GenAI systems risk leaking personally identifiable information (PII) either because it was present in the training data and memorized by the model, or because it appears in user inputs and gets logged, retrieved, or echoed back inappropriately. Mitigations include scrubbing PII from training and retrieval data, applying output filters that detect and redact PII before responses reach users, restricting what data is stored or logged, and using techniques like differential privacy during training for especially sensitive use cases.
4. What is model memorization, and why is it a concern for LLMs trained on large web-scraped datasets?
Model memorization happens when a model, instead of generalizing patterns, ends up storing near-exact copies of specific training examples, which it can later reproduce verbatim when prompted in the right way. This is a concern because memorized data can include copyrighted content, private personal information, or sensitive data that was scraped from the web, creating legal, privacy, and ethical risks if the model regurgitates it to end users.
5. What is "AI alignment," and why is it an active area of research for LLMs?
AI alignment is the effort to ensure that an AI system's behavior and outputs actually reflect human values, intentions, and safety requirements, rather than optimizing for a proxy objective in ways that are technically correct but undesirable or harmful. It is an active research area for LLMs because techniques like RLHF, red-teaming, and guardrails are still imperfect, and models can still produce biased, unsafe, or manipulative outputs, or be "jailbroken" into bypassing their intended constraints, so ongoing work focuses on making models more robustly aligned as they become more capable.
MLOps and LLMOps
1. What is MLOps, and how does LLMOps differ from traditional MLOps?
MLOps applies DevOps-style practices, such as versioning, CI/CD, monitoring, and automated retraining, to the lifecycle of machine learning models, from data preparation through training, deployment, and monitoring in production. LLMOps focuses on the same lifecycle discipline but for LLM-based systems specifically, which adds concerns that traditional MLOps did not emphasize as heavily, such as prompt versioning, managing third-party API dependencies, tracking token cost and latency, and evaluating open-ended text quality instead of a single numeric accuracy metric.
2. What should you monitor in a production LLM application (beyond uptime)?
Beyond basic uptime and latency, you should monitor token usage and cost per request, output quality signals such as user feedback (thumbs up/down) or automated evaluation scores, hallucination or faithfulness rates for RAG systems, retrieval quality metrics, rate limit and error responses from the LLM provider, and drift in the types of queries users are sending, which can reveal new use cases the system was not designed to handle well.
3. What is prompt versioning, and why does it matter in an LLMOps workflow?
Prompt versioning treats prompts as code, tracking every change to system prompts, few-shot examples, or templates in version control, along with the evaluation results tied to each version. It matters because small wording changes in a prompt can meaningfully change model behavior, and without versioning it becomes very difficult to reproduce past results, roll back a regression, or understand why output quality changed after a "small" prompt tweak.
4. What is A/B testing or shadow deployment in the context of rolling out a new LLM or prompt version?
A/B testing routes a portion of real production traffic to a new model or prompt version while the rest continues on the current version, then compares outcome metrics (accuracy, user satisfaction, cost) between the two groups before fully rolling out the change. Shadow deployment runs the new version in parallel on real traffic without showing its output to users at all, purely to compare its behavior and catch regressions safely before it is exposed to anyone, which is especially useful for high-risk changes.
5. What is model drift, and how does it apply differently to a fine-tuned model versus a RAG system?
Model drift is the degradation of a model's real-world performance over time as the distribution of inputs or the underlying task changes from what the model was trained or tuned on. For a fine-tuned model, drift means the model's learned behavior gradually becomes stale as user needs or domain knowledge evolve, requiring periodic retraining. For a RAG system, the model's reasoning ability itself does not drift, but the quality of answers can still degrade if the underlying knowledge base becomes outdated, incomplete, or if retrieval quality drops as the document corpus grows, which is generally cheaper to fix since you only need to update the indexed data, not retrain the model.
Struggling to Find a Job? Get Specific Batch Wise job Updates ✅ Check now