#Deep Learning

Last updated    - V7 (09-Jan-2026)

Key Concepts

S.No	Topic	Sub-Topics
1	Deep Learning	What is deep learning, History, DL vs ML, Applications, Challenges
2	Linear Algebra Refresher	Vectors, Matrices, Dot product, Eigenvalues, Matrix operations
3	Probability & Statistics	Random variables, Probability distributions, Mean & variance, Bayes theorem, Expectation
4	Optimization Basics	Loss functions, Cost functions, Convex vs non-convex, Gradient descent, Learning rate
5	Neural Network Basics	Perceptron, Neurons, Weights & bias, Activation functions, Forward propagation
6	Activation Functions	Sigmoid, Tanh, ReLU, Leaky ReLU, Softmax
7	Backpropagation	Chain rule, Gradient computation, Weight updates, Vanishing gradients, Exploding gradients
8	Training Deep Neural Networks	Epochs, Batch size, Initialization, Convergence, Overfitting
9	Regularization Techniques	L1, L2, Dropout, Early stopping, Data augmentation
10	Optimizers	SGD, Momentum, RMSProp, Adam, AdamW
11	Loss Functions	MSE, MAE, Cross-entropy, Hinge loss, KL divergence
12	Deep Learning Frameworks	TensorFlow basics, Keras API, PyTorch basics, Autograd, Model training loop
13	Convolutional Neural Networks (CNN)	Convolution layers, Pooling, Padding, Stride, Feature maps
14	CNN Architectures	LeNet, AlexNet, VGG, ResNet, Inception
15	Image Classification	Dataset preparation, Transfer learning, Fine-tuning, Evaluation metrics, Deployment basics
16	Sequence Modeling	Time series, Sequential data, Tokenization, Padding, Masking
17	Recurrent Neural Networks (RNN)	RNN architecture, BPTT, Vanishing gradients, Use cases, Limitations
18	LSTM & GRU	Cell states, Gates, LSTM vs GRU, Applications, Training tips
19	Attention Mechanism	Why attention, Self-attention, Encoder-decoder attention, Scaled dot-product, Benefits
20	Transformers	Transformer architecture, Positional encoding, Multi-head attention, Encoder-decoder, Training
21	Natural Language Processing with DL	Word embeddings, RNN for NLP, Transformers for NLP, Text classification, NER
22	Autoencoders	Basic autoencoder, Sparse AE, Denoising AE, Variational AE, Use cases
23	Generative Models	GAN basics, Generator & discriminator, Training instability, DCGAN, Applications
24	Advanced CNN Applications	Object detection, Image segmentation, Face recognition, Medical imaging, OCR
25	Transfer Learning & Fine-tuning	Pretrained models, Layer freezing, Domain adaptation, Benefits, Limitations
26	Model Evaluation	Accuracy, Precision, Recall, F1-score, ROC-AUC
27	Hyperparameter Tuning	Grid search, Random search, Bayesian optimization, Learning rate schedules, Batch tuning
28	Model Optimization	Pruning, Quantization, Knowledge distillation, Mixed precision, Inference optimization
29	Deployment of DL Models	REST APIs, Model serving, Cloud deployment, Edge deployment, Monitoring
30	Advanced & Emerging Topics	Self-supervised learning, Multimodal models, Foundation models, Ethical AI, Future trends

Interview question

Basic Level

1. What is Deep Learning?
2. Difference between Machine Learning and Deep Learning.
3. What is a Neural Network?
4. What is a perceptron?
5. What is an activation function?
6. What are weights and biases?
7. What is forward propagation?
8. What is backpropagation?
9. What is a loss function?
10. What is gradient descent?
11. What is a learning rate?
12. What are epochs and batches?
13. What is overfitting?
14. What is underfitting?
15. What is regularization?
16. What is dropout?
17. What is a convolutional neural network (CNN)?
18. What is pooling in CNN?
19. What is a recurrent neural network (RNN)?
20. What is an LSTM?
21. What is a GRU?
22. What is batch normalization?
23. What is data augmentation?
24. What is feature extraction?
25. What is a deep neural network (DNN)?

---

Intermediate Level

1. Explain how backpropagation works.
2. What is the vanishing gradient problem?
3. What is the exploding gradient problem?
4. What is weight initialization?
5. What is Xavier/Glorot initialization?
6. What is He initialization?
7. What are optimizers (SGD, Adam, RMSProp)?
8. What is model capacity?
9. What is early stopping?
10. What is cross-entropy loss?
11. What is mean squared error?
12. What are CNN kernels/filters?
13. What is padding and stride?
14. What is a fully connected layer?
15. What is transfer learning?
16. What are embeddings?
17. What is a sequence-to-sequence (Seq2Seq) model?
18. What is teacher forcing?
19. What is attention mechanism?
20. What is self-attention?
21. What is multi-head attention?
22. What is positional encoding?
23. What is a transformer?
24. What are residual connections?
25. What is a skip connection?

---

Advanced Level

1. Explain the architecture of a CNN end-to-end.
2. Explain the architecture of an RNN end-to-end.
3. Explain the architecture of LSTM and GRU in detail.
4. What is the receptive field in CNN?
5. What is dilated convolution?
6. What is depthwise separable convolution?
7. What is batch vs layer vs group normalization?
8. What is label smoothing?
9. What is attention score calculation?
10. What is cross-attention?
11. What are encoder and decoder blocks in transformers?
12. What is beam search decoding?
13. What is scheduled sampling?
14. What is gradient clipping?
15. What is gradient checkpointing?
16. Explain vanishing gradient mitigation methods.
17. What are VAEs (Variational Autoencoders)?
18. What are GANs (Generative Adversarial Networks)?
19. What is mode collapse in GANs?
20. What is contrastive learning?
21. What is self-supervised learning?
22. What is metric learning?
23. What is Siamese network?
24. What is cosine similarity in embeddings?
25. What is knowledge distillation?

---

Expert Level

1. Explain the transformer architecture at scale.
2. What are Mixture-of-Experts (MoE) models?
3. What is sparse attention?
4. What is FlashAttention?
5. What is rotary positional embedding (RoPE)?
6. What is ALiBi (Attention with Linear Biases)?
7. What is diffusion modeling?
8. Explain the U-Net architecture used in diffusion.
9. What is reinforcement learning in deep learning?
10. What is Deep Q-Learning?
11. What is policy gradient?
12. What is PPO (Proximal Policy Optimization)?
13. What is distributed training?
14. Difference between data parallelism and model parallelism.
15. What is pipeline parallelism?
16. What is tensor parallelism?
17. What is quantization-aware training?
18. What is pruning in neural networks?
19. What is model compression?
20. What is federated learning?
21. What is edge deployment for deep learning models?
22. What are safety and alignment concerns in deep models?
23. What are adversarial attacks?
24. What are adversarial defense techniques?
25. What are current research trends in deep learning?

Related Topics

#Spark SQL

Last updated    - V7 (09-Jan-2026)

Key Concepts

S.No	Topic	Sub-Topics
1	Spark SQL	What is Spark SQL, SQL vs DataFrame API, Use cases, Architecture overview, Components
2	Spark SQL Architecture	Catalyst optimizer, Logical plan, Physical plan, Tungsten engine, Execution flow
3	SparkSession & SQL Entry Points	SparkSession.sql(), SQLContext, HiveContext, Configurations, Best practices
4	Creating Tables & Views	Managed tables, External tables, Temporary views, Global views, CTAS
5	Data Types & Schema	Primitive types, Complex types, Struct, Array, Map
6	Reading Data Sources	CSV, JSON, Parquet, ORC, Avro basics
7	Writing Data using SQL	INSERT INTO, INSERT OVERWRITE, Save modes, Partition writes, Bucketing
8	Basic SELECT Queries	Select columns, Expressions, Aliases, DISTINCT, LIMIT
9	Filtering & WHERE Clause	WHERE conditions, AND/OR, BETWEEN, IN, LIKE
10	Sorting & Ordering	ORDER BY, SORT BY, DISTRIBUTE BY, CLUSTER BY, Performance impact
11	Aggregation Functions	COUNT, SUM, AVG, MIN, MAX
12	GROUP BY & HAVING	Grouping rules, Multiple group keys, HAVING clause, Aggregation filters, Optimization
13	Join Types in Spark SQL	Inner join, Left join, Right join, Full join, Cross join
14	Join Optimization Techniques	Broadcast joins, Shuffle joins, Join hints, Skew handling, AQE
15	Subqueries & CTEs	Scalar subqueries, Correlated subqueries, WITH clause, Nested queries, Optimization
16	Window Functions	OVER clause, PARTITION BY, ORDER BY, Ranking functions, Analytical functions
17	String & Date Functions	String manipulation, Date arithmetic, Timestamp functions, Formatting, Parsing
18	Handling NULLs	IS NULL, IS NOT NULL, COALESCE, NVL, NULLIF
19	Complex Data Types	Arrays, Maps, Structs, explode, lateral view, JSON functions
20	User Defined Functions (UDF)	Creating UDFs, Registering UDFs, Performance impact, When to avoid UDFs, Alternatives
21	Partitioning & Bucketing	Table partitioning, Static vs dynamic partitions, Bucketing concepts, Query pruning, Performance
22	Performance Optimization	Predicate pushdown, Column pruning, Caching tables, AQE, Cost-based optimizer
23	Execution Plans & Debugging	EXPLAIN, Logical vs physical plans, DAG stages, Common bottlenecks, Tuning
24	Integration with Hive	Hive metastore, HiveQL support, External tables, SerDe, Compatibility issues
25	Transactional Tables	ACID tables, Delta Lake basics, MERGE, UPDATE, DELETE, Time travel
26	Structured Streaming with SQL	Streaming tables, Continuous queries, Watermarking, Window aggregations, Triggers
27	Security & Access Control	Table permissions, Column masking, Row-level security, Auditing, Best practices
28	Error Handling & Data Quality	Bad records handling, Schema mismatch, Try-catch patterns, Data validation, Logging
29	Best Practices & SQL Standards	Naming conventions, Query readability, Anti-patterns, Reusability, Testing
30	Real-world Use Cases & Projects	ETL pipelines, Data warehousing, Reporting, Optimization review, End-to-end project

Interview question

Basic Level

1. What is Spark SQL?
2. How is Spark SQL different from traditional SQL?
3. What are the main components of Spark SQL?
4. What is a SparkSession?
5. How do you create a DataFrame using Spark SQL?
6. What is a temporary view?
7. What is a global temporary view?
8. Difference between temporary view and global temporary view?
9. What is a table in Spark SQL?
10. What is a managed table?
11. What is an external table?
12. Difference between managed and external tables?
13. What is a database in Spark SQL?
14. How do you show all databases?
15. How do you use USE database command?
16. What is show tables command?
17. What is select statement in Spark SQL?
18. What is where clause?
19. What is order by clause?
20. What is group by clause?
21. What is having clause?
22. What is limit clause?
23. What is distinct keyword?
24. What is count() function?
25. What is null handling in Spark SQL?

Intermediate Level

26. What is lazy evaluation in Spark SQL?
27. What is Catalyst Optimizer?
28. What are logical plans in Spark SQL?
29. What are physical plans in Spark SQL?
30. What is explain() command?
31. Difference between explain() and explain(true)?
32. What is schema inference in Spark SQL?
33. What is SQLContext?
34. Difference between SQLContext and SparkSession?
35. What are built-in SQL functions?
36. What is case when expression?
37. What is cast() in Spark SQL?
38. What are date and time functions?
39. What is join in Spark SQL?
40. What are different types of joins?
41. What is inner join?
42. What is left outer join?
43. What is right outer join?
44. What is full outer join?
45. What is cross join?
46. What is subquery in Spark SQL?
47. What is correlated subquery?
48. What is union vs union all?
49. What is view vs table?
50. What is window function?

Advanced Level

51. What is broadcast join in Spark SQL?
52. How does Spark decide join strategies?
53. What is shuffle in Spark SQL?
54. What is adaptive query execution (AQE)?
55. What is cost-based optimizer (CBO)?
56. How does Spark SQL handle data skew?
57. What is salting technique in Spark SQL?
58. What is partition pruning?
59. What is dynamic partition pruning?
60. What is bucketing in Spark SQL?
61. Difference between partitioning and bucketing?
62. What is column pruning?
63. What is predicate pushdown?
64. What is vectorized execution?
65. What is whole-stage code generation?
66. What is Tungsten engine?
67. How does Spark SQL manage memory?
68. What is caching in Spark SQL?
69. Difference between cache and persist?
70. What is checkpointing?
71. What is Spark SQL UDF?
72. Difference between UDF and built-in functions?
73. Performance impact of UDFs?
74. What is map-side aggregation?
75. How to optimize aggregation queries?

Expert Level

76. Why Spark SQL is faster than Hive?
77. How does Catalyst apply rule-based optimization?
78. How does Catalyst apply cost-based optimization?
79. Explain Spark SQL query execution lifecycle.
80. What happens internally when a Spark SQL query is executed?
81. How does Spark SQL generate Java bytecode?
82. What is off-heap memory in Spark SQL?
83. How does AQE modify execution plans at runtime?
84. How to debug slow Spark SQL queries?
85. How to analyze Spark UI for SQL queries?
86. What are common Spark SQL performance anti-patterns?
87. Why SELECT * is discouraged?
88. How does Spark SQL handle schema evolution?
89. How does Spark SQL work with semi-structured data?
90. How does Spark SQL handle nested data?
91. What is explode() in Spark SQL?
92. What is from_json() and to_json()?
93. What is Delta Lake integration with Spark SQL?
94. How does Spark SQL support ACID transactions?
95. What is time travel in Spark SQL?
96. What is Z-Ordering?
97. What is OPTIMIZE command?
98. How do you tune Spark SQL configurations?
99. Explain real-world Spark SQL optimization scenario.
100. When should Spark SQL not be used?

Related Topics

#RDD

Last updated    - V7 (09-Jan-2026)

Key Concepts

S.No	Topic	Sub-Topics
1	Apache Spark & RDD	What is Spark, What is RDD, RDD characteristics, Use cases, Spark ecosystem
2	Spark Architecture	Driver, Executors, Cluster manager, Jobs, Stages & Tasks
3	SparkContext & RDD Creation	SparkContext, parallelize(), textFile(), wholeTextFiles(), makeRDD()
4	RDD Types & Characteristics	Parallel collections, Hadoop RDDs, Typed RDDs, Immutable nature, Partitioning
5	RDD Transformations Basics	map, flatMap, filter, distinct, union
6	RDD Actions Basics	collect, count, first, take, reduce
7	RDD Key-Value Pairs	Pair RDDs, mapToPair, reduceByKey, groupByKey, combineByKey
8	Key-Value Aggregations	reduceByKey, foldByKey, aggregateByKey, countByKey, sortByKey
9	RDD Joins	join, leftOuterJoin, rightOuterJoin, fullOuterJoin, Cartesian
10	RDD Set Operations	union, intersection, subtract, distinct, zip
11	Partitioning in RDDs	HashPartitioner, RangePartitioner, Custom partitioner, Repartition, Coalesce
12	Shuffle Operations	Shuffle concept, Wide vs narrow transformations, Optimization, Skew issues, Tuning
13	RDD Persistence & Caching	cache, persist, Storage levels, Memory management, When to cache
14	Fault Tolerance & Lineage	RDD lineage, DAG, Re-computation, Checkpointing, Recovery
15	Shared Variables	Broadcast variables, Accumulators, Use cases, Performance benefits, Limitations
16	RDD with Files	Text files, Sequence files, Object files, Hadoop input formats, Output formats
17	RDD vs DataFrame vs Dataset	API differences, Performance, Type safety, Optimization, When to use RDD
18	Custom Transformations	mapPartitions, mapPartitionsWithIndex, foreachPartition, Efficiency, Use cases
19	RDD Sorting & Ordering	sortBy, sortByKey, takeOrdered, top, Ordering logic
20	Error Handling & Debugging	Common errors, Serialization issues, Logging, Debug techniques, Best practices
21	Performance Optimization	Partition sizing, Avoiding shuffles, Serialization tuning, Memory tuning, Best practices
22	Handling Large Datasets	Skew handling, Sampling, Checkpointing, Spill management, Resource tuning
23	RDD Execution Model	Job submission, Stage creation, Task execution, DAG scheduling, Execution flow
24	RDD with Hive & HDFS	HDFS integration, Hive tables, InputFormat, OutputFormat, Metadata handling
25	Advanced Key-Value Operations	combineByKey internals, map-side aggregation, Custom combiners, Performance tuning, Examples
26	Checkpointing & Reliability	Checkpoint types, Reliable storage, Performance trade-offs, Use cases, Configuration
27	Security & Access Control	Authentication, Authorization, Secure clusters, Kerberos basics, Best practices
28	Testing RDD Code	Local mode testing, Unit testing, Test data, Assertions, Debug runs
29	Best Practices & Anti-patterns	Common mistakes, Performance anti-patterns, Code structure, Reusability, Guidelines
30	Real-world Use Cases & Projects	Log processing, ETL pipelines, Graph processing, Analytics workloads, Review

Interview question

Basic Level

1. What is an RDD in Apache Spark?
2. What does RDD stand for?
3. What are the main characteristics of RDDs?
4. How is RDD different from a DataFrame?
5. Is RDD immutable? Why?
6. How do you create an RDD in Spark?
7. What are the two ways to create an RDD?
8. What is parallelize() in Spark?
9. What is textFile() method?
10. What is lazy evaluation in RDD?
11. What are transformations in RDD?
12. What are actions in RDD?
13. Give examples of RDD transformations.
14. Give examples of RDD actions.
15. What is map() transformation?
16. What is flatMap()?
17. Difference between map() and flatMap()?
18. What is filter() in RDD?
19. What is collect() action?
20. What is count() action?
21. What is take() action?
22. What is foreach() in RDD?
23. What is SparkContext?
24. What is an RDD partition?
25. Why is RDD fault tolerant?

Intermediate Level

26. What is lineage in RDD?
27. How does RDD achieve fault tolerance?
28. What is DAG in Spark?
29. What is narrow transformation?
30. What is wide transformation?
31. Difference between narrow and wide transformations?
32. What is shuffle in Spark?
33. Which RDD operations cause shuffle?
34. What is reduceByKey()?
35. Difference between reduceByKey() and groupByKey()?
36. What is aggregateByKey()?
37. What is combineByKey()?
38. What is cache() in RDD?
39. What is persist()?
40. Difference between cache() and persist()?
41. What storage levels are supported in RDD?
42. What is MEMORY_ONLY storage level?
43. What happens if cached RDD does not fit in memory?
44. What is checkpointing?
45. Difference between cache and checkpoint?
46. What is coalesce()?
47. Difference between coalesce() and repartition()?
48. What is union() operation?
49. What is intersection()?
50. What is subtract() in RDD?

Advanced Level

51. What is a Pair RDD?
52. How do you create a Pair RDD?
53. What is sortByKey()?
54. How does partitioning work in RDD?
55. What is HashPartitioner?
56. What is RangePartitioner?
57. Why is partitioning important?
58. What is mapPartitions()?
59. Difference between map() and mapPartitions()?
60. What is foreachPartition()?
61. What is zip() in RDD?
62. What are broadcast variables?
63. What are accumulators?
64. Can RDDs be shared between Spark applications?
65. What is task in Spark?
66. What is stage in Spark?
67. How are stages created from RDD operations?
68. What is speculative execution?
69. What is memory spill in Spark?
70. How does Spark handle executor memory?
71. What is a long lineage problem?
72. How does checkpoint help performance?
73. What is serialization in RDD?
74. Java serializer vs Kryo serializer?
75. How to optimize RDD performance?

Expert Level

76. When should you use RDD over DataFrame?
77. Why are RDDs considered low-level APIs?
78. How does RDD differ from Dataset?
79. How does Spark recover lost RDD partitions?
80. What happens if a node fails during RDD computation?
81. How does Spark schedule RDD tasks?
82. What is locality level in Spark?
83. Explain process of RDD execution internally.
84. How does Spark avoid recomputation?
85. Explain RDD execution with an example job.
86. How do you debug RDD performance issues?
87. How does garbage collection affect RDD performance?
88. What is out-of-memory error in RDD jobs?
89. How to tune Spark for large RDD jobs?
90. What are common RDD anti-patterns?
91. Why groupByKey is discouraged?
92. How do you manage skewed data in RDD?
93. Explain custom partitioner usage.
94. What is RDD persistence across stages?
95. Can RDD operations be optimized by Catalyst?
96. Why Spark recommends DataFrame over RDD?
97. How does RDD handle schema-less data?
98. What are the limitations of RDD?
99. How does RDD support iterative algorithms?
100. Explain RDD usage in real-time production scenarios.

Related Topics

#DataFrames

Last updated    - V7 (09-Jan-2026)

Key Concepts

S.No	Topic	Sub-Topics
1	Apache Spark & DataFrames	Spark overview, RDD vs DataFrame, Spark architecture, Lazy evaluation, Use cases
2	Spark Setup & Environment	Local mode, Cluster mode, SparkSession, spark-submit, Configuration basics
3	SparkSession & Entry Points	SparkSession creation, SQLContext, HiveContext, Config options, Best practices
4	Creating DataFrames	From files, From RDD, From collections, Schema inference, Explicit schema
5	DataFrame Schema & Data Types	StructType, StructField, Primitive types, Complex types, Schema evolution
6	Reading Data Sources	CSV, JSON, Parquet, ORC, Avro basics
7	Writing DataFrames	Save modes, Partitioning, Bucketing, File formats, Compression
8	DataFrame Basic Operations	select, withColumn, drop, filter, where
9	Column Operations	Column expressions, alias, cast, when/otherwise, lit
10	Row Operations & Actions	show, collect, take, count, first
11	DataFrame Functions	Built-in functions, String functions, Date functions, Math functions, Null handling
12	Filtering & Conditional Logic	filter vs where, isin, like, rlike, case when
13	Sorting & Deduplication	orderBy, sort, distinct, dropDuplicates, Sorting optimization
14	Aggregation & Grouping	groupBy, agg, count, sum, avg
15	Joins in DataFrames	Inner join, Left/Right join, Full join, Semi/Anti join
16	Join Optimization	Broadcast join, Shuffle join, Join hints, Skew handling, AQE
17	Handling Missing & Bad Data	dropna, fillna, replace, Null checks, Data validation
18	Window Functions	Window spec, row_number, rank, lead/lag, Running totals
19	UDF & UDAF	UDF creation, Performance impact, Pandas UDF, Serialization, Best practices
20	DataFrame Caching & Persistence	cache, persist, Storage levels, Memory vs disk, When to cache
21	Spark SQL with DataFrames	Temp views, Global views, SQL queries, Mixing SQL & DF, Optimization
22	Partitioning & Repartitioning	repartition, coalesce, Partition pruning, File partitioning, Performance tuning
23	Performance Optimization Basics	Catalyst optimizer, Tungsten, Predicate pushdown, Column pruning, AQE
24	DataFrame Execution Plan	Logical plan, Physical plan, explain(), DAG, Stage breakdown
25	Handling Large Datasets	Skew issues, Sampling, Checkpointing, Memory tuning, Spill handling
26	Integration with Hive	Hive tables, External tables, Metastore, Partitioned tables, Hive SQL
27	Streaming DataFrames (Structured Streaming)	Streaming sources, Sinks, Watermarking, Windowed aggregations, Triggers
28	Error Handling & Debugging	Common errors, Serialization issues, Logging, Debug tools, Retry strategies
29	Best Practices & Design Patterns	Code structure, Reusability, Performance patterns, Anti-patterns, Testing
30	Real-world Use Cases & Projects	ETL pipelines, Data lake processing, Analytics workloads, Reporting, Optimization review

Interview question

Basic Level

1. What is a DataFrame in Apache Spark?
2. How is a DataFrame different from an RDD?
3. What are the advantages of DataFrames?
4. Is DataFrame immutable?
5. What is SparkSession?
6. How do you create a DataFrame in Spark?
7. What are the different data sources supported by DataFrames?
8. What is a schema in DataFrame?
9. How can you infer schema automatically?
10. How do you define a custom schema?
11. What is show() in DataFrame?
12. What is printSchema()?
13. What is select() in DataFrame?
14. What is withColumn()?
15. Difference between withColumn() and select()?
16. What is filter() / where() in DataFrame?
17. Difference between filter() and where()?
18. What is limit()?
19. What is collect()?
20. What is count()?
21. What is distinct()?
22. What is drop()?
23. What is dropDuplicates()?
24. What is alias()?
25. How do you rename a column in DataFrame?

Intermediate Level

26. What are DataFrame transformations?
27. What are DataFrame actions?
28. What is lazy evaluation in DataFrames?
29. What is explain() in DataFrame?
30. What is logical plan?
31. What is physical plan?
32. What is Catalyst Optimizer?
33. What is Tungsten execution engine?
34. What is column expression?
35. What are built-in Spark SQL functions?
36. Difference between UDF and built-in functions?
37. What is UDF?
38. Performance impact of UDFs?
39. What is groupBy()?
40. What is agg()?
41. Difference between groupBy and window functions?
42. What is orderBy()?
43. Difference between orderBy() and sort()?
44. What is join() in DataFrame?
45. What are the types of joins in Spark?
46. What is inner join?
47. What is left outer join?
48. What is right outer join?
49. What is full outer join?
50. What is cross join?

Advanced Level

51. What is broadcast join?
52. When does Spark automatically use broadcast join?
53. What is shuffle join?
54. How does Spark handle join optimization?
55. What is partitioning in DataFrame?
56. Difference between repartition() and coalesce()?
57. What is bucketing in Spark?
58. Difference between bucketing and partitioning?
59. What is window function?
60. Explain row_number(), rank(), dense_rank().
61. What is caching in DataFrame?
62. Difference between cache() and persist()?
63. What storage levels are available?
64. What is checkpointing in DataFrame?
65. Difference between cache and checkpoint?
66. What is skewed data?
67. How to handle data skew in joins?
68. What is salting technique?
69. What is adaptive query execution (AQE)?
70. What are Spark SQL hints?
71. What is explain(true)?
72. How to optimize wide transformations?
73. What is column pruning?
74. What is predicate pushdown?
75. What is vectorized reader?

Expert Level

76. Why DataFrames are faster than RDDs?
77. How does Catalyst optimize DataFrame queries?
78. How does Tungsten improve performance?
79. What happens internally when a DataFrame action is triggered?
80. How does Spark generate optimized bytecode?
81. What is whole-stage code generation?
82. What is off-heap memory?
83. How does Spark handle memory management for DataFrames?
84. How does AQE change execution plans at runtime?
85. How do you debug slow DataFrame jobs?
86. How do you analyze Spark UI for DataFrame jobs?
87. What are common DataFrame performance anti-patterns?
88. Why excessive withColumn() is discouraged?
89. How do you design efficient Spark SQL pipelines?
90. What are limitations of DataFrames?
91. How do DataFrames handle schema evolution?
92. How does DataFrame support semi-structured data?
93. What is explode() function?
94. What is from_json() and to_json()?
95. How to handle nested columns efficiently?
96. What is Delta Lake DataFrame integration?
97. How does DataFrame handle ACID properties?
98. What is cost-based optimization (CBO)?
99. How do you tune Spark SQL configurations?
100. Explain real-time production use cases of DataFrames.

Related Topics

#Vector Databases

Last updated    - V7 (09-Jan-2026)

Key Concepts

S.No	Topic	Sub-Topics
1	Vector Databases	What is a vector, embeddings basics, similarity search, use cases, traditional DB vs vector DB
2	Mathematics Behind Vectors	Linear algebra basics, cosine similarity, Euclidean distance, dot product, normalization
3	Embeddings Fundamentals	Text embeddings, image embeddings, dimensionality, dense vs sparse vectors, embedding quality
4	Embedding Models	Word2Vec, GloVe, FastText, Sentence Transformers, OpenAI embeddings
5	Vector Similarity Metrics	Cosine similarity, L2 distance, inner product, Hamming distance, trade-offs
6	Indexing Techniques	Flat index, inverted index, IVF, HNSW, PQ
7	Approximate Nearest Neighbor (ANN)	Why ANN, accuracy vs speed, recall, latency, scalability
8	HNSW Algorithm	Graph layers, insertion, search process, parameters, performance tuning
9	IVF and PQ Indexes	Clustering, centroids, compression, memory optimization, search flow
10	Index Optimization	Re-indexing, shard sizing, dimension reduction, caching, pruning
11	Popular Vector Databases	Pinecone, Weaviate, Milvus, Qdrant, FAISS overview
12	Pinecone Architecture	Indexes, namespaces, pods, metadata filtering, scaling
13	Weaviate Architecture	Schema, classes, modules, GraphQL API, hybrid search
14	Milvus Architecture	Collections, partitions, segments, coordinators, storage layers
15	Open Source vs Managed Vector DBs	Cost, scalability, maintenance, performance, use cases
16	Data Modeling in Vector DBs	Vector schema, metadata fields, hybrid models, versioning, updates
17	CRUD Operations	Insert vectors, update vectors, delete vectors, batch operations, upserts
18	Metadata Filtering	Structured filters, boolean logic, range queries, hybrid search, performance impact
19	Hybrid Search	Keyword + vector search, BM25, re-ranking, score fusion, use cases
20	Vector Search APIs	REST APIs, SDKs, query parameters, pagination, result tuning
21	Scalability & Sharding	Horizontal scaling, replicas, partitioning, load balancing, fault tolerance
22	Performance Tuning	Latency optimization, recall tuning, batch queries, memory usage, caching
23	Consistency & Durability	Replication, WAL, backups, recovery, data integrity
24	Security in Vector Databases	Authentication, authorization, encryption, network security, compliance
25	Monitoring & Observability	Metrics, logging, tracing, alerts, capacity planning
26	Vector DB + LLM Integration	RAG pattern, prompt injection, context windowing, retrieval tuning, pipelines
27	Semantic Search Applications	Search engines, document retrieval, FAQs, chatbots, recommendation systems
28	Multimodal Vector Search	Text-image search, audio embeddings, video embeddings, fusion strategies, use cases
29	Production Best Practices	Schema evolution, re-embedding, cost optimization, SLA planning, testing
30	Advanced & Future Trends	Vector + graph DBs, real-time embeddings, edge deployment, AI agents, research trends

Interview question

Related Topics

#RAG

Last updated    - V7 (09-Jan-2026)

Key Concepts

S.No	Topic	Sub-Topics
1	RAG	What is RAG, Why RAG, RAG vs LLM-only, RAG use cases, RAG limitations
2	LLM Fundamentals for RAG	Transformer basics, Context window, Tokens, Prompt-response flow, Hallucinations
3	Text Embeddings	What are embeddings, Vector representation, Embedding models, Dimensionality, Similarity meaning
4	Embedding Models	OpenAI embeddings, SentenceTransformers, Multilingual embeddings, Trade-offs, Model selection
5	Vector Databases Basics	Vector DB concept, ANN search, Indexing basics, Metadata storage, Vector lifecycle
6	Vector DB Tools	FAISS, Pinecone, Weaviate, Milvus, ChromaDB
7	Distance Metrics	Cosine similarity, Dot product, Euclidean distance, Trade-offs, Metric selection
8	Chunking Strategies	Fixed chunking, Semantic chunking, Chunk size, Overlap, Parent-child chunks
9	Document Ingestion	PDF ingestion, Text files, HTML ingestion, Cleaning text, Normalization
10	Indexing Pipeline	Embedding generation, Batch indexing, Metadata tagging, Versioning, Index updates
11	Retrieval Basics	Top-k retrieval, Similarity threshold, Recall vs precision, Retrieval latency, Query flow
12	Hybrid Search	Dense search, Sparse search, Keyword search, BM25, Hybrid ranking
13	Metadata Filtering	Structured filters, Access control, User-based filtering, Time filters, Security filters
14	Prompt Engineering for RAG	Prompt templates, Context injection, Instructions, Citations, Answer formatting
15	Naive RAG Architecture	Single retriever, Single prompt, Context stuffing, Limitations, Failure cases
16	Advanced RAG Architecture	Multi-retriever, Reranking, Compression, Query rewriting, Modular design
17	Reranking Techniques	Cross-encoders, Relevance scoring, Latency trade-off, Top-n rerank, Quality boost
18	Context Optimization	Token limits, Context pruning, Compression, Redundancy removal, Ordering chunks
19	Multi-hop Retrieval	Complex queries, Query decomposition, Iterative retrieval, Chain-of-thought, Examples
20	Agentic RAG	LLM agents, Tool calling, Planner-executor, Memory, Autonomous retrieval
21	Structured Data RAG	SQL integration, CSV data, APIs, Knowledge graphs, Hybrid retrieval
22	RAG with LangChain	Retrievers, Chains, Vector stores, Memory, RAG pipelines
23	RAG with LlamaIndex	Indexes, Query engines, Node parsing, Storage context, Tools
24	Evaluation of RAG	Retrieval metrics, Answer quality, Faithfulness, Relevance, Latency
25	RAGAS Framework	Faithfulness score, Context recall, Answer relevance, Ground truth, Automation
26	Security in RAG	Prompt injection, Data leakage, RBAC, PII handling, Secure retrieval
27	Scalability & Performance	Index sharding, Caching, Async retrieval, Load balancing, Cost control
28	Production Deployment	API design, Model hosting, Vector DB hosting, Monitoring, Logging
29	Monitoring & Feedback	User feedback, Drift detection, Retrieval errors, Continuous improvement, Alerts
30	Enterprise RAG Use Cases	Chatbots, Search engines, Knowledge assistants, Analytics, Decision support

Interview question

Basic Level

1. What is Retrieval-Augmented Generation (RAG)?
2. Why is RAG needed for LLM applications?
3. What problems does RAG solve?
4. What are the core components of a RAG system?
5. What is retrieval in RAG?
6. What is generation in RAG?
7. How is RAG different from fine-tuning?
8. How is RAG different from prompt engineering?
9. What is a knowledge base in RAG?
10. What type of data can RAG consume?
11. What are embeddings?
12. Why are embeddings used in RAG?
13. What is a vector database?
14. Examples of vector databases?
15. What is semantic search?
16. What is similarity search?
17. What distance metrics are commonly used?
18. What is cosine similarity?
19. What is text chunking?
20. Why is chunking important in RAG?
21. What is context window?
22. What is prompt grounding?
23. What is hallucination in LLMs?
24. How does RAG reduce hallucinations?
25. What are common RAG use cases?

Intermediate Level

26. Explain the end-to-end RAG workflow.
27. How are embeddings generated?
28. Which embedding models are commonly used?
29. What is embedding dimensionality?
30. How does chunk size affect retrieval?
31. What is chunk overlap?
32. What is metadata filtering?
33. What is hybrid search?
34. Difference between sparse and dense retrieval?
35. What is keyword search vs vector search?
36. What is top-k retrieval?
37. How do you decide the value of k?
38. What is reranking?
39. Why is reranking important?
40. What is prompt templating in RAG?
41. How is retrieved context injected into prompts?
42. What is latency challenge in RAG?
43. How do you improve RAG response speed?
44. What is document indexing?
45. How do you update knowledge base data?
46. What is FAISS?
47. What is Pinecone?
48. What is Weaviate?
49. What is Chroma DB?
50. What role does LangChain play in RAG?

Advanced Level

51. What are different RAG architectures?
52. What is naive RAG?
53. What is advanced RAG?
54. What is agentic RAG?
55. What is multi-hop retrieval?
56. What is query rewriting?
57. What is a self-query retriever?
58. What is parent-child chunking?
59. Difference between document-level and chunk-level retrieval?
60. What is contextual compression?
61. How do you handle long documents in RAG?
62. How does RAG integrate with structured data?
63. How can SQL databases be used in RAG?
64. What is retrieval evaluation?
65. What metrics are used to evaluate RAG?
66. What is recall vs precision in RAG?
67. What is MMR (Max Marginal Relevance)?
68. How does MMR help improve answer quality?
69. What is data skew in retrieval?
70. How do you handle stale data?
71. How do you implement real-time RAG?
72. How is access control handled in RAG?
73. How do you secure sensitive documents?
74. How does multilingual RAG work?
75. What are common RAG failure patterns?

Expert Level

76. How do you design a production-grade RAG system?
77. How does RAG scale to millions of documents?
78. What are trade-offs between RAG and fine-tuning?
79. How do you optimize RAG for low latency?
80. How do you debug poor RAG responses?
81. What causes irrelevant retrieval?
82. How do you improve retrieval accuracy?
83. How do context limits impact RAG?
84. What strategies help reduce token usage?
85. How do you prevent prompt injection in RAG?
86. How do you measure answer faithfulness?
87. What is RAGAS evaluation framework?
88. How do you monitor RAG systems in production?
89. How do you build feedback loops?
90. What is continuous indexing?
91. How do you version embeddings?
92. How do you migrate vector databases safely?
93. How do you control RAG operational costs?
94. How do you handle LLM model upgrades?
95. How does RAG enable explainability?
96. What is citation-based RAG?
97. How does RAG work with AI agents?
98. What are emerging RAG patterns?
99. What are the limitations of RAG?
100. Explain enterprise-level RAG use cases.

Related Topics