| What is PySpark and how is it different from Apache Spark? |
| Why is PySpark used for large-scale data processing? |
| What are the main components of PySpark? |
| What is SparkSession in PySpark? |
| What is SparkContext and how is it different from SparkSession? |
| What is the difference between SparkSession, SparkContext, and SQLContext? |
| What is a Spark application in PySpark? |
| What is a driver program in PySpark? |
| What are executors in PySpark? |
| What is a cluster manager in Spark? |
| What are the different cluster deployment modes in Spark? |
| What is the difference between client mode and cluster mode? |
| What is an RDD in PySpark? |
| What are the main characteristics of an RDD? |
| What is the difference between RDD and DataFrame? |
| What is the difference between DataFrame and Dataset? |
| Why are DataFrames preferred over RDDs in most PySpark applications? |
| What is lazy evaluation in PySpark? |
| What are transformations and actions in PySpark? |
| What is the difference between narrow and wide transformations? |
| What is a shuffle operation in PySpark? |
| Which PySpark operations cause a shuffle? |
| What is the difference between map(), flatMap(), and mapPartitions()? |
| What is the difference between map() and filter()? |
| What is reduceByKey() in PySpark? |
| What is the difference between reduceByKey() and groupByKey()? |
| What is aggregateByKey() and when should you use it? |
| What is combineByKey() in PySpark? |
| What is the difference between sortByKey() and sortBy()? |
| What is a Pair RDD in PySpark? |
| What is partitioning in PySpark? |
| What is a partition? |
| How does PySpark determine the number of partitions? |
| What is repartition()? |
| What is coalesce()? |
| What is the difference between repartition() and coalesce()? |
| What is data skew in PySpark? |
| How do you identify data skew in PySpark? |
| How do you handle data skew in PySpark? |
| What is a DataFrame in PySpark? |
| How do you create a DataFrame in PySpark? |
| How do you define a schema in PySpark? |
| What is the difference between inferSchema and explicitly defining a schema? |
| What are StructType and StructField in PySpark? |
| What are the commonly used PySpark data types? |
| How do you inspect the schema of a DataFrame? |
| What is the difference between show(), display(), and collect()? |
| What is the difference between count(), collect(), and take()? |
| How do you select columns from a PySpark DataFrame? |
| What is the difference between select() and selectExpr()? |
| How do you rename columns in PySpark? |
| How do you add a new column using withColumn()? |
| What is the difference between withColumn() and withColumns()? |
| How do you drop a column from a DataFrame? |
| How do you filter rows in PySpark? |
| What is the difference between filter() and where()? |
| How do you remove duplicate records in PySpark? |
| What is the difference between distinct() and dropDuplicates()? |
| How do you sort a DataFrame in PySpark? |
| What is the difference between orderBy() and sort()? |
| How do you handle null values in PySpark? |
| What is the difference between isNull() and isNotNull()? |
| How do you replace null values using fillna()? |
| How do you replace values using replace()? |
| What is groupBy() in PySpark? |
| What are aggregate functions in PySpark? |
| How do you calculate sum, average, minimum, and maximum in PySpark? |
| What is the difference between groupBy().agg() and groupBy().sum()? |
| How do you perform joins in PySpark? |
| What are the different types of joins supported by PySpark? |
| What is the difference between inner join and left join? |
| What is a broadcast join in PySpark? |
| When should you use a broadcast join? |
| What is the difference between broadcast join and shuffle join? |
| What is a cross join and why can it be expensive? |
| How do you join two DataFrames with different column names? |
| How do you handle duplicate columns after a join? |
| What are window functions in PySpark? |
| What is Window.partitionBy()? |
| What is Window.orderBy()? |
| What is the difference between row_number(), rank(), and dense_rank()? |
| How do you find the top N records per group using window functions? |
| How do you calculate running totals using PySpark window functions? |
| How do you calculate lag and lead values in PySpark? |
| What are PySpark SQL functions? |
| What is the difference between built-in functions and UDFs? |
| What is a PySpark UDF? |
| Why should UDFs generally be avoided when built-in functions are available? |
| What is a Pandas UDF in PySpark? |
| What is the difference between a regular Python UDF and a Pandas UDF? |
| What is vectorized execution in PySpark? |
| How do you read CSV, JSON, and Parquet files using PySpark? |
| Why is Parquet preferred for analytical workloads in PySpark? |
| What is partition pruning in PySpark? |
| What is predicate pushdown? |
| What is column pruning? |
| What is caching in PySpark? |
| What is the difference between cache() and persist()? |
| What are the different storage levels in PySpark? |
| When should you cache a DataFrame? |
| What is checkpointing in PySpark? |
| What is the difference between caching and checkpointing? |
| What is Spark Catalyst Optimizer? |
| What is Tungsten in Apache Spark? |
| What is Adaptive Query Execution (AQE) in Spark? |
| How does AQE improve PySpark query performance? |
| How do you optimize a slow PySpark job? |
| How do you monitor and troubleshoot a PySpark application? |
| What is the Spark UI and how is it used for performance debugging? |
| What are stages, tasks, and jobs in Spark? |
| How do you identify a bottleneck in a PySpark job? |
| How do you reduce shuffle operations in PySpark? |
| What are common causes of out-of-memory errors in PySpark? |
| How do you handle executor memory issues in PySpark? |
| How do you optimize PySpark code for production workloads? |
| What are common PySpark performance optimization techniques used in real-world projects? |
