Performance benefits of predict_batch_udf over a Pandas UDF?

119 Views Asked by At

Are there any performance benefits to using predict_batch_udf over creating a pandas_udf?

It does expose a batch_size parameter but is this any different from adjusting spark.sql.execution.arrow.maxRecordsPerBatch to control the Pandas UDF batch sizes?

0

There are 0 best solutions below