[Audit][BUG] Ensure GPU handles user specified repartition in Spark 3.4 when AQE is enabled

[NVnavkumar]

Describe the bug
Spark updated AQE to handle a user-specified repartition (e.g using df.partition(<num>) in the output when AQE was enabled. Previously Spark did not have to respect this when AQE was enabled because AQE optimization by definition could never fully respect this partitioning after shuffle. Spark now ensures that the output partitions will match when the user uses repartition by adjusting shuffle partitions afterwards.

Spark commit - apache/spark@801ca252f4

[sameerz]

We need to validate that we are not impacted by doing testing.

[NVnavkumar]

Sort of related, this issue was filed for Spark 3.1 a while ago: #1527