Merk
Tilgang til denne siden krever autorisasjon. Du kan prøve å logge på eller endre kataloger.
Tilgang til denne siden krever autorisasjon. Du kan prøve å endre kataloger.
Returns a new DataFrame partitioned by the given partitioning expressions. The resulting DataFrame is partitioned by column identifier.
Syntax
repartitionById(numPartitions: int, *cols: "ColumnOrName")
Parameters
| Parameter | Type | Description |
|---|---|---|
numPartitions |
int | the target number of partitions. |
cols |
str or Column | partitioning columns. |
Returns
DataFrame: Repartitioned DataFrame.
Notes
At least one partition-by expression must be specified. This is similar to repartition in distribution, but preserves the ordering of the rows within each partition.
This is an experimental API.
Examples
from pyspark.sql import functions as sf
spark.createDataFrame(
[(14, "Tom"), (23, "Alice"), (16, "Bob"), (18, "Alice"), (21, "Alice")],
["age", "name"]
).repartitionById(2, "name").select(
"age", "name", sf.spark_partition_id()
).show()
# +---+-----+--------------------+
# |age| name|SPARK_PARTITION_ID()|
# +---+-----+--------------------+
# | 14| Tom| 0|
# | 23|Alice| 1|
# | 18|Alice| 1|
# | 21|Alice| 1|
# | 16| Bob| 0|
# +---+-----+--------------------+