pivot (GroupdData)

Pivote une colonne de l’agrégation actuelle DataFrame et effectue l’agrégation spécifiée.

Syntaxe

pivot(pivot_col, values=None)

Paramètres

Paramètre Type Description
pivot_col str Nom de la colonne à pivoter.
values list, facultatif Liste des valeurs qui seront traduites en colonnes dans la sortie DataFrame. S’il n’est pas fourni, Spark calcule avec impatience les valeurs distinctes dans pivot_col le schéma obtenu. Fournir une liste explicite évite ce calcul impatient.

Retours

GroupedData

Exemples

from pyspark.sql import Row, functions as sf

df1 = spark.createDataFrame([
    Row(course="dotNET", year=2012, earnings=10000),
    Row(course="Java", year=2012, earnings=20000),
    Row(course="dotNET", year=2012, earnings=5000),
    Row(course="dotNET", year=2013, earnings=48000),
    Row(course="Java", year=2013, earnings=30000),
])

# Compute the sum of earnings for each year by course with each course as a separate column.
df1.groupBy("year").pivot("course", ["dotNET", "Java"]).sum("earnings").sort("year").show()
# +----+------+-----+
# |year|dotNET| Java|
# +----+------+-----+
# |2012| 15000|20000|
# |2013| 48000|30000|
# +----+------+-----+

# Without specifying column values (less efficient).
df1.groupBy("year").pivot("course").sum("earnings").sort("year").show()
# +----+-----+------+
# |year| Java|dotNET|
# +----+-----+------+
# |2012|20000| 15000|
# |2013|30000| 48000|
# +----+-----+------+

# Using a nested column as the pivot column.
df2 = spark.createDataFrame([
    Row(training="expert", sales=Row(course="dotNET", year=2012, earnings=10000)),
    Row(training="junior", sales=Row(course="Java", year=2012, earnings=20000)),
    Row(training="expert", sales=Row(course="dotNET", year=2012, earnings=5000)),
    Row(training="junior", sales=Row(course="dotNET", year=2013, earnings=48000)),
    Row(training="expert", sales=Row(course="Java", year=2013, earnings=30000)),
])
df2.groupBy("sales.year").pivot("sales.course").agg(sf.sum("sales.earnings")).sort("year").show()
# +----+-----+------+
# |year| Java|dotNET|
# +----+-----+------+
# |2012|20000| 15000|
# |2013|30000| 48000|
# +----+-----+------+
  • Last updated on