distinct

Returnerar en ny DataFrame som innehåller de distinkta raderna i den här dataramen.

Syntax

distinct()

Retur

DataFrame: DataFrame med distinkta poster.

Exempel

df = spark.createDataFrame(
    [(14, "Tom"), (23, "Alice"), (23, "Alice")], ["age", "name"])
df.distinct().show()
# +---+-----+
# |age| name|
# +---+-----+
# | 14|  Tom|
# | 23|Alice|
# +---+-----+

df.distinct().count()
# 2

df = spark.createDataFrame(
    [(14, "Tom", "M"), (23, "Alice", "F"), (23, "Alice", "F"), (14, "Tom", "M")],
    ["age", "name", "gender"])
df.distinct().show()
# +---+-----+------+
# |age| name|gender|
# +---+-----+------+
# | 14|  Tom|     M|
# | 23|Alice|     F|
# +---+-----+------+