Merk
Tilgang til denne siden krever autorisasjon. Du kan prøve å logge på eller endre kataloger.
Tilgang til denne siden krever autorisasjon. Du kan prøve å endre kataloger.
Returns a new DataFrame omitting rows with null or NaN values. DataFrame.dropna and DataFrameNaFunctions.drop are aliases of each other.
Syntax
dropna(how: str = "any", thresh: Optional[int] = None, subset: Optional[Union[str, Tuple[str, ...], List[str]]] = None)
Parameters
| Parameter | Type | Description |
|---|---|---|
how |
str, optional, default 'any' | the values that can be 'any' or 'all'. If 'any', drop a row if it contains any nulls. If 'all', drop a row only if all its values are null. |
thresh |
int, optional, default None | If specified, drop rows that have less than thresh non-null values. This overwrites the how parameter. |
subset |
str, tuple or list, optional | optional list of column names to consider. |
Returns
DataFrame: DataFrame with null only rows excluded.
Examples
from pyspark.sql import Row
df = spark.createDataFrame([
Row(age=10, height=80.0, name="Alice"),
Row(age=5, height=float("nan"), name="Bob"),
Row(age=None, height=None, name="Tom"),
Row(age=None, height=float("nan"), name=None),
])
df.na.drop().show()
# +---+------+-----+
# |age|height| name|
# +---+------+-----+
# | 10| 80.0|Alice|
# +---+------+-----+
df.na.drop(how='all').show()
# +----+------+-----+
# | age|height| name|
# +----+------+-----+
# | 10| 80.0|Alice|
# | 5| NaN| Bob|
# |NULL| NULL| Tom|
# +----+------+-----+
df.na.drop(thresh=2).show()
# +---+------+-----+
# |age|height| name|
# +---+------+-----+
# | 10| 80.0|Alice|
# | 5| NaN| Bob|
# +---+------+-----+