Check If Two Pyspark Rows Are Equal

I am writing unit tests for a Spark job, and some of the outputs are named tuples: pyspark.sql.Row How can I assert their equality? actual = get_data(df) expected = Row(total=4, un

Solution 1:

Your code should work as written because according to the docs:

the fields will be sorted by names.

Nevertheless, another way is to use the asDict() method of the pySpark.sql.Row and compare them as dictionaries:

actual = get_data(df)
expected = Row(total=4, unique_ids=2)
self.assertEqual(actual.asDict(), expected.asDict())