对于数据集，如何使用窗口函数使用其中一列中的值将行拆分为逻辑部分？
例如。对于给定的示例 title = Erev Rosh Hashana
转换将由此产生:
+----------+-------------------------+--------+------+--------------------------------------------------------------------------------+
|date |title |category|subcat|memo |
+----------+-------------------------+--------+------+--------------------------------------------------------------------------------+
|2018-01-31|Tu BiShvat |holiday |minor |New Year for Trees |
|2018-02-28|Erev Purim |holiday |major |Purim is one of the most joyous and fun holidays on the Jewish calendar |
|2018-03-01|Purim |holiday |major |Purim is one of the most joyous and fun holidays on the Jewish calendar |
|2018-03-02|Shushan Purim |holiday |minor |Purim celebrated in Jerusalem and walled cities |
|2018-03-30|Erev Pesach |holiday |null |Passover, the Feast of Unleavened Bread |
|2018-03-31|Pesach I |holiday |major |Passover, the Feast of Unleavened Bread |
|2018-04-01|Pesach II |holiday |major |Passover, the Feast of Unleavened Bread |
|2018-04-02|Pesach III (CH''M) |holiday |major |Passover, the Feast of Unleavened Bread |
|2018-04-03|Pesach IV (CH''M) |holiday |major |Passover, the Feast of Unleavened Bread |
|2018-04-04|Pesach V (CH''M) |holiday |major |Passover, the Feast of Unleavened Bread |
|2018-04-05|Pesach VI (CH''M) |holiday |major |Passover, the Feast of Unleavened Bread |
|2018-04-06|Pesach VII |holiday |major |Passover, the Feast of Unleavened Bread |
|2018-04-07|Pesach VIII |holiday |major |Passover, the Feast of Unleavened Bread |
|2018-04-12|Yom HaShoah |holiday |modern|Holocaust Memorial Day |
|2018-04-18|Yom HaZikaron |holiday |modern|Israeli Memorial Day |
|2018-04-19|Yom HaAtzma'ut |holiday |modern|Israeli Independence Day |
|2018-04-29|Pesach Sheni |holiday |minor |Second Passover, one month after Passover |
|2018-05-03|Lag BaOmer |holiday |minor |33rd day of counting the Omer |
|2018-05-13|Yom Yerushalayim |holiday |modern|Jerusalem Day |
|2018-05-19|Erev Shavuot |holiday |null |Festival of Weeks, commemorates the giving of the Torah at Mount Sinai |
|2018-05-20|Shavuot I |holiday |major |Festival of Weeks, commemorates the giving of the Torah at Mount Sinai |
|2018-05-21|Shavuot II |holiday |major |Festival of Weeks, commemorates the giving of the Torah at Mount Sinai |
|2018-07-21|Erev Tish'a B'Av |holiday |major |The Ninth of Av, fast commemorating the destruction of the two Temples |
|2018-07-22|Tish'a B'Av |holiday |major |The Ninth of Av, fast commemorating the destruction of the two Temples |
|2018-07-27|Tu B'Av |holiday |minor |minor Jewish holiday of love, observed on the 15th day of the Hebrew month of Av|
|2018-09-01|Leil Selichot |holiday |minor |Prayers for forgiveness in preparation for the High Holidays |
|2018-09-09|Erev Rosh Hashana |holiday |null |The Jewish New Year |
|2018-09-10|Rosh Hashana 5779 |holiday |null |The Jewish New Year |
|2018-09-11|Rosh Hashana II |holiday |major |The Jewish New Year |
|2018-09-18|Erev Yom Kippur |holiday |null |Day of Atonement |
|2018-09-19|Yom Kippur |holiday |major |Day of Atonement |
|2018-09-23|Erev Sukkot |holiday |null |Feast of Tabernacles |
|2018-09-24|Sukkot I |holiday |major |Feast of Tabernacles |
|2018-09-25|Sukkot II |holiday |major |Feast of Tabernacles |
|2018-09-26|Sukkot III (CH''M) |holiday |major |Feast of Tabernacles |
|2018-09-27|Sukkot IV (CH''M) |holiday |major |Feast of Tabernacles |
|2018-09-28|Sukkot V (CH''M) |holiday |major |Feast of Tabernacles |
|2018-09-29|Sukkot VI (CH''M) |holiday |major |Feast of Tabernacles |
|2018-09-30|Sukkot VII (Hoshana Raba)|holiday |major |Feast of Tabernacles |
|2018-10-01|Shmini Atzeret |holiday |major |Eighth Day of Assembly |
|2018-10-02|Simchat Torah |holiday |major |Day of Celebrating the Torah |
|2018-10-16|Yom HaAliyah |holiday |modern|Recognizes Aliyah, immigration to the Jewish State of Israel |
|2018-11-07|Sigd |holiday |modern|Ethiopian Jewish holiday occurring 50 days after Yom Kippur |
|2018-12-02|Chanukah: 1 Candle |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2018-12-03|Chanukah: 2 Candles |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2018-12-04|Chanukah: 3 Candles |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2018-12-05|Chanukah: 4 Candles |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2018-12-06|Chanukah: 5 Candles |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2018-12-07|Chanukah: 6 Candles |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2018-12-08|Chanukah: 7 Candles |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2018-12-09|Chanukah: 8 Candles |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2018-12-10|Chanukah: 8th Day |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2019-01-21|Tu BiShvat |holiday |minor |New Year for Trees |
|2019-02-19|Purim Katan |holiday |minor |Minor Purim celebration during Adar I on leap years |
|2019-03-20|Erev Purim |holiday |major |Purim is one of the most joyous and fun holidays on the Jewish calendar |
|2019-03-21|Purim |holiday |major |Purim is one of the most joyous and fun holidays on the Jewish calendar |
|2019-03-22|Shushan Purim |holiday |minor |Purim celebrated in Jerusalem and walled cities |
|2019-04-19|Erev Pesach |holiday |null |Passover, the Feast of Unleavened Bread |
|2019-04-20|Pesach I |holiday |major |Passover, the Feast of Unleavened Bread |
|2019-04-21|Pesach II |holiday |major |Passover, the Feast of Unleavened Bread |
|2019-04-22|Pesach III (CH''M) |holiday |major |Passover, the Feast of Unleavened Bread |
|2019-04-23|Pesach IV (CH''M) |holiday |major |Passover, the Feast of Unleavened Bread |
|2019-04-24|Pesach V (CH''M) |holiday |major |Passover, the Feast of Unleavened Bread |
|2019-04-25|Pesach VI (CH''M) |holiday |major |Passover, the Feast of Unleavened Bread |
|2019-04-26|Pesach VII |holiday |major |Passover, the Feast of Unleavened Bread |
|2019-04-27|Pesach VIII |holiday |major |Passover, the Feast of Unleavened Bread |
|2019-05-02|Yom HaShoah |holiday |modern|Holocaust Memorial Day |
|2019-05-08|Yom HaZikaron |holiday |modern|Israeli Memorial Day |
|2019-05-09|Yom HaAtzma'ut |holiday |modern|Israeli Independence Day |
|2019-05-19|Pesach Sheni |holiday |minor |Second Passover, one month after Passover |
|2019-05-23|Lag BaOmer |holiday |minor |33rd day of counting the Omer |
|2019-06-02|Yom Yerushalayim |holiday |modern|Jerusalem Day |
|2019-06-08|Erev Shavuot |holiday |null |Festival of Weeks, commemorates the giving of the Torah at Mount Sinai |
|2019-06-09|Shavuot I |holiday |major |Festival of Weeks, commemorates the giving of the Torah at Mount Sinai |
|2019-06-10|Shavuot II |holiday |major |Festival of Weeks, commemorates the giving of the Torah at Mount Sinai |
|2019-08-10|Erev Tish'a B'Av |holiday |major |The Ninth of Av, fast commemorating the destruction of the two Temples |
|2019-08-11|Tish'a B'Av |holiday |major |The Ninth of Av, fast commemorating the destruction of the two Temples |
|2019-08-16|Tu B'Av |holiday |minor |minor Jewish holiday of love, observed on the 15th day of the Hebrew month of Av|
|2019-09-21|Leil Selichot |holiday |minor |Prayers for forgiveness in preparation for the High Holidays |
|2019-09-29|Erev Rosh Hashana |holiday |null |The Jewish New Year |
|2019-09-30|Rosh Hashana 5780 |holiday |null |The Jewish New Year |
|2019-10-01|Rosh Hashana II |holiday |major |The Jewish New Year |
|2019-10-08|Erev Yom Kippur |holiday |null |Day of Atonement |
|2019-10-09|Yom Kippur |holiday |major |Day of Atonement |
|2019-10-13|Erev Sukkot |holiday |null |Feast of Tabernacles |
|2019-10-14|Sukkot I |holiday |major |Feast of Tabernacles |
|2019-10-15|Sukkot II |holiday |major |Feast of Tabernacles |
|2019-10-16|Sukkot III (CH''M) |holiday |major |Feast of Tabernacles |
|2019-10-17|Sukkot IV (CH''M) |holiday |major |Feast of Tabernacles |
|2019-10-18|Sukkot V (CH''M) |holiday |major |Feast of Tabernacles |
|2019-10-19|Sukkot VI (CH''M) |holiday |major |Feast of Tabernacles |
|2019-10-20|Sukkot VII (Hoshana Raba)|holiday |major |Feast of Tabernacles |
|2019-10-21|Shmini Atzeret |holiday |major |Eighth Day of Assembly |
|2019-10-22|Simchat Torah |holiday |major |Day of Celebrating the Torah |
|2019-11-05|Yom HaAliyah |holiday |modern|Recognizes Aliyah, immigration to the Jewish State of Israel |
|2019-11-27|Sigd |holiday |modern|Ethiopian Jewish holiday occurring 50 days after Yom Kippur |
|2019-12-22|Chanukah: 1 Candle |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2019-12-23|Chanukah: 2 Candles |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2019-12-24|Chanukah: 3 Candles |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2019-12-25|Chanukah: 4 Candles |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2019-12-26|Chanukah: 5 Candles |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2019-12-27|Chanukah: 6 Candles |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2019-12-28|Chanukah: 7 Candles |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2019-12-29|Chanukah: 8 Candles |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2019-12-30|Chanukah: 8th Day |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
+----------+-------------------------+--------+------+--------------------------------------------------------------------------------+
以下内容:
+----+----------+-------------------------+--------+------+--------------------------------------------------------------------------------+
|rank|date |title |category|subcat|memo |
+----+----------+-------------------------+--------+------+--------------------------------------------------------------------------------+
|1 |2018-01-31|Tu BiShvat |holiday |minor |New Year for Trees |
|1 |2018-02-28|Erev Purim |holiday |major |Purim is one of the most joyous and fun holidays on the Jewish calendar |
|1 |2018-03-01|Purim |holiday |major |Purim is one of the most joyous and fun holidays on the Jewish calendar |
|1 |2018-03-02|Shushan Purim |holiday |minor |Purim celebrated in Jerusalem and walled cities |
|1 |2018-03-30|Erev Pesach |holiday |null |Passover, the Feast of Unleavened Bread |
|1 |2018-03-31|Pesach I |holiday |major |Passover, the Feast of Unleavened Bread |
|1 |2018-04-01|Pesach II |holiday |major |Passover, the Feast of Unleavened Bread |
|1 |2018-04-02|Pesach III (CH''M) |holiday |major |Passover, the Feast of Unleavened Bread |
|1 |2018-04-03|Pesach IV (CH''M) |holiday |major |Passover, the Feast of Unleavened Bread |
|1 |2018-04-04|Pesach V (CH''M) |holiday |major |Passover, the Feast of Unleavened Bread |
|1 |2018-04-05|Pesach VI (CH''M) |holiday |major |Passover, the Feast of Unleavened Bread |
|1 |2018-04-06|Pesach VII |holiday |major |Passover, the Feast of Unleavened Bread |
|1 |2018-04-07|Pesach VIII |holiday |major |Passover, the Feast of Unleavened Bread |
|1 |2018-04-12|Yom HaShoah |holiday |modern|Holocaust Memorial Day |
|1 |2018-04-18|Yom HaZikaron |holiday |modern|Israeli Memorial Day |
|1 |2018-04-19|Yom HaAtzma'ut |holiday |modern|Israeli Independence Day |
|1 |2018-04-29|Pesach Sheni |holiday |minor |Second Passover, one month after Passover |
|1 |2018-05-03|Lag BaOmer |holiday |minor |33rd day of counting the Omer |
|1 |2018-05-13|Yom Yerushalayim |holiday |modern|Jerusalem Day |
|1 |2018-05-19|Erev Shavuot |holiday |null |Festival of Weeks, commemorates the giving of the Torah at Mount Sinai |
|1 |2018-05-20|Shavuot I |holiday |major |Festival of Weeks, commemorates the giving of the Torah at Mount Sinai |
|1 |2018-05-21|Shavuot II |holiday |major |Festival of Weeks, commemorates the giving of the Torah at Mount Sinai |
|1 |2018-07-21|Erev Tish'a B'Av |holiday |major |The Ninth of Av, fast commemorating the destruction of the two Temples |
|1 |2018-07-22|Tish'a B'Av |holiday |major |The Ninth of Av, fast commemorating the destruction of the two Temples |
|1 |2018-07-27|Tu B'Av |holiday |minor |minor Jewish holiday of love, observed on the 15th day of the Hebrew month of Av|
|1 |2018-09-01|Leil Selichot |holiday |minor |Prayers for forgiveness in preparation for the High Holidays |
|2 |2018-09-09|Erev Rosh Hashana |holiday |null |The Jewish New Year |
|2 |2018-09-10|Rosh Hashana 5779 |holiday |null |The Jewish New Year |
|2 |2018-09-11|Rosh Hashana II |holiday |major |The Jewish New Year |
|2 |2018-09-18|Erev Yom Kippur |holiday |null |Day of Atonement |
|2 |2018-09-19|Yom Kippur |holiday |major |Day of Atonement |
|2 |2018-09-23|Erev Sukkot |holiday |null |Feast of Tabernacles |
|2 |2018-09-24|Sukkot I |holiday |major |Feast of Tabernacles |
|2 |2018-09-25|Sukkot II |holiday |major |Feast of Tabernacles |
|2 |2018-09-26|Sukkot III (CH''M) |holiday |major |Feast of Tabernacles |
|2 |2018-09-27|Sukkot IV (CH''M) |holiday |major |Feast of Tabernacles |
|2 |2018-09-28|Sukkot V (CH''M) |holiday |major |Feast of Tabernacles |
|2 |2018-09-29|Sukkot VI (CH''M) |holiday |major |Feast of Tabernacles |
|2 |2018-09-30|Sukkot VII (Hoshana Raba)|holiday |major |Feast of Tabernacles |
|2 |2018-10-01|Shmini Atzeret |holiday |major |Eighth Day of Assembly |
|2 |2018-10-02|Simchat Torah |holiday |major |Day of Celebrating the Torah |
|2 |2018-10-16|Yom HaAliyah |holiday |modern|Recognizes Aliyah, immigration to the Jewish State of Israel |
|2 |2018-11-07|Sigd |holiday |modern|Ethiopian Jewish holiday occurring 50 days after Yom Kippur |
|2 |2018-12-02|Chanukah: 1 Candle |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2 |2018-12-03|Chanukah: 2 Candles |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2 |2018-12-04|Chanukah: 3 Candles |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2 |2018-12-05|Chanukah: 4 Candles |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2 |2018-12-06|Chanukah: 5 Candles |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2 |2018-12-07|Chanukah: 6 Candles |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2 |2018-12-08|Chanukah: 7 Candles |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2 |2018-12-09|Chanukah: 8 Candles |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2 |2018-12-10|Chanukah: 8th Day |holiday |major |The Jewish festival of rededication, also known as the Festival of Lights |
|2 |2019-01-21|Tu BiShvat |holiday |minor |New Year for Trees |
|2 |2019-02-19|Purim Katan |holiday |minor |Minor Purim celebration during Adar I on leap years |
|2 |2019-03-20|Erev Purim |holiday |major |Purim is one of the most joyous and fun holidays on the Jewish calendar |
|2 |2019-03-21|Purim |holiday |major |Purim is one of the most joyous and fun holidays on the Jewish calendar |
|2 |2019-03-22|Shushan Purim |holiday |minor |Purim celebrated in Jerusalem and walled cities |
|2 |2019-04-19|Erev Pesach |holiday |null |Passover, the Feast of Unleavened Bread |
|2 |2019-04-20|Pesach I |holiday |major |Passover, the Feast of Unleavened Bread |
|2 |2019-04-21|Pesach II |holiday |major |Passover, the Feast of Unleavened Bread |
|2 |2019-04-22|Pesach III (CH''M) |holiday |major |Passover, the Feast of Unleavened Bread |
|2 |2019-04-23|Pesach IV (CH''M) |holiday |major |Passover, the Feast of Unleavened Bread |
|2 |2019-04-24|Pesach V (CH''M) |holiday |major |Passover, the Feast of Unleavened Bread |
|2 |2019-04-25|Pesach VI (CH''M) |holiday |major |Passover, the Feast of Unleavened Bread |
|2 |2019-04-26|Pesach VII |holiday |major |Passover, the Feast of Unleavened Bread |
|2 |2019-04-27|Pesach VIII |holiday |major |Passover, the Feast of Unleavened Bread |
|2 |2019-05-02|Yom HaShoah |holiday |modern|Holocaust Memorial Day |
|2 |2019-05-08|Yom HaZikaron |holiday |modern|Israeli Memorial Day |
|2 |2019-05-09|Yom HaAtzma'ut |holiday |modern|Israeli Independence Day |
|2 |2019-05-19|Pesach Sheni |holiday |minor |Second Passover, one month after Passover |
|2 |2019-05-23|Lag BaOmer |holiday |minor |33rd day of counting the Omer |
|2 |2019-06-02|Yom Yerushalayim |holiday |modern|Jerusalem Day |
|2 |2019-06-08|Erev Shavuot |holiday |null |Festival of Weeks, commemorates the giving of the Torah at Mount Sinai |
|2 |2019-06-09|Shavuot I |holiday |major |Festival of Weeks, commemorates the giving of the Torah at Mount Sinai |
|2 |2019-06-10|Shavuot II |holiday |major |Festival of Weeks, commemorates the giving of the Torah at Mount Sinai |
|2 |2019-08-10|Erev Tish'a B'Av |holiday |major |The Ninth of Av, fast commemorating the destruction of the two Temples |
|2 |2019-08-11|Tish'a B'Av |holiday |major |The Ninth of Av, fast commemorating the destruction of the two Temples |
|2 |2019-08-16|Tu B'Av |holiday |minor |minor Jewish holiday of love, observed on the 15th day of the Hebrew month of Av|
|2 |2019-09-21|Leil Selichot |holiday |minor |Prayers for forgiveness in preparation for the High Holidays |
|3 |2019-09-29|Erev Rosh Hashana |holiday |null |The Jewish New Year |
|3 |2019-09-30|Rosh Hashana 5780 |holiday |null |The Jewish New Year |
|3 |2019-10-01|Rosh Hashana II |holiday |major |The Jewish New Year |
|3 |2019-10-08|Erev Yom Kippur |holiday |null |Day of Atonement |
|3 |2019-10-09|Yom Kippur |holiday |major |Day of Atonement |
|3 |2019-10-13|Erev Sukkot |holiday |null |Feast of Tabernacles |
|3 |2019-10-14|Sukkot I |holiday |major |Feast of Tabernacles |
|3 |2019-10-15|Sukkot II |holiday |major |Feast of Tabernacles |
|3 |2019-10-16|Sukkot III (CH''M) |holiday |major |Feast of Tabernacles |
|3 |2019-10-17|Sukkot IV (CH''M) |holiday |major |Feast of Tabernacles |
|3 |2019-10-18|Sukkot V (CH''M) |holiday |major |Feast of Tabernacles |
|3 |2019-10-19|Sukkot VI (CH''M) |holiday |major |Feast of Tabernacles |
|3 |2019-10-20|Sukkot VII (Hoshana Raba)|holiday |major |Feast of Tabernacles |
|3 |2019-10-21|Shmini Atzeret |holiday |major |Eighth Day of Assembly |
|3 |2019-10-22|Simchat Torah |holiday |major |Day of Celebrating the Torah |
...
+----+----------+-------------------------+--------+------+--------------------------------------------------------------------------------+
最佳答案
创建一个新列，其中包含 1，其中 title 等于“Erev Rosh Hashana”，否则为 0。然后计算按日期排序的该列的总和。最后，删除临时列。
这可以按如下方式完成:
w = Window.orderBy("date")
df.withColumn("hit", when(df["title"] == "Erev Rosh Hashana", 1).otherwise(0))\
.withColumn("rank", sum("hit").over(w) + 1)
.drop("hit")
将 +1 添加到总和中，使排名从 1 开始。
请注意，对于大型数据帧来说，这可能会很慢。最好有一列作为窗口操作的分区依据，即添加 partitionBy("col")。然而，这不会给出想要的输出。
关于python - Spark窗口函数，根据数据集中的值创建排名列，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/57388963/