Is your feature request related to a problem or challenge? Please describe what you are trying to do.
This is related to #807 but slightly different. As a user I want to be able to apply a transform function to every element of an array. Suppose I have a dataframe where one column contains an array.
DataFrame()
+-----------+
| a |
+-----------+
| [1, 2, 3] |
| [4, 5] |
| [6] |
| [] |
+-----------+
I might want to convert this with a single line such that each element is converted to whether it is odd.
DataFrame()
+---------------------+
| a_odd |
+---------------------+
| [True, False, True] |
| [False, True] |
| [False] |
| [] |
+---------------------+
It would be great to do something like datafusion.functions.transform(col("a"), my_func).alias("a_odd")
Describe the solution you'd like
Right now I'm not completely sure how I would do this. I think I might have to do an unnest, transform, and aggregate.
Describe alternatives you've considered
These could be done with a UDF.
Additional context
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
This is related to #807 but slightly different. As a user I want to be able to apply a transform function to every element of an array. Suppose I have a dataframe where one column contains an array.
I might want to convert this with a single line such that each element is converted to whether it is odd.
It would be great to do something like
datafusion.functions.transform(col("a"), my_func).alias("a_odd")Describe the solution you'd like
Right now I'm not completely sure how I would do this. I think I might have to do an unnest, transform, and aggregate.
Describe alternatives you've considered
These could be done with a UDF.
Additional context