Array Functions

Array

Returns an array with the given elements.

Input Parameters:

arg0: The given columns to create array column.

Configuration Parameters No Returns Returns an array with the given elements. Throws Application Exception

Use Case:

array(Column… arg0)

Example:

array(1, 2, 3) will return [1,2,3]

Array_union

Returns an array of the elements in the union of array1 and array2, without duplicates.

Input Parameters:

arg0: The first array column.

arg1: The second array column.

Configuration Parameters No Returns Returns an array of the elements in the union of array1 and array2, without duplicates. Throws Application Exception

Use Case:

array_union(Column arg0, Column arg1)

Example:

array_union(array(1, 2, 3), array(1, 3, 5)) will return [1,2,3,5]

array_distinct

Removes duplicate values from the array.

Input Parameters:

arg0: The given array column.

Configuration Parameters No Returns Returns the array with duplicate values removed. Throws Application Exception

Use Case:

array_distinct(Column arg0)

Example:

array_distinct(array(1, 2, 3, null, 3)) will return [1,2,3,null]

array_except

Returns an array of the elements in array1 but not in array2, without duplicates.

Input Parameters:

arg0: First array column.

arg1: Second array column.

Configuration Parameters No Returns Returns an array of the elements in array1 but not in array2, without duplicates. Throws Application Exception

Use Case:

array_except(Column arg0, Column arg1)

Example:

array_except(array(1, 2, 3), array(1, 3, 5)) will return [2]

array_intersect

Performs intersection of array1 and array2, without duplicates.

Input Parameters:

arg0: First array column.

arg1: Second array column.

Configuration Parameters No Returns Returns an array of the elements in the intersection of array1 and array2, without duplicates. Throws Application Exception

Use Case:

array_intersect(Column arg0, Column arg1)

Example:

array_intersect(array(1, 2, 3), array(1, 3, 5)) will return [1,3]

array_join

Concatenates the elements of the given array using the delimiter and an optional string to replace nulls. If no value is set for nullReplacement, any null value is filtered.

Input Parameters:

arg0: array column.

arg1: delimiter.

arg2: nullReplacement.

Configuration Parameters No Returns Returns the concatenated array. Throws Application Exception

Use Case:

array_join(Column arg0| Object arg1| Object arg2)

Example:

array_join(array(‘hello’, null ,‘world’), ’ ‘,’,’) will return hello , world

array_max

Returns the maximum value in the array. NULL elements are skipped.

Input Parameters:

arg0: The array column.

Configuration Parameters No Returns Returns the maximum value in the array. NULL elements are skipped. Throws Application Exception

Use Case:

array_max(Column arg0)

Example:

array_max(array(1, 20, null, 3)) will return 20

array_min

Returns the minimum value in the array. NULL elements are skipped.

Input Parameters:

arg0: The array column.

Configuration Parameters No Returns Returns the minimum value in the array. NULL elements are skipped. Throws Application Exception

Use Case:

array_min(Column arg0)

Example:

array_min(array(1, 20, null, 3)) will return 1

array_position

Returns the (1-based) index of the first element of the array as long.

Input Parameters:

arg0: The array column.

arg1: The position.

Configuration Parameters No Returns Returns the (1-based) index of the first element of the array as long. Throws Application Exception

Use Case:

array_position(Column arg0, Object arg1)

Example:

array_position(array(3, 2, 1), 1) will return 3

array_remove

Remove all elements that equal to element from array.

Input Parameters:

arg0: The array column.

arg1: The position.

Configuration Parameters No Returns Returns the array with elements removed. Throws Application Exception

Use Case:

array_remove(Column arg0, Object arg1)

Example:

array_remove(array(1, 2, 3, null, 3), 3) will return [1,2,null]

array_repeat

Returns the array containing element count times.

Input Parameters:

arg0: The array column.

arg1: The count

Configuration Parameters No Returns Returns the array containing element count times. Throws Application Exception

Use Case:

array_repeat(Column arg0, Column arg1)

Example:

array_repeat(‘123’, 2) will return [“123”,“123”]

array_sort

Sorts the input array in ascending order. The elements of the input array should be order-able. Null elements will be placed at the end of the returned array.

Input Parameters:

arg0: The array column.

Configuration Parameters No Returns Returns the sorted array. Throws Application Exception

Use Case:

array_sort(Column arg0)

Example:

array_sort(array(‘b’, ‘d’, null, ‘c’, ‘a’)) will return [“a”,“b”,“c”,“d”,null]

array_union

Returns an array of the elements in the union of array1 and array2, without duplicates.

Input Parameters:

arg0: The first array column.

arg1: The second array column.

Configuration Parameters No Returns Returns an array of the elements in the union of array1 and array2, without duplicates. Throws Application Exception

Use Case:

array_union(Column arg0, Column arg1)

Example:

array_union(array(1, 2, 3), array(1, 3, 5)) will return [1,2,3,5]

array_overlap

Returns true if a1 contains at least a non-null element present also in a2. If the arrays have no common element and they are both non-empty and either of them contains a null element null is returned, false otherwise.

Input Parameters:

arg0: The first array column.

arg1: The second array column.

Configuration Parameters No Returns Returns true or false. Throws Application Exception

Use Case:

arrays_overlap(Column arg0, Column arg1)

Example:

arrays_overlap(array(1, 2, 3), array(3, 4, 5)) will return true.

array_zip

Returns a merged array of structs in which the N-th struct contains all N-th values of input arrays.

Input Parameters:

arg0: The Columns to be zipped.

Configuration Parameters No Returns Returns a merged array of structs in which the N-th struct contains all N-th values of input arrays. Throws Application Exception

Use Case:

arrays_zip(Column… arg0)

Example:

arrays_zip(array(1, 2, 3), array(2, 3, 4)) Will return [{“0”:1,“1”:2},{“0”:2,“1”:3},{“0”:3,“1”:4}]

Array_contains

Returns TRUE if the array contains value.

Input Parameters: Column arg0, Object arg1

arg0 (required) - An array column.

arg1 (required) - A value to be checked.

Output Type Boolean Column Configuration Parameters No Returns A boolean true/false Throws Application Exception

Use Case:

array_contains(Column arg0,Object arg1)

Example:

array_contains([“black”,“red”] ,“red”) will return true

explode

Separates the elements of array expr into multiple rows, or the elements of map expr into multiple rows and columns.

Input Parameters:

arg0: The expr column.

Configuration Parameters No Returns Returns the exploded column. Throws Application Exception

Use Case:

explode(Column arg0)

Example:

explode(array(10, 20)) will return 10, 20 in a new column.

explode_outer

Separates the elements of array expr into multiple rows, or the elements of map expr into multiple rows and columns.

Input Parameters:

arg0: The expr column.

Configuration Parameters No Returns Returns the exploded column. Throws Application Exception

Use Case:

explode_outer(Column arg0)

Example:

explode_outer(array(10, 20)) will return 10, 20.

Top