Gathr Store Ingestion Target

Gathr Store target allows you to briefly store data in Gathr.

An application with Gathr Store target is supported to run on Gathr clusters and not on registered clusters.

Such applications can be deployed only on a Gathr cluster. If you want to use a registered cluster, please choose another target.

Target Configuration

File Format: Output file format choice in which the results should be processed.

Delimiter: A message field separator should be selected for CSV (Delimited) file formats.

Output Fields: Select the preferred columns that should get written into the Gathr Store Output →

Compression Type: Based on the File Format, you can select the supported compression type in which the output files will get compressed and stored in the Gathr Store.

See the sub-topic Supported Compression Types to know more about the compression types supported for each file format.

Generate single output file: The file generated in the Gathr Store Output → will be a single file if this option is selected.

Otherwise, the output zip file will contain multiple files as per the partitions applied.

Add configuration: Additional properties can be added using Add Configuration link.

Supported Compression Types

The compression types supported for each file format in Gathr Store are given in the table below:

Supported Compression TypesFile Formats
DelimitedJSONParquetORCAVRO
NONEYesYesYesYesNo
SNAPPYYesYesYesYesYes
GZIPYesYesYesNoNo
LZONoNoYesYesNo
ZLIBNoNoNoYesNo
DEFLATEYesYesNoNoYes
BZIP2YesYesNoNoYes
XZNoNoNoNoYes
Z4YesYesYesNoNo
UNCOMPRESSEDYesYesYesYesYes
Top