Gathr Store Ingestion Target
In this article
Gathr Store target allows you to briefly store data in Gathr.
An application with Gathr Store target is supported to run on Gathr clusters and not on registered clusters.
Such applications can be deployed only on a Gathr cluster. If you want to use a registered cluster, please choose another target.
Target Configuration
File Format: Output file format choice in which the results should be processed.
Delimiter: A message field separator should be selected for CSV (Delimited) file formats.
Output Fields: Select the preferred columns that should get written into the Gathr Store Output →
Compression Type: Based on the File Format, you can select the supported compression type in which the output files will get compressed and stored in the Gathr Store.
See the sub-topic Supported Compression Types to know more about the compression types supported for each file format.
Generate single output file: The file generated in the Gathr Store Output → will be a single file if this option is selected.
Otherwise, the output zip file will contain multiple files as per the partitions applied.
Add configuration: Additional properties can be added using Add Configuration link.
Supported Compression Types
The compression types supported for each file format in Gathr Store are given in the table below:
Supported Compression Types | File Formats | ||||
---|---|---|---|---|---|
Delimited | JSON | Parquet | ORC | AVRO | |
— | — | — | — | — | — |
NONE | Yes | Yes | Yes | Yes | No |
SNAPPY | Yes | Yes | Yes | Yes | Yes |
GZIP | Yes | Yes | Yes | No | No |
LZO | No | No | Yes | Yes | No |
ZLIB | No | No | No | Yes | No |
DEFLATE | Yes | Yes | No | No | Yes |
BZIP2 | Yes | Yes | No | No | Yes |
XZ | No | No | No | No | Yes |
Z4 | Yes | Yes | Yes | No | No |
UNCOMPRESSED | Yes | Yes | Yes | Yes | Yes |
If you have any feedback on Gathr documentation, please email us!