Pinecone Lookup Processor

The Pinecone Lookup Processor in Gathr is your tool for extracting valuable insights from your data.

It allows you to explore and analyze information by performing lookups based on vectors or IDs.

This topic helps you configure the processor effortlessly.

Processor Configuration

Configure the processor parameters as explained below.

Connection Name

A connection name can be selected from the list if you have created and saved connection details for OpenAI earlier. Or create one as explained in the topic - Pinecone Connection →


Index

Select an index to read the data from.


Index Info

Additional information about the selected Index is provided here.

For example: Name, Metric, Dimension, Pod Type, Status, Shards, Replicas, Pods, Metadata Config, and Source Collection.


QUERY CONFIGURATION

Configure the below fields by providing query details.

Query By

Select whether to perform the lookup by Vector or ID.


Input Column

Choose the column based on the above selection (Vector or ID). Example: If Vector is selected, choose the column containing vectors.


Top Matches

Specify the number of top matches to retrieve. Example: Enter ‘5’ to get the top 5 matches.


Include Metadata

Enable this option to include the metadata in the lookup results.


Include Values

Enable this option to include the vectors in the lookup results.


PROCESS LOOKUP RESULTS (MATCHES)

Specify how to process the results from lookup.

Organize Matches

Choose whether to consolidate all matches obtained from lookup results in one row or create a new row for each match.


Process JSON

Specify how to handle JSON data retrieved from the lookup matches. You may parse and assign the JSON attributes, such as ID, Score, Values, and Metadata, to separate output columns or preserve the original JSON structure by assigning it to a designated column.

Update below fields for Parse and assign attributes to columns:

ID

Enter or select a column name where the ID attribute obtained from the matched results should be captured.

Score

Enter or select a column name where the Score attribute obtained from the matched results should be captured.

Values

Enter or select a column name where the Values attribute (i.e., Vectors) obtained from the matched results should be captured.

Metadata

Specify how to handle JSON data retrieved from the lookup matches. You may parse and assign the JSON attributes to separate output columns or preserve the original JSON structure by assigning it to a designated column.

Update below field for Parse and assign attributes to column:

Attribute

Enter the name of the metadata attribute to be parsed.

Column

Enter or select a column name where the parsed metadata attribute should be captured.

Data Type

Select the data type of the metadata attribute being stored.


Update below field for Assign to a column:

Metadata Column

Enter or select a column name where the entire metadata from the matched results should be captured as it is.


Update below field for Assign to a column:

Matches Column

Enter or select a column name where the entire matched results should be captured as it is.


Add Configuration: Additional properties can be added using this option as key-value pairs.

Top