Pre-requisites

The Redshift connector requires a user that can be used to login to the database. This user should be configured with sufficient permissions to read rows from the given table that you want to retrieve data from. More information on database security can be found here.

The following sections outlines provides an example for how to set up a user with read-only access to a specific table in the database using SQL commands.

Create a user

# Create a user with a password. Use a username and password of your choice.
CREATE USER validio_user WITH password 'securepassword'

# Create a group for the user. Use a group name of your choice.
CREATE GROUP validio_group;

# Add the newly created user to the created group.
ALTER GROUP validio_group ADD USER validio_user;

Grant the user read only access to the configured table.

# Grant access for the group to user schema containing the table.
GRANT USAGE ON SCHEMA my_schema TO GROUP validio_group;

# Grant read only access for the group to the table in the schema.
GRANT SELECT ON my_schema.my_table TO validio_group;

Redshift configuration parameters

The table below lists the configuration parameters for the connector and provides an explanation as to what each field is and whether it is mandatory.

Field Required Description
Name Identifier for the connector. Used when setting up pipelines.
Host Endpoint of the Redshift cluster database. e.g test-redshift-cluster-1.ynypdanx10uk.eu-north-1.redshift.amazonaws.com
Port Port number of the database.
Username Username of a Redshift user that has access to read the desired table in the database.
Password Password of the specified Redshift user.
Database name Name of the Redshift database.
Schema name name Name of the schema containing the table to read from. This can be left empty if the table resides in the default public schema.
Table name Name of the table from which to fetch data.
Polling interval value How often to query the database for new data. This value is combined with the unit in order to create a polling interval (e.g. a polling interval value of 2 with a unit of “hours” will poll every two hours).
Unit The time unit used for the polling interval value.
Incrementing column name Name of a designated column in the table on which the connector can order rows by when fetching data. See the section on Incrementing Column for more info. Make sure that the selected column is also included in the list of columns that the connector reads.
Data since If you have specified a column of datatype Timestamp as Incrementing column, you can specify a point in time to read data from. You may want to do this if you are only interested in data beginning from a given date. Only records with a timestamp after this given time will be processed.