A Simple and Fast Redshift Loader
Are you struggling with the complexities of loading data into Redshift? Is your existing
process running efficiently? Are you looking for an easy button? PreClarity has the answer. The PreClarity Redshift Loader software module makes loading data into AWS Redshift a snap. Our software covers all the complex and time-consuming tasks including the required AWS S3 provisioning.
- Fully Automated S3 provisioning and management at no cost to you
- No reams of web pages to read and figure out how to set up S3 and load processes and no configuration hassles
- Reduce days of work to a few minutes
- Automatic creation of the database table structure with the correct column names & datatypes saving you time & eliminating manual effort!
- Easily loads files from local or remote mounted file systems with a single command. Save days and weeks of work!
- Runs On Any Flavor Of Windows Or Linux
- Supports Files In CSV Format (comma, tab, pipe, caret or any other field separator)
- Works With Uncompressed Files Or With Anyl Popular File Compression/Archive Format (zip, gz, 7z, bz2, tar, gzip)
- Other File Formats Available (JSON, XML, ASN.1, Binary, Custom)
How it Works…
- Register your username and company & download the loader software
- We’ll send you confirmation and get the necessary security information from you to load into your Redshift instance
- Run the downloaded software by providing your table name and file(s) and your data will be seamlessly loaded into your Redshift instance
- It’s simple, easy and fully secure.
PreClarity no longer offers this as a free service. If you would like to inquire about implementing this as a paid service, please Contact us.
A Quick Use Case
So you’ve just got your shiny new Redshift instance and you’re ready to load data.
If you use PreClarity’s Redshift Loader : You’ll be automatically loading your data in minutes
If you don’t…
You’ll have a few things to think about and work through…
Did you buy an S3 bucket to use as an interim staging area for the files you want to load? You’re going to need one, that’s the only way Redshift can load your file.
You’ll need to configure your S3 bucket and ensure it is in the same region as your database. You should probably configure a different user and set up proper permissions and get a new public and private key pair. Once that is done and you’re ready to move files, do some research on the SDK and write a program to move your files to S3, but as Amazon says in their documentation “this can be cumbersome because it requires you to write code to authenticate your requests”, so they suggest an alternative way which requires more authentication keys and another few pages of instructions.
Hours if not days later you might have been able to upload a file or two up into the S3 interim staging area. Now your headed over to the Redshift side, time to visually inspect all the data in the file you are trying to load and manually create the DDL that matches your file layout with appropriate column names and the appropriate Redshift column data types. Once all of that is accomplished it is time to go back to S3 and get your s3 bucket root, object prefixes and aws-auth-args so you can knuckle in the Redshift command to load your file.
If the stars align and this all goes well you’ll have a single file loaded into Redshift. Now on to cleanup to remove the file from S3 and ensure that it doesn’t get re-uploaded and loaded again. This is left as an exercise for the reader.