So, what’s new with QuickSight SPICE?
Amazon QuickSight is a scalable business intelligence (BI) service built for the cloud, which allows insights to be shared with all users in the organization. QuickSight offers SPICE, an in-memory, cloud-native data store that allows end-users to interactively explore data. SPICE provides consistently fast query performance and automatically scales for high concurrency.
With SPICE, you save time and cost because you don’t need to retrieve data from the data source (whether a database or data warehouse) every time you change an analysis or update a visual. You also remove the load of concurrent access or analytical complexity off the underlying data source with the data.
The latest updates with SPICE have enabled the following exciting features:
• Incremental refresh – QuickSight now supports incrementally loading new data to SPICE datasets without needing to refresh the full set of data. With incremental refresh, you can update SPICE datasets in a fraction of the time a full refresh would take, enabling access to the most recent insights much sooner. You can schedule incremental refresh to run up to every 15 minutes on a dataset on SQL-based data sources, such as Amazon Redshift, Amazon Athena, PostgreSQL, Microsoft SQL Server or Snowflake.
• 500 million row SPICE capacity (now that’s a lot of rows) – The QuickSight SPICE engine now supports datasets up to 500 million rows or 500 GB in size. This change lets you use SPICE for datasets twice as large than before!
Creating larger datasets, using 32 tables and up to 500 million rows.
Larger datasets and incremental refresh
Let’s say you’re part of the central data team that has access to data tables in data sources. Now you want to create a central dataset for analysts. SPICE can now scale to double the capacity, so you can create a large scaled dataset rather than create and maintain several unconnected datasets.
You can bring in up to 32 tables (from different data sources) in a single dataset to a total of 500 million rows. You can enjoy the double capacity of SPICE with no extra step – it’s automatically available!
With incremental refresh, QuickSight now allows you to ingest data incrementally for your SQL-based sources (such as Amazon Redshift, Athena, PostgreSQL or Snowflake) in a specified time period. When you’re on the Datasets page within Amazon QuickSight, choose the dataset and choose Refresh now or Schedule a refresh.
Setting up incremental refresh (it’s super simple).
Want to learn more about using SPICE features?
Be sure to view the full AWS Big Data article by visiting the link here.
The AWS team have gone to great depths to highlight the key advantages of configuring look-back windows, full ingestion cycles and the benefit to incremental refreshes.
Alternatively, if you’d like assistance with setting up Amazon QuickSight within your own Business Intelligence platform or making the most of larger datasets, please get in touch with us below.