Downsampling in the Database: Processing Data With Ruby or SQL?

If you're a database professional, you're likely well aware of the importance of data locality when processing and querying data. Downsampling your data according to the data locality principle—processing it as close as possible to where it’s stored—will accelerate the work of developers work and business decision-making.

In the latest edition of RubyConfTH, I used a watermelon juice analogy (yes, you read that right) to discuss the many benefits of downsampling data in the database, including reduced network and I/O overhead, improved performance and scalability, and more accurate processing time measurements.

After the conference, I condensed this information into a blog post about downsampling data in the database and how it can help you improve data analysis.

But if you want to take a deep dive into data locality and explore how data travels, don’t miss this session, where I benchmark the lttb () downsampling algorithm in Ruby against a SQL version powered by Timescale’s hyperfunctions, among other comparisons.

The result? It’s not that Ruby is slow, but the lighter the data travels, the faster it will go (SQL was up to 10x faster seeding data, for example).

Whether you're an experienced database professional or just getting started, I hope this talk will make you consider a different approach regarding data processing and querying efforts—just by doing it in the database. Reach out on our Community Slack (@Jônatas) if you have any questions!

And if you want to try hyperfunctions and see first-hand how they can dramatically accelerate your work, sign up for Timescale (30-day free trial, no credit card required).