Lazy load data in R

6 years ago
Anonymous $yysEBM5EYi

https://towardsdatascience.com/lazy-loading-data-in-r-2b100acb63fc

Recently I found myself in a position where I needed to load and process a large data files to display on a Shiny dashboard, but loading and processing the whole file, in one go, took a long time. This would force the user to stare at a blank screen for sometime before the results are displayed. I came up with a crude solution to “lazy load” data and process them as needed. When I say lazy load, I mean load/process only the parts that the user currently needs and cache them. Imagine YouTube, loading parts of video as you watch.

There are two key R concepts that I feel are worth explaining as these really shape the technique of lazy loading: Environments and method dispatch.