Using the {arrow} and {duckdb} packages to wrangle medical datasets that are Larger than RAM

[@tomfenn4]

Really useful presentation, and timely for me. Personally I find data.table statements are greatly improved with just a little whitespace.

[@tmuffly1]

This talk blew my mind. Thank you very much!

[@higgi13425]

For further learning, here are the links from the next to last slide:
Arrow
cheatsheet: raw.githubusercontent.com/rstudio/cheatsheets/master/arrow.pdf
video intro: th-cam.com/video/O42LUmJZPx0/w-d-xo.html
full workshop from useR!: arrow-user2022.netlify.app
DuckDB
website: duckdb.org
R package: cran.r-project.org/web/packages/duckdb/index.html
data.table
website: rdatatable.gitlab.io/data.table
dtplyr (a data.table translator): dtplyr.tidyverse.org

[@tdawry]

A neat question to answer.
I'm using the duckplyr library and it's nice to not have to think about anything. It does make a strong argument for having a fast hard drive (an SSD is an order of magnitude faster than a traditional HDD, an M2 is an order of magnitude faster than that, and modern nvme drives are even faster).

[@musicspinner]

Masterful deployment of the "Kobayashi Maru" reference. 🖖

[@VictorOrdu]

Wow, thank you for this illuminating presentation.

[@gueyenono]

Great presentation.

[@matthewson8917]

Perfectly summarizes my big data journey. Really good!

[@JohnoScott]

Great talk. Concise and to the point.

[@torbjornstorli2880]

Loved your presentation. Well done Sir!😊

[@multitaskprueba1]

You are a genius! Fantastic video! Thanks!

[@porlando12]

Excellent presentation!

[@ZachRenwickData]

great video and interesting analysis use case!

[@arunabhbarua1924]

How about just using duckdb and SQL?