2 years ago

#40032

test-img

Jonathan B.

Do Magrittr pipes actually "stream"? Or are they just syntactic suger?

Are the pipes from tidyr / magrittr actually doing a streamed pipe operation? Or are these "pipes" just syntactic sugar?

For example, when connecting to a database server to pull down records to apply some model, is the entire data set downloaded first before the next step in the pipe executed? Or is it being streamed across all steps in an async lazy-load manner?

The background here relates to the need to avoid loading tons of records in memory prior to applying a model to the data set.

If piping operations are streaming, then in theory we could avoid running into memory limitations. But, it's not clear if we are using the term "pipe" here in a non-traditional manner.

r

tidyr

magrittr

0 Answers

Your Answer

Accepted video resources