Great ML modelling usecases Julia, I’m wondering if tidymodels can be used along side sparklyr? It would be great to see an use case example using the spark engine.
Yes, in fact, Spark is one of the engines we support for models such as random forest, decision tree, logistic regression, etc. You can see how to set your engine to Spark in the README for parsnip: parsnip.tidymodels.org/
Hey Julia, congrats. Your videos are super kool. May I ask something? Why you don't use the Boruta methodology for choose which variables are better for the model, instead choose them by yourself? I really like your way to programming.
Well, that algorithm can be great but is related to random forest variable importance, as I understand it, so you end up using your training data to decide what to include, when you run into "no free lunch": en.wikipedia.org/wiki/No_free_lunch_theorem That doesn't mean it's not a good option to have but it's certainly not a good default.
@@JuliaSilge yeah...actually this message is not about this specific regression problem, but in classification problem in other videos saw where you did a decision tree. But it's ok. It was just a commentary. Glad for your attention, you are a careful person. Be safe.
If you mean what font am I using in the IDE, I haven't picked anything fancy, just Monaco (maybe even the default?). If you mean what fonts am I using for plotting in ggplot2, those are IBM Plex Sans and I made a little theme for it in this package: github.com/juliasilge/silgelib/
Once again, thanks. Another great explanation of ML modeling. It was beautiful to see that you also have good taste in posters .
You're a brilliant teacher. Thank you
You and David Robinson have inspired me to start my own R channel.
yo bro.. they also inspired me
This is an awesome tutorial. Well explained and thorough. Thank you!
I am having an issue installing tidymodels which is happening while trying to compile rlang, tibble, and yardstick. I get the following (long) message:
* installing *source* package 'backports' ...
** package 'backports' successfully unpacked and MD5 sums checked
** using staged installation
** libs
*** arch - i386
/mingw32/bin/gcc -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c dotsElt.c -o dotsElt.o
0 [main] make 1030 dofork: child -1 - forked process 60900 died unexpectedly, retry 0, exit code 0xC0000142, errno 11
make: fork: Resource temporarily unavailable
ERROR: compilation failed for package 'backports'
* removing 'C:/Program Files/R/R-4.0.2/library/backports'
* restoring previous 'C:/Program Files/R/R-4.0.2/library/backports'
Warning in install.packages :
installation of package ‘backports’ had non-zero exit status
* installing *source* package 'rlang' ...
** package 'rlang' successfully unpacked and MD5 sums checked
ERROR: cannot remove earlier installation, is it in use?
* removing 'C:/Program Files/R/R-4.0.2/library/rlang'
* restoring previous 'C:/Program Files/R/R-4.0.2/library/rlang'
Warning in file.copy(lp, dirname(pkgdir), recursive = TRUE, copy.date = TRUE) :
problem copying C:\Program Files\R\R-4.0.2\library\00LOCK-rlang
lang\libs\x64
lang.dll to C:\Program Files\R\R-4.0.2\library
lang\libs\x64
lang.dll: Permission denied
Warning in install.packages :
installation of package ‘rlang’ had non-zero exit status
* installing *source* package 'vctrs' ...
** package 'vctrs' successfully unpacked and MD5 sums checked
ERROR: cannot remove earlier installation, is it in use?
* removing 'C:/Program Files/R/R-4.0.2/library/vctrs'
* restoring previous 'C:/Program Files/R/R-4.0.2/library/vctrs'
Warning in file.copy(lp, dirname(pkgdir), recursive = TRUE, copy.date = TRUE) :
problem copying C:\Program Files\R\R-4.0.2\library\00LOCK-vctrs\vctrs\libs\x64\vctrs.dll to C:\Program Files\R\R-4.0.2\library\vctrs\libs\x64\vctrs.dll: Permission denied
Warning in install.packages :
installation of package ‘vctrs’ had non-zero exit status
* installing *source* package 'yardstick' ...
** package 'yardstick' successfully unpacked and MD5 sums checked
** using staged installation
** libs
*** arch - i386
/mingw32/bin/gcc -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c init.c -o init.o
1 [main] make 498 dofork: child -1 - forked process 60864 died unexpectedly, retry 0, exit code 0xC0000142, errno 11
make: fork: Resource temporarily unavailable
ERROR: compilation failed for package 'yardstick'
* removing 'C:/Program Files/R/R-4.0.2/library/yardstick'
* restoring previous 'C:/Program Files/R/R-4.0.2/library/yardstick'
Warning in install.packages :
installation of package ‘yardstick’ had non-zero exit status
* installing *source* package 'tibble' ...
** package 'tibble' successfully unpacked and MD5 sums checked
ERROR: cannot remove earlier installation, is it in use?
* removing 'C:/Program Files/R/R-4.0.2/library/tibble'
* restoring previous 'C:/Program Files/R/R-4.0.2/library/tibble'
Warning in file.copy(lp, dirname(pkgdir), recursive = TRUE, copy.date = TRUE) :
problem copying C:\Program Files\R\R-4.0.2\library\00LOCK-tibble\tibble\libs\x64\tibble.dll to C:\Program Files\R\R-4.0.2\library\tibble\libs\x64\tibble.dll: Permission denied
Warning in install.packages :
installation of package ‘tibble’ had non-zero exit status
The* installing *source* package 'backports' ...
** package 'backports' successfully unpacked and MD5 sums checked
** using staged installation
** libs
*** arch - i386
/mingw32/bin/gcc -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c dotsElt.c -o dotsElt.o
0 [main] make 1030 dofork: child -1 - forked process 60900 died unexpectedly, retry 0, exit code 0xC0000142, errno 11
make: fork: Resource temporarily unavailable
ERROR: compilation failed for package 'backports'
* removing 'C:/Program Files/R/R-4.0.2/library/backports'
* restoring previous 'C:/Program Files/R/R-4.0.2/library/backports'
Warning in install.packages :
installation of package ‘backports’ had non-zero exit status
* installing *source* package 'rlang' ...
** package 'rlang' successfully unpacked and MD5 sums checked
ERROR: cannot remove earlier installation, is it in use?
* removing 'C:/Program Files/R/R-4.0.2/library/rlang'
* restoring previous 'C:/Program Files/R/R-4.0.2/library/rlang'
Warning in file.copy(lp, dirname(pkgdir), recursive = TRUE, copy.date = TRUE) :
problem copying C:\Program Files\R\R-4.0.2\library\00LOCK-rlang
lang\libs\x64
lang.dll to C:\Program Files\R\R-4.0.2\library
lang\libs\x64
lang.dll: Permission denied
Warning in install.packages :
installation of package ‘rlang’ had non-zero exit status
* installing *source* package 'vctrs' ...
** package 'vctrs' successfully unpacked and MD5 sums checked
ERROR: cannot remove earlier installation, is it in use?
* removing 'C:/Program Files/R/R-4.0.2/library/vctrs'
* restoring previous 'C:/Program Files/R/R-4.0.2/library/vctrs'
Warning in file.copy(lp, dirname(pkgdir), recursive = TRUE, copy.date = TRUE) :
problem copying C:\Program Files\R\R-4.0.2\library\00LOCK-vctrs\vctrs\libs\x64\vctrs.dll to C:\Program Files\R\R-4.0.2\library\vctrs\libs\x64\vctrs.dll: Permission denied
Warning in install.packages :
installation of package ‘vctrs’ had non-zero exit status
* installing *source* package 'yardstick' ...
** package 'yardstick' successfully unpacked and MD5 sums checked
** using staged installation
** libs
*** arch - i386
/mingw32/bin/gcc -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c init.c -o init.o
1 [main] make 498 dofork: child -1 - forked process 60864 died unexpectedly, retry 0, exit code 0xC0000142, errno 11
make: fork: Resource temporarily unavailable
ERROR: compilation failed for package 'yardstick'
* removing 'C:/Program Files/R/R-4.0.2/library/yardstick'
* restoring previous 'C:/Program Files/R/R-4.0.2/library/yardstick'
Warning in install.packages :
installation of package ‘yardstick’ had non-zero exit status
* installing *source* package 'tibble' ...
** package 'tibble' successfully unpacked and MD5 sums checked
ERROR: cannot remove earlier installation, is it in use?
* removing 'C:/Program Files/R/R-4.0.2/library/tibble'
* restoring previous 'C:/Program Files/R/R-4.0.2/library/tibble'
Warning in file.copy(lp, dirname(pkgdir), recursive = TRUE, copy.date = TRUE) :
problem copying C:\Program Files\R\R-4.0.2\library\00LOCK-tibble\tibble\libs\x64\tibble.dll to C:\Program Files\R\R-4.0.2\library\tibble\libs\x64\tibble.dll: Permission denied
Warning in install.packages :
installation of package ‘tibble’ had non-zero exit status
The
As always, an amazing class! Thanks!!
Great ML modelling usecases Julia, I’m wondering if tidymodels can be used along side sparklyr? It would be great to see an use case example using the spark engine.
Yes, in fact, Spark is one of the engines we support for models such as random forest, decision tree, logistic regression, etc. You can see how to set your engine to Spark in the README for parsnip:
parsnip.tidymodels.org/
Hey Julia, congrats. Your videos are super kool. May I ask something? Why you don't use the Boruta methodology for choose which variables are better for the model, instead choose them by yourself? I really like your way to programming.
Well, that algorithm can be great but is related to random forest variable importance, as I understand it, so you end up using your training data to decide what to include, when you run into "no free lunch": en.wikipedia.org/wiki/No_free_lunch_theorem
That doesn't mean it's not a good option to have but it's certainly not a good default.
@@JuliaSilge yeah...actually this message is not about this specific regression problem, but in classification problem in other videos saw where you did a decision tree. But it's ok. It was just a commentary. Glad for your attention, you are a careful person. Be safe.
Hi, what font do you use for R studio? it's nice
If you mean what font am I using in the IDE, I haven't picked anything fancy, just Monaco (maybe even the default?).
If you mean what fonts am I using for plotting in ggplot2, those are IBM Plex Sans and I made a little theme for it in this package:
github.com/juliasilge/silgelib/