Yeah, I suppose you could, since you can interpret the fraction of number of neighbors from a given class over k (the number of neighbors) as the posterior probability, p(class=c | features).
Sebastian Raschka I haven’t checked the literature yet. I have no idea how efficient would it be. It’s super efficient for our BMA model search algorithm. Let me do some research, this might be a good class project or even a paper if it hasn’t been done yet :)
Technically, in its simplest form, it doesn't. However, if you consider more efficient variants that use a ball-tree or kd-tree data structure to sub-partition the data for faster look-ups, then creating this data structure can be considered a form or training.
Dear Sebastian, I think there is a typo in the notes (sebastianraschka.com/pdf/lecture-notes/stat451fs20/02-knn__notes.pdf) in page 11. It says complexity for KNN using priority queues is O(k*log(n)), while I guess it should be O(n*log(k)) according to your video lecture.
Thanks for the note. It's been some time, but I think I updated the lecture notes later after making the video -- I think O(k*log(n)) is the correct one
23:49 can we use Occam's window? We use it for choosing the models with higher posterior probabilities on the Bayesian Model Averaging.
Yeah, I suppose you could, since you can interpret the fraction of number of neighbors from a given class over k (the number of neighbors) as the posterior probability, p(class=c | features).
Sebastian Raschka
I haven’t checked the literature yet. I have no idea how efficient would it be. It’s super efficient for our BMA model search algorithm. Let me do some research, this might be a good class project or even a paper if it hasn’t been done yet :)
Does knn need training? From the algorithm, i dont think knn needs training before prediction
Technically, in its simplest form, it doesn't. However, if you consider more efficient variants that use a ball-tree or kd-tree data structure to sub-partition the data for faster look-ups, then creating this data structure can be considered a form or training.
Dear Sebastian,
I think there is a typo in the notes (sebastianraschka.com/pdf/lecture-notes/stat451fs20/02-knn__notes.pdf) in page 11. It says complexity for KNN using priority queues is O(k*log(n)), while I guess it should be O(n*log(k)) according to your video lecture.
Thanks for the note. It's been some time, but I think I updated the lecture notes later after making the video -- I think O(k*log(n)) is the correct one
@@SebastianRaschka was not it n*log k since priority queue has k elements at a time