APL vs BQN vs J vs Q vs NumPy vs Julia vs R
ฝัง
- เผยแพร่เมื่อ 16 ก.ย. 2022
- A comparison of seven different programming languages across a few different language criteria.
Content Repo: github.com/codereport/Content...
Array Language Comparison Repo: github.com/codereport/array-l...
ArrayCast Podcast: www.arraycast.com/
ADSP Podcast: adspthepodcast.com/
Programming Language Virtual Meetup: www.meetup.com/programming-la...
Follow me on Github: github.com/codereport
Follow me on Twitter: / code_report
Follow me on LinkedIn: / codereport
As someone who has moved from C++ and traditional Python to Rust, I've really grown to appreciate the expressiveness of non-imperative styles of programming. Learning about array-style and algorithm-centric programming has greatly improved my ability to reason about certain problems. My initial solution for this problem used just chained iterators. Six months ago, I would have written that nested for loop solution. Fantastic talk!
What's easier to write software in? C++, Python or Rust?
@@vectoralphaSec for all but the shortest programs/scripts, Rust is the easiest and best language I have used by almost every measure. I will sound hyperbolic, but learning Rust was world-changing for me.
@@vectoralphaSecdepends on what you want to write, but python is undoubtably the easiest of the three you mentioned
@@multivitamin7also the shittiest😂
@@sirajmussafirr147 Again. Entirely depends on what you're wanting to write. Not everything calls for what Rust and C++ provide. There are definitely instances where Python is the better tool for the job than C++ and Rust. Stop with the elitist nonsense.
More videos using Julia please!
Great content :)
Seconded! I've done a couple of projects in Julia, but I'm never entirely sure I'm using it to its full potential.
I gave you a like just for your intro; so many people just list all that stuff but man putting a timeline and visuals to it really painted a clear picture of your experience and journey. Kudos.
absolutely brilliant talk. I learned APL fluently 40 years ago and then saw it perish, being replaced by languages with 5% of the functionality, always wondering why nobody gave a damn....
In Julia size() will return a tuple with sizes of all dimensions.
Even better the second time :)
Can't wait for a combinators video and would love to see more content about (all the different) polymorphisms
I appreciated the line of code in the languages, and it was some smart highlighting of the similarities. Thanks for putting this together!
MATLAB being an array language makes a lot of sense looking back towards my time in college. None of the classes would preface it with that, so in my (thankfully) limited interactions with it I’d simply approach it wrong. Like it was some kind of imperative language.
Yeah it's pretty important to know that because it's very slow if you don't vectorize. That's why I love Julia because you can write whatever style you want and It works well across the board
For R you can take advantage of coercion to logical matrices. Examples:
check_matrix
Great talk, Conor! BTW APL solution also works if you move the fork after the selfie and should be faster. Then change max to min for clarity.
looking forward to the combinatory logic video! :)
Thank you very much for your videos. It would be great to see videos showing how to use APL in simpler actions such as sorting of texts and other basic operations (an APL tutorial and tutorials on other similar programming languages). It will be great to hear your opinion about which programming language to learn out of these choices: APL, BQN, J, Q, Kx, Julia, R, and other similar ones. Why so? because J and some others apparently improve on some things that people do not like about APL such as the weird symbols and because maybe array programming languages are apparently mainly used for arrays/matrices and alike. What do you think of the Pure programming language and how it compares do the others I mentioned?. Which free array programming languages you recommend to learn and are they also recommended for regular programming work (not mainly for matrices and alike)?
From an intermediates perspective I would say that the readability of the Julia, R, NumPy and C++ implementations are much much better than the combinatorics format. Additionally, while the code has been reduced to one line, what is happening behind the scenes, what intermediates are being created by the compiler? This is why I prefer the C++ implementation, it is much clearer what the code is doing and what additional variables are being created. Additionally with the C++ solution you can eliminate the branching statements and allow for SIMD optimizations.
Please more J stuff, especially at the intermediate level even beginner level. Please. Loved this.
In Numpy at 28:38 you can also do outer product just like in Julia at 33:10 by first transposing the array: i[:,None] == i
This is generic, I think, which means you can run arbitrary functions like foo(i[:,None], i)
It's not really a transpose, but yes this is a seriously powerful tool in Numpy. It's supported by any ufunc, and you can construct arbitrarily complicated outer products: f(x[None,None,:], y [None,:,None], z[:,None,None])
Looking forward to the video on combinators. I understand the concept, but I just don’t know them all. And I certainly don’t remember their bird/letter names.
When there’s a visualization for the combinator, I find it really easy to remember what they do
Maybe I should try to write a web app that takes a chain of combinators and arguments and creates a graph (or DAG). I could just output in graphviz format, like what CMake does for your dependency tree
I don’t know how to do this part, but it would be cool to have a keyboard like Dyalog has for all the symbols - but the symbols are all the combinators.
And then this is even more ambitious but if you could toggle the languages, and have your combinators symbol change (which by default I would represent with their single letter form).
Could support letter form, bird image form, APL, BQN, etc. ofc some of the languages won’t work for a given piece of code because of nonexistent combinators. But if the system could replace the non-existent combinators with phrases that create said combinators… that would work
12:34 this is soooo super awesome talk i am soo liking this coecept of dimain specific languages
FYI, the function does not work with negative elements:
using LinearAlgebra
function check_matrix(grid)
i = size(grid, 1) |> I |> Matrix
return min.(grid, 1) == max.(i, reverse(i, dims=1))
end
x = [1 0 0 2; 0 2 -3 0; 0 5 5 0 ; 10 0 0 -2]
check_matrix(x) # returns false
Nice talk, Conor🤩. Sad to see cpp solution being more vebose 😪. I am CUDA/C++ programmer.
Are the elements in the diagonals are assumed to positive? If those were negative the final min will not yield 1s along the diagonals. Could you please clarify whether my understand is right?
Imagine getting 200k lines of BQN code that they tell you to find a bug in.
Finding a bug in 200k lines of any language is not a simple TASK also BQN will save you from those many lines
This must be exactly the same as asking you to find a bug in equivalent 5M lines of Python or 20M lines of Java.
these are the kind of ppl who say "imagine writing string parser in SQL"
hipsters of code
.2M lines of BQN is an OS.
12:21 Maybe you should use fmap (functor map):
fmap (min 1) x
Sure, but then he'll also want to use an array rather than a list of lists.
why not compare the solutions in terms of performance and memory usage?
Please include R in more of your videos!! 🤓🤓🤓🤓
Julia can be written
checkmatrix(grid) =min. (grid, 1)==max.(1:size(grid,1).==1:size(grid,1)|>transpose, reverse(1:size(grid,1)|>transpose, dims=1))
So, I recently made a cli calculator program, something small I can use and not get unfocused like I do w/ a GUI calc or something. I think I went overboard though.
I implemented lists and functions, so I could handle a bit more complexity when necessary, but now, after watching this video, it's got me thinking.
I may have unintentionally implemented tabling and rank polymorphism with how I handled operations between variables.
So, I may have accidentally created a tiny array programming language while making a calculator.
Just curious, were you against just using a language that came with a REPL, which is essentially a fancy CLI calculator?
@@lunalect Yeah
Basically, the idea is that I don't usually need a calculator for the things I do, so I don't have something open.
Anything other than just immediately typing an expression including opening a GUI app, starting up a REPL, using weird syntax, etc takes me out of my groove. I wanted it to just be something like open terminal, type "c 2+2", and then get back to work.
Basically I wanted a calculator sitting on my desk ready to go without having to have a calculator sitting on my desk. It's a very specific set of restrictions, but it's what I needed.
I did still allow it to be used in a REPL (bc why not?), but it doesn't have to be
I think you reinvented `bc`
Wow that may be exactly what I need! Thanks!
I don't think it can do complex numbers by default, but I don't have to use them very often anyway.
There's a GNU bc, and it's got a POSIX standard. Why the heck is it not in the coreutils???
Cool video
Nice solution. Here is the tacit form of J solution:
(1&.|.)&=/~&i.&#)
However, the question does not state that the numbers that are in the array are larger than zero.
(0&~:-:(>.|.)&=/~&i.&#) // Replaced 1&.|.)&=/~&i.&#)
For languages that don't have combinators built-in, couldn't your create functions that do that? Like this: `def forkCombinator(unary_fn1, unary_fn, binary_fn): ...`
7:44 what if there are negative elements?
His solution is incomplete
Great talk, as always.
One observation.
I see the reverse columns glyph as a horizontal mirror. The result has mirror symmetry if you imagine putting a horizontal line below/above the matrix acted on.
Similarly the reverse rows looks like a vertical mirror. The reversed rows have a vertical mirror symmetry to the matrix you get when this operation is acted on.
At least this is how I remember which is which.
ps: besides the APL, J, BQN solutions I really like Julia's. Eat that, Python!
I know Dyalog APL has user defined operators, do any of these other languages have support for them?
Yes, you can define all kinds of operators in BQN and J.
I'm new to all this... Can you download BQN to your computer like with J and Dyalog APL?
I prefer BQNPAD but you can definitely download it. There is a useful page on all the ways to run BQN here: mlochbaum.github.io/BQN/running.html
26:32 awesome venn diagram of languages based on: array, iversonian, combinator, functional;;
5:35 , 5:45 | 6:29 , 7:16 | 16:46 this combinator logic is superrr awesome.
similar happens sufficient times in normal non-array situations as well, where i am checking a thing for aome condition and then using that again in the body...
and yeah, its super irritating there as well
I'm more excited for composition intuition than I have ever been for any movie
Wonder how's the Composition Intuition video coming along. 👀
Would've loved to see MATLAB compared with them
A Q solution without any temporaries:
{(0x)~{x=/:x}{x&reverse x}til count x}
why not k?
{(0x)~{x=/:x}{x&|x}@!# x}
@@nanthilrodriguez Because it requres no skill to rewrite it in k4 but most ppl wouldn't understand. :)
Superb! More like this. And what is the justification for using Q? It is a proprietary language owned by KX and it looks pretty ugly to me.
Q (and KDB+, the database that comes with it) is widely used in the finance industry. If you look at github.com/interregna/arraylanguage-companies, you will notice there are 25 companies you Q/K which is more than the 21 using APL. I know it is not an exhaustive list, but Q/KDB+ might be the most widely used array language in the world. Therefore worth studying IMO.
20:50 > _"now rhat would u expect"_
based on the name "leading axis th" & teachings of linear algebra; i would guess that being smaller cardinality, the leading axis here would be column
21:10 i was right :D
19:40 what's up with the empty node?
There is no combinator in BQN that corresponds to that combinator
@@code_report ohh lol i was expecting you to say something about the absence in the video, i was like, trying to work it out lol. Great, thank you
old video, but just wanted to comment that julia's broadcast operator will (most of the time) compile down to SIMD instructions (it's one of the main reasons it was put in the language in the first place) and for all that weirdness, i'll happily take the performance boost
5:22 .= that seems like the fxn broadcast from julia (:
More videos on Julia please! Tutorial videos from beginner to advanced and implementation of problem solving or projects
dude you gotta include ocaml in these :)
Hey bud, you should think about getting a pop filter for that mic...
That's not the equivalent in Haskell for the element-wise scalar minimum! First off you've got a list of lists and not a matrix, a matrix would have a functor instance that you can use directly. Secondly, a linear algebra library like you have installed with your array languages would have an Object type that can be a scalar or a matrix and that would have an PartialOrd instance so that '1 `pmin` mymat' would work just fine. You might say that you get free conversion in those array languages for when you try to use functions from elsewhere but you shouldn't and if you really want it you can cheaply, with flexible instances, define a class with all your array operations for all sorts of mismatched types as well as matched types or you can use typereflection and type families and all sorts.
Do you have to write code to allow each pair of types to interface? Does a rank N matrix act like a rank M matrix? Can I use the same code to apply a scalar to either one of them.
A list of lists is inherently rank 2. The strictly typed divide between those types seems needless, in comparison to the array languages.
Of course, this is a very Array-Language oriented perspective :)
@@arisweedler4703 yes, yes, and yes up to polymorphism. It is possible for someone to write an interface that offers what the array languages offer then you need only download that just as you download your array language but we already learned that it goes wrong and obstructs one a lot so we can ask of the array languages, does it save me when everything is the same as everything else?
@@tricky778 > but we already learned that it goes wrong and obstructs one a lot
> does it save me when everything is the same as everything else?
sorry, I'm just not following you. what's going wrong? Your point was that if you have the right Haskell libraries, then Haskell is as expressive and easy to rank polymorphize as APL. Right?
> download that just as you download your array language
This is uncharitable. None of these are exactly the same: Base language, standard library, popular 3rd party library, existing but not common library, possible-to-write-but-not-written-yet library.
Incorrect. Array languages don't have "linear algebra libraries". These functions he's using are primitives, as in native to the language notation, and not special libraries. It supports linear algebra out of the box because it is an array notation.
@@nanthilrodriguez why is that fact important?
29:45 for impure languages it's probably difficult to decide when it's safe to memoise the result of a call and avoid calling the same function twice with the same arguments. Referential transparency and all that.
numPy is usable by Julia, feels a bit weird to include it as it's own thing
Fork in APL but hook in J :) Of course it could be a fork in J too, just one char longer. And I guess, even more "J-ish" way would be to use a tacit definition like check_matrix =. 1&.|.)[:=/~ i.&#
33:16 , 33:28 transposing a vector increases rank in julia is broken
interesting.
When I first saw the behaviour I was surprised, but now I'm quite happy with it. Transposing really just permutes dimensions 1 and 2, and it only works when dim
It is obviously a linear algebra oriented decision. If a person does not have exposure in higher level linear algebra, he may not aware of the generalization, thus thinking it is broken or surprised...
15:05 reduce, scan, outer product
and guido rossem said rather than reduce, u r better off of writing as for loop lol
Julia is based, and ur based for showing it
@code_report what did you use to create the presentation?
I have a video coming out in a few days that is a behinds the scenes of how I make my slide decks.
@@code_report that would be very much appreciated!
def is_X_matrix(m):
return ~ np.any(((I := np.identity(m.shape[0], dtype=bool))
| np.flip(I, 0))
^ m.astype(bool))
Around 8:24 - I wonder whether × could have been used instead of 1⌊. Does it break down for some inputs?
0 = 1 × 0
If you multiply the 2 matrices together, of course the diagonals will be equal, but if the diagonal contains a 0... then it will match in the multiplied matrix. But the rule stipulates that there can be no 0's in the diagonals.
Great video! Though one thing that bothered me was that that 1⌊ should really have been a 0≠
both for readability reasons and because 1⌊ is plain wrong if your matrix has numbers less than 1 that aren't 0 (like 0.5 or -1)