วีดีโอ

While those schemes could give some improvement, if I was maybe training it with backprop, the biggest problem is that with a random model, the CFR model on top simply doesn't have informative enough features to do anything useful. It cannot learn to read the board, and just turns the information into mush. This isn't something that is surprising to me, back in 2021 I tried training a NN to learn the hand ranking function in a supervised manner and it couldn't do it perfectly. In the follow up videos, I try to get around the lack of feature learning using the cross-entropy method, and you'll see how it goes...

Back in 2021, I did manage to get an agent to train on flop poker using actor critic methods even though it just died on NL Holdem, so you are right that reducing the amount of game states would make the problem more tractable. But it's still very unsatisfying to get it to work like that. You don't want the training process to choke on something like NL Holdem, because if it's having trouble with a toy game like that, how will it handle the bigger games? It won't. And I don't want a path where I am putting in hack after hack to make it work well in a domain. I want to go beyond that. But I can't really do it, so in the end developing the Spiral language and getting better at programming becomes the goal of the exercise. Embarrassingly, my journey is one where I get better at programming, but don't get closer to the essence of intelligence. I think that won't hold forever and eventually programming and understanding of intelligence will become synonymous. Programming is a weak proxy to modifying one's own mind right now. Eventually, it will become a strong proxy, and it's only a matter of time until it stops being a proxy at all.

Puno hvala.

Thank you.

I used to think like that, but my brain and my heart are in disagreement. Even if the Nvidia competitors are destined to make an impact, it will take at least 3 years before it hits. I've been in this for a while and I've been looking forward to novel hardware for nearly a decade, which is why the performance of Nvidia's competitors is so disappointing. Anyway, the C++ backend is complete. You can already give it a try in Spiral v2.11. The screencast chronicling its creation will come out on the 28th July, in 4 days. Thanks for leaving a comment, I want more of those.

​@@markogrdinic5126 I really want to try out Spiral. Didn't yet find the time, but I am very much looking forward to it. I am still an amateurish programmer, so we will see how it goes :) This might be a far fetch, but please tell me, was the anime Gurren Lagann inspiration for the name? Because how the Spiral is presented there, as the thing which describes the exponential human progress, would also fit your ambitions well I think. And I noticed you used anime soundtracks in some videos :)

Sorry, it's been so long since I made this vid that I forgot what was in it, and I'd need to study it for a few hours to give you satisfactory answers. Currently, I am busy with making the RPS game work on the GPU in the Spiral series, so I don't have time for that. You'll have to put in some effort yourself in order to understand this, and if you manage it, please do answer your own questions here so that others might benefit from your insight. Good luck!

What is C? 😄Another lame slow language no one needs. Assembly and only Assembly is the fastest best programming language for that.

@@markogrdinic5126 Oh I see you do not have any clue. C is directly converted into machine code. Try it ! Assembler same but is hard to read too low level.

@@markogrdinic5126 real programer punch cards.

Missed another 0

I heard from another user who got temporarily banned for making too many API calls during the startup. I've never ran into such an error myself, but then again, I haven't been using Paperspace since the start of the year. If anybody knows how to get around this, please post it in the comments. Maybe Paperspace added something in order to rate limit the number of queries.

Even assuming you are right, just how many ways are there of implementing sorting using HLS C++? Yes, there are various algorithms for doing it, some of which are O(n^2) and others O(n * log n). And assuming you need to mess with pragmas, that would multiply the number of variations by some amount. Even if you couldn't manipulate the inlining of a library sort function, just why isn't there a library of numerous different kinds of sorting? > Do you want to sort millions of objects stored in off-chip SDRAM? > Do you want to sort half a dozen items in one clock cycle? > Do you want to sort a small number of items in a moderate number of cycles (say 100 items in 10 cycles)? > Do you want only the top X or bottom Y items in the data set? > Do you need predictable latency? > What are you trying to optimize for? throughput? latency? chip area? power? > All give you a very different hardware design Why not all of that? That is what libraries are for, so the users do not have to waste their time implementing them themselves. Sorting isn't that trivial, and certainly something you wouldn't attempt to do in a few minutes, not if you wanted to do it correctly. I admit, some of my assumptions might be broken. I am going to have to study how the C++ code I've been compiling runs at the hardware level because I have no idea how passing pointers to arrays into functions could work otherwise. I know that functions get converted to state machines using FIFO/PIPOs, but being able to pass pointers in that regime makes less sense, not more. Don't tell me they compile it to some kind of chain and pass data from FIFO to FIFO? But that burden of learning is something I am going to have to take on. It doesn't excuse people in the community telling me to implement bubble and insertion sort, and getting upvoted by others.

@@markogrdinic5126 Hi! The language is not the issue, it's the approach. For an effective HDL design a lot more design work needs to be done up front. Say we wanted logic to perform a simplified poker ranking, for highest card, pair, two pair, three of a kind, four of a kind and full house. One possible hardware designed approach: The input: 6 bits for each card - 2-bits for suit and 4 bits for value, so 30 bits in total for a 5-card hand. The output: A 15-bit vector, consisting of - a pair flag - a two pair flag - a three of a kind flag - a four of a kind flag - a full house - The 5-bit value for the highest card or first set of cards - The 5-bit value for the next set of cards (or zero). If memory was no constraint, we could just look up a value in a table with 2^30 entries and get the rack almost instantly. However, it is a constraint - large memories take many cycles to access, and expensive (in terms of silicon, power, I/O and board space).- so a better solution is needed. here's one possible way. First, compare each card with each other in the hand. This will need ten 4-bit comparators, and generate a 10-bit value. That value can then be used to index a 1024-entry lookup table. The rows of this table consist of: - A bit that indicates if it a one pair - A bit that indicates if it is two pairs - A bit that indicates if it is three of a kind - A bit that indicates if it is four of a kind - A bit that indicates if it is a full house - A "sort highest values" flag - A five-bit mask for the first set of cards to find the highest value - A five-bit mask for the second set of cards to find the highest value So that's a 16-bit x 1024-entry read-only memory - or one 'Block RAM' in a common FPGA. With the output of the table lookup, then highest value in the two masked off sets of cards can be determined, and if required the output of those two values can be sorted if the the "sort highest values" flag is set. It requires four 6-bit comparisons to find the highest value in each set of cards, and another comparison that is needed to rank two pairs properly. You then end up with a blob of logic that takes as input a 30-bit 'poker hand' and returns 15-bit 'rankings' a small but fixed number of cycles later (maybe two or three cycles, depending on target clock rate). Throughput could be one hand each clock cycle. You could easily implement in a HLS, it would look radically different from the usual software-oriented solution, mostly because all the time consuming work of working out which cards have matching face values has been pre-calculated in generating the lookup table. Looking back over the design gives an estimate for the FPGA resources required - maybe 150 LUT6s, one Block RAM and 45 flipflops, or maybe 100 flipflops if pipelining was required.

​@@hamsterworks > maybe 150 LUT6s, one Block RAM and 45 flipflops, or maybe 100 flipflops if pipelining was required. Huh, really? Lol, when I compiled my hand ranker it was close to 20k LUTs. Could you really squeeze it down that much if you used a lookup table? I haven't really thought about it. But all the loops needed to match the cards, as well as do it in parallel are likely to take up hardware, and the kinds of LUTs I am getting don't feel unreasonable given the quantity of the outputted code even though it would be like 2% of an FPGA. > You then end up with a blob of logic that takes as input a 30-bit 'poker hand' and returns 15-bit 'rankings' a small but fixed number of cycles later (maybe two or three cycles, depending on target clock rate). Throughput could be one hand each clock cycle. That'd be pretty awesome performance, way better than in my implementation. I am interested in learning more. Let me think about what you are suggesting... > First, compare each card with each other in the hand. This will need ten 4-bit comparators, and generate a 10-bit value. That value can then be used to index a 1024-entry lookup table. The rows of this table consist of: ``` int c = 0; for (int i=0; i < num_cards; i++) { for (int j=i+1; j < num_cards; j++) { table[c++] = compare_card_ranks(i,j) } } ``` So assuming 5 cards per hand, that would be 4+3+2+1 = 10 loop iterations. We'd be comparing the ranks here which take 4 bits. And that 10 bit value is an array of booleans for each comparison. ...Which we can use to check for pair type hands. What about the straight flushes, flushes and straights? Also, I am sorry to be so misleading in my Reddit post, but I am not just sorting 5 card hands, at the time I wrote the post I was running into trouble, so I used 5 cards per hand to make it easier to debug them. The game I have in mind is NL Holdem which uses 7 cards. That one would require 6+5+4+3+2+1 = 21, a 16-bit * 2^21 table for the pair style table. And in addition to considering the straights and flushes, I'd also want to get back the actual hand, so I can compare it with the opponents. The hand ranker I wrote would work for getting the top hand from an arbitrary sized array of cards. ...Let me think a bit more. If we wanted to take this same approach to detect straight type hands, we'd need to sort them first. This is what motivated my question on the /r/FPGA sub. Flushes are easy to count. For straight flushes, we'd need to make a separate table that also considers the suits. The troublesome thing about them is that each of the possible suits would need to be considered individually, so number of comparators would be 4x that it would be for a straight. Also, I am sorry to heap requirements, but I thought it would be neat to have a hand ranker that would also take care of Omaha, which has 9 cards. At that point the approach would break. With more than 5 cards, straights become troublesome as it is no longer enough to just sort the hand, we need to skip over the pairs in a sequence. All of that makes a hand ranker complex. --- I know there are some sophisticated hand rankers which use hashing, but those do require long tables, much like the ones we are considering here. I was thinking that maybe I'd want to reuse this code on future AI hardware, so it wouldn't be good to use large memories where logic would suffice. The hashing based approaches would be a lot harder to implement than in just two days like I did for the hand ranker here. Actually, it took me a few days longer than that partly because I had to work on the language and the HLS C++ backend for Spiral while at the same time implementing it.

@@markogrdinic5126 If you were working with 5 cards, and you were to sort them before ranking you could make things simpler. Just compare for equality with the neighbour, to give a four bit vector So the hand "2 2 9 K K" would give "1 0 0 1". The lookup table for "1001" would indicate that it is two pair, taking card 2nd and 5th as the highest card in each pair. Likewise "2 4 4 4 4" would give "0 1 1 1" and that entry would indicate that it is four of a kind, with card #5 begin the high card There's only 16 different options in the lookup table, so it is much more compact, small enough to be implemented using just LUTS rather than block RAM

Oh, no. It would not make sense to run it in a gradient notebook, since that script is what is used to start it to begin with. Were you the one who opened that issue on the repo? If you told me what kind of error you are getting maybe I could help you. You really shouldn't be having any trouble with running `pip install gradient` from the terminal. Maybe your Python installation isn't in PATH? Can you run the regular Python interpreter? For this kind of issue, maybe you could also ask Bing Chat to help you.

yeah, i have issue to run 'pip install gradient', I did use Ananconda, Window powershell and command prompt but no luck. I followed Bard, Claude and chatGPT instructions to locate the PATH, and it ends up with error: :AttributeError: cython_sources'... Getting requirements to build wheel did not run successfully." I did install Visual C++ , install cython, but in the end it still show the same error again. I did use different pc to try. May be I miss something in python. But thank you very much for your reply :) @@markogrdinic5126

Sorry, I haven't used PS in a while, I won't be able to resolve this issue for you. If the setup script is out of date, you are going to have to ask around.

No, but I wouldn't even think of using something like ChatGPT to generate Fable bindings for Typescript libraries. AI models are good for things where you can be loose like generating images with SD, or pointing me to the right documentation like Bing, or lately, helping me write science fiction like ChatGPT. I think that current models are just too weak for a task that requires deep reasoning like translating between incompatible type systems. Also, `.fsi` files are F# type signature files that are typically generated from regular `.fs` files. They are similar, but just contain types and not code, and tend to be used for documentation. The actual Fable bindings for Typescript libraries have to be placed in `.fs` files. I hope I got the intent behind your question right.

That is great. I am glad somebody found this useful.

The problem with transcribing TS's types to F# is that TS has a lot more powerful type system, and a lot of what would be expressible in it, simply isn't in F#. All of type level mappings, type literals, type conditionals, nested variadic args. Moreover, F#'s (.NET's) and JS's object systems are different. You'll always be faced with incompatibilities in such a situation, and TS which was made to be a superset of JS will always have an edge when interacting with it. Currently, I am trying out Bolero & Blazor, so depending on how that turns out, it might be better to use those for web dev with F#. I need to spend some time using them before I decide whether I want to recommend it or not. I'd recommend F# in general as it is a lot easier to use (and a better language) than TS, but after experiencing Fable, but as far as webdev is concerned, I am not sure whether it would be worth the users time to use it over TS for their own projects. Maybe if you are an experienced webdev who is familiar with the libraries on the TS side, you could get use out of Fable, but for somebody coming in, definitely not. Bolero might be worth a try.

@@markogrdinic5126 Thank you for your detailed reply. I'll check out your video on Bolero. I don't mainly do webdev, but still, I need to do work on the frontend sometimes, and since I do programming as a hobby, I like using languages that feel elegant to me

github.com/invoke-ai/InvokeAI#command-line-installation-for-users-familiar-with-terminals My guess would be yes, but I've never used it before. It has a web server, but I am not sure how it could be exposed publicly. You'd have to follow the instruction here and figure out if there is a setting to do that. The best way if you can't find it after some searching, is to open an issue in the relevant repo asking how to do it. Sorry, I am busy with other projects so I cannot make the time to make a tutorial on how to do it right now.

@@markogrdinic5126 Thanks for the reply. It is hard for me to figure it out. If you have the time in the future and create it i will be really thankful. Please if you do reply to this message.

I've never tried it before, sorry. But I recall that you can setup Linux so it runs shell scripts during startup by editing some config files, you'll have to look for that on your own. What do you want them to do? Right now my focus is on webdev, but I'll get back and do more AI art stuff before long. I'll merge my various interests into one, and make some content that might be of interest to a wider audience, so don't hesitate to give me suggestions on what you'd want to see.

A few minutes unfortunately. It is not a problem if you want to prompt for a while, but if you want do a few quick pics it is a pain in the ass.

@@markogrdinic5126 usually it takes 10 minutes on google colab. Of paperspace is quicker than that. It's great

Hvala.

@@markogrdinic5126 može neki kontakt il nešto imao bih par amaterskih pitanja

@@EasyWay-M.I.S. Nema problema. Moj email možeš naći na mojem Github profilu. Username mi je mrakgr.

I'll keep your request in mind, though I do not have any plans to do more vids in this playlist for the foreseeable future. One thing I'd really like to do is change the Automatic1111 WebUI to something else, as I tried it yesterday and it was as buggy and unstable as usual, which irks me to no end. I'd really like to try out ComfyUI, but my interest right now is on programming. At some point though, I might return to doing visual novels, and then I'll do more Stable Diffusion videos. In the future, I'd like to do a playlist on how to do a visual novel as a web application, which would feature using generative AIs to assist writing, audio, voice and art.