“Act On Press” - John Carmack's Hot Take On UI
ฝัง
- เผยแพร่เมื่อ 22 พ.ค. 2024
- This John Carmack tweet got me thinking. Like, REALLY thinking. onMouseDown might be my goto in the future
SOURCE
/ 1787850053912064005
Check out my Twitch, Twitter, Discord more at t3.gg
S/O Ph4se0n3 for the awesome edit 🙏 - วิทยาศาสตร์และเทคโนโลยี
My video sparked debate on Twitter, and John Carmack left some (really good) comments. Also pointed out that Chrome changes tabs on press, not on "click".
"To perform an action with act-on-release you need to both hit the button, then keep the pointer on target until you perform the release action. If you have a jittery point location, as you do with hand tracking, controllers held away from the body, or old age, this can result in many fail-to-activate cases.
You also want generously sized and spaced buttons, but that is an orthogonal effect."
It's up to the programmer to pick a solution and then be responsible for the fail cases that result from his/her choices. When it comes to the 'on press' vs 'on release' choice, if you had to pick one, I think using 'on press' and emulating 'on release' with some additional code is easier than the reverse. It's hard to fake true responsiveness, where as slow error-compensation cases can be more effectively managed due to the inherent time buffers available.
I wonder if, at some point, such functional choices might be offered to users, much like light/dark mode is offered today. This could emulate the preferences offered to gamers, where they can toggle between shoot on press versus shoot on release. Allowing users to choose their preferred interaction style could enhance user experience by catering to different preferences and needs, ultimately leading to more versatile and adaptable software solutions.
100% agree. This is very noticeable in menus in VR where you try to click a small button and have to put a lot of effort and focus into keeping the cursor within the bounds of the box for the entire interaction. You feel a noticeable slowdown as you have to steady your hand, stop moving your body, and pay attention to the buttons rather than the content.
"act-on-click/release" is only really valid with a stable cursor and that's antithetical to the VR experience and intent. VR is meant to bring the UI into your world, not pull you out of it to play "aim at the box simulator".
My Firefox on Windows changes tab on mouse press too. Interesting. Don't know why tab changes are on press while other things are generally on release. Although there is a disadvantage to act on press for tabs, sometimes I don't want to open a tab, I want to drag it somewhere else.
I've implement all my kiosk software with onmousedown/ontouch as people otherwise complain that their text doesn't appear when they click a button and/or keep pressing it and it is faster/crispier to multiclick on increment/decrement buttons etc. Also with elderly people, they don't understand even the concept of long press or double click, so it is much safer to just support single click and do it on mousedown/touch events.
Tabs react on Press because there is no probability of regret. We don't submit forms / pay cash by changing tabs. In HTML however, buttons often trigger payments, likes, follows and other sensitive actions that the user may want to cancel mid-way by releasing outside. This is also why most actions on your desktop OS act on Press - while opening applications on desktop require double-click because you could open them accidentally.
Since I was the person that implemented the original OpenGL based keyboard on Quest (originally GearVR), I thought I’d provide some additional context.
John is correct wrt static UI, but I was also one of the people pushing back on his requests to apply this more broadly (such as to nearly everything). Although, I was not the person requesting a study, as the keyboard case was obvious to me ONLY because a text keyboard can easily roll-back characters if you end up holding the key (try holding “n” on the stock iphone keyboard and also note they do act on release).
The main issue is still how often buttons are in scrollable UI. At one point I sent him several screenshots with all scrollable containers tinted red (most of the screen) to illustrate how common it was.
In this video, the impression was given that scrollable containers aren’t a concern, but that was only because the mobile browser ignored your request to act immediately on press/down, and delayed invoking your handler until it could determine you weren’t scrolling. Thus you also weren’t getting the full latency improvements John was advocating for.
In the end, there is some validity to John’s argument but it’s not as simple as just switching things over to press/down events.
* UI frameworks would need to be updated to do a delayed down event while detecting intent-to-scroll
* not-currently-scrollable containers are more common that never-scrollable so they could certainly benefit opportunistically at the cost of minor inconsistency
* any press-and-hold interfaces (think context menus) would need to be replaced or support roll-back logic.
* any drag & drop UI would need similar intent-to-drag delays
Also note - on mobile, intent-to-scroll (or drag) is significantly easier/better than in VR because a touch digitizer samples input at 300-1000hz vs 72-120hz in VR for controllers and significantly less for hand tracking. The precision of your finger is also higher on mobile vs. VR controllers and esp. hand tracking (requiring bigger deadzones and/or delays).
So there you have it - it’s not like the eng at Meta were dumb and didn’t want to have latency improvements, it’s just that we actually have to implement/support all this along with all the edge cases and it’s competing with a million other requests. If it was as trivial as just replacing click with onmousedown, we would have done that years ago.
It seems ideal to act on press and then roll back if the action was unintended for a lot of stuff. Of course, this is not straightforward to implement with our current frameworks.
I hate mobile, its ruining everything! it is the source of the enshittification of everything on and related to the internet.
Thanks for writing it down
@@niamhleeson3522 That is easy to say, but determining if the action was unintended packs a lot of challenges. Even if you define a good enough heuristics to mark the event "unintended" then you can't reliably ensure that this tell of the lack of intention wasn't itself unintended (like few pixels of drag when you only wanted to click). Plus doing all that may take more time then you're trying to save. And the problem isn't necessarily connected to the state of current frameworks. It's just due to human imprecision.
Well... Yeah, mobile browser *is* supposed to delay that. Mobile apps aren't as versatile as desktop ones, so they need to encode different inputs in different ways of tapping on screen. And that's their problem to deal with - on how to treat screen taps as mouse events.
The whole "undoing" a click is HUGE for me, and I'd hate to see it go...
Why though? If you type a command and hit Enter you can't undo, and it's ok. When dialog is closed on Escape press is ok. But closing with mouse on press is not ok?
@@w01dnick Its like keeping your hand on a chess piece even after you've made your move- just in case you re-think it at the last second.
@@Rushil69420 It's just habit. You can't undo Escape/Enter/etc and it's ok.
I so get it, I clicked the like button on your comment and just moved my cursor away without clicking it, it feels so good to have this power
@@w01dnick Right because that irreversibility is associated with mechanical objects that we interact with- like switches and buttons- but not with our hands. We can put down anything we pick up. We can hover our finger over a button and decide not to press it. We can even touch a button and then remove our hand from it. Moving the cursor on screen is more analogous to the physical interactions we have and initiate with our bodies than key-presses are.
HTMX supports "fetch on press, show on release", which massively cuts back the time it takes to navigate between webpages. It also supports "fetch on hover", which saves even more time, because it usually takes ~100ms before you press after a hover. It does risk fetching too much, like when you quickly swipe your cursor across a menu, but I believe they thought of that too and solved it for most cases.
Very clever approach for stateless/non-destructive updates
This is just a basic preloading strategy. HTMX didn't invent anything.
@@hunterxgthey don’t invent anything new hear, but made it trivial to do something that a neglected by most developers.
Its there in all modern frameworks (Remix, NextJS, Solid etc). Not HTMX specific.
No. Don't just do this everywhere. Understand your game/app. Then apply it where it's needed.
Assistive technologies have far better support for the click event than the mousedown event. The click event also includes handlers for keyboard interactions with the space bar or the return key. Please don't forget about accessibility!
Honestly your comment needs to be pinned. Mousedown is not a suitable alternative for click, and suggesting it without further considerations is harmful for accessibility. It also breaks the workflow of heavy keyboard users and tools like Vimium
Just bind both
Came here to say this. As a huge proponent for accessibility, I feel uncomfortable viewing controversial opinions like Carmack's. Oftentimes, you only find what you're looking for, and nothing else. In this, Theo found the responsiveness to be better because that's all he was looking for.
@@hugh5356 exactly.. there is nothing wrong with binding both and cancelling the click event after a mousedown.
@@GottZ Good idea! onMouseDown and onClick together, if onMouseDown is triggered then cancel the onClick. How would you cancel the onClick in that case?
And then there's WCAG, Success Criterion 2.5.2 Pointer Cancellation
If the change can be reverted then it is fine to do it on the down event. That article mentions it in the "Abort or Undo" section.
Although... no, it says it should be on the up event still... then I guess not.
Thank you for the rule drop, I learned something new
@CottidaeSEA WCAG phrasing can be confusing. This SC states that mousedown is only acceptable when it is "essential", i.e. when mouseup is needed for something else. The part about "abort or undo" applies only to mouseup events, basically saying that dragging the mouse off the button and releasing (like the click event) is preferred.
@@joostkersjes4349 Yup, it can be very confusing. As others have written in this thread there is however clear advantages for ALL to be able to cancel accidental clicks
Thank you for citing the standard. This 100%
I think it's better to trigger on mouse up most of the time due to the user being able to cancel accidental clicks.
Makes additional sense on mobile/touch screens where a long press is often required to achieve hovering.
A lot of interfaces have things where there's hover effects for info popups or similar, but we lose that on mobile/touchscreens if you active on down rather than up.
Yup.
Mobile: mouse up
Desktop: mouse down
Just out of curiosity, how do you get "accidental clicks" and do you also get them when handling guns?
I wonder if it's feasible to start doing work on mouse down, but only actually commit to it on mouse up. For example, if a user clicks a link, you load the page in the background but only flip to it on mouse up
This. I don't care about any additional features, just let me cancel my mouseclicks.
He is wrong. So many times I've held down on a button, then realized I've changed my mind. So I can release it without clicking.
Exactly. I have come accross some UIs that react on holding down and I always hate it.
exactly...I love Carmack, but he's just simply wrong here.
yeah, agreed
He's not "wrong". He's saying it feels better and he's entirely right.
Are there tradeoffs for that he's not talking about? He mentioned a couple, but also yes.
It's an idea. It doesn't have to be the best everywhere. Keep it in mind for a use case where it might be worth it.
Personally, accessibility compliance is the part I'd care about. I let Radix make this decision for me on a lot of interactions, and I'm not overriding their behaviour to make things happen on mouse down.
For the indecisive people like us, I think onRelease is the way to go xD
There can still be interaction and visual cue to actually pressing a button though. Which I agree.
One thing to mention: on things that should be on release, you can still have a little effect on press. Buttons are like that, to have 4 states: normal, hover, active (aka on pressed, no released) and disabled .... if I remember correctly. Having something like a color change on press would still maintain the feeling of very fast interaction.
as far as styling goes you're right. If my understanding is correct, Carmack's take is a step further than that, like sending a network request or changing the state of the program
@@privateanon7623 Yeah, but you need to see the bigger picture then. There are cases where it should be on release. Like allowing drag-and-drop (which Carmack mentioned).
Whatever happens directly on press has be easily undoable (like on a virtual keyboard) or not that important.
@@Winnetou17 I agree with the principle. Unfortunately that kills accessibility for people prone to accidental clicks, especially those with cognitive disabilities who might not necessarily realize they did. That narrows it down to a small subset of user actions in the non important easily reversible category for me.
the name for "active" is hold
I would add "focus" as a 5th state.
This not only goes against accessibility, but also user expectation
Imagine 99.9% of websites follow one standard, but you decide to go against it, just so your contact button has a potential saving of ~10ms
I especially hope no one uses on press for important actions, such as confirmations
This is highly context dependent. In most real-time applications, like games (where John Carmack has most of his experience), the user expectation is pretty much always on mouse down. Imagine if you were trying to fight a big scary boss monster and when you go to attack with your sword, you have to release the button before the character swings. Disgusting. Nobody would ever claim "but I wanted the option to not attack the monster sometimes, after half-pressing the button, no fair!"
99.99% of the webpages does UI wrong. The majority of UI should use OnMouseDown with exceptions
Wouldn't on mouse down also be better for accessibility.
If they have jittery hands that means it's very likely for it to go out of the bounds of the box and not activate the click.
@@dandymcgee Highly debatable whether that's true, but even it is, games tend to have garbage ui especially on pc so it's not really a good argument.
@@Utrilus The click event doesn't just mean a literal click. It can also be triggered using the keyboard, and when browsers are used with screen readers they will read clickable events out as clickable like links. If you use a different event like onMouseDown then I think that can only be triggered by an actual pointing device of some sort.
I thought Carmack denied that he was the one who wrote the fast inverse square root function. AFAIK it's actually unknown who wrote it.
Edit: did mobile always handle onMouseDown correctly? I swear I have unintentionally clicked buttons while scrolling a lot in the past.
same, but i suspect this issue is mostly prominent in horizontal sliders/scroll.
Yeah, scrolling down to the history section of the Wikipedia article shows that William Kahan and K.C. Ng first wrote a paper on the fast inverse square root in 1986, and then Greg Walsh wrote the exact algorithm in the late 80s.
@@JediMB I might be wrong but I think it's not the algo that john is praised for it's the WAY he implemented it in C with the weird address bitshift one liner. Don't quote me on this though. Just thinking out loud
I'm pretty sure mouse/touch actions used to have a ton more issues early on. I remember having to use an entire library just to handle simple interactions confidently. Nice to see it's so intuitive out of the box now.
@@TheBendixSA From what the article said, Carmack has never taken credit for it. It was just speculated that he'd written it after the Q3 code was made public.
If you read the history section, you'll see there's a pretty good timeline of how the algorithm presumably traveled from person to person until someone at 3dfx told Carmack about it.
I'd hate this on "send"/"confirm" buttons, where I might decide to NOT do that at the last second, and I can just drag my mouse away to cancel it. App devs, Carmack is a legend, but please don't listen to him here.
As a webdev, I'll never do act on press. This is simply anti-user practice
Mixed feelings. I can see the value in some games / high speed situations... But for most work apps etc, it speeds me up to be able to click at the speed of thought, and be able to "undo" the click by dragging off the button before letting go...
That way, each click is less deliberate / more optimistic, and the few extra millisecond before I let go of the button are enough time to hit the brakes on the action if it's wrong.
I feel like with "act on press", I'd need to be way more deliberate with every interaction, and it's stressing me out just thinking about it.
Although to be honest I'm sure this is something one would adjust to quite quickly, and is probably less significant than any of us here is making it out to be.
I feel like every instance of "I made a mistake" is just bad UI design, if it happens often enough then the design is broken.
The real solution for bad UI design is every device have a dedicated ctrl-z button to undo last action.
The best approach is probably somewhere in the middle. while mousedown events would improve mouse UX, high stakes user actions where modal dialogs aren't an option are probably best implemented through keyup type events to allow knowledgeable mouse users to cancel the interaction, even if that userbase is a tiny %
However I still think user accidents should be better handled in UIs. As it stands, proper action cancellation support is nonexistent which is why the alternative of dragging the click outside the box exists in the first place
Edit: Another comment mentioned WCAG success criterion 2.5.2 that ties in nicely with the subject
Word. There is (or was?) also a bug in KDE Plasma where you cannot use some modifiers involving Meta (or Windows, or Command) key in keyboard shortcuts, because it's used to invoke the application menu. Why? Because the action happens on key press, and it won't wait for you pressing other keys.
@@YaroslavFedevych and I always thought KDE was just one large bug that also had a DE attached to it.
"High speed situations" make it sound like you're in one of those cheesy hacker scenes where you say "I'm in" at the end
I really don't get the point of this video, if you cared so massively about quick responsive UI on such small timescales would you not be using keybinds for everything anyway? This whole thing seems like it's massively overstating the issue
Navigation on mouse down sucks when you just want to grab a link and drop it into a new tab or a separate window as the url. The same goes for buttons that navigate you with js without being anchors so you also can't middle click / ctrl click them into a new tab that way.
While I do understand his arguments.
I really don't want accidental click when I use my touchpad, cause yeah, if I was unlucky enough to stop using the touchpad with a cursor on an element and have touch to click, damn. retaking control of the cursor without clicking is going to be hard.
Being able to move the cursor OUT of the element before release to avoid clicking have saved me more than a few times.
Also, it's not than phone browser knows to ignore, but that the phone do not send the event to the browser in the first place. at least, it used to be the case on older iphone if I remember correctly
It is still a case on new iPhones, checked just right now. Wonderful feature, Carmack is wrong
It can be annoying though - tiktok uses the ontouchstart trigger for one of its buttons but if you touch it when scrolling, it gets pressed and the scroll is cancelled.
If you're on Tiktok, being annoyed is the least of your problems.
The amount of times ive clicked a button by accident and ive saved myself by holding down left click and dragging ny mouse off of the button is crazy. I actually love that part of it
I love John, and I love that you prefaced this video with some background on why people should listen to him when he talks. I love, even more, that you still thought critically as you listened and went and tried things out for yourself instead of accepting his experience and opinion as fact. Keep up the good work.
I think you could get the "best of both worlds" if you have an `onMouseUp` registered to `` that executes an undo on the last operation done in the UI if it is hit, while you have an `onMouseUp` directly on the element that simply cancels event bubbling. Then if you press and hold, then move away before releasing, it reverts whatever that change was but if you press and release then it already started doing that operation on the `onMouseDown` event as Carmack is calling for. This would require that your UI maintains an undo mechanism under the hood, but people are people and make mistakes, so that's *probably* a good idea to have anyway?
I agree with him in a game.
I do not agree with him on the web.
People tend to forget that Carmack is a game developer, do not apply his wisdom outside of games, that does not really work.
You can absolutely do things on press even on the web, increment and decrement quantity in a shopping cart for example. If it is easy to revert the change, go ahead. If not, use the release event.
Really?
I just noticed that switching tabs in Chrome happens on press. But yeah, a link, or a Cancel or Submit button triggering on press? That sounds awful.
Well said. I 100% agree
> If it is easy to revert the change, go ahead
So you apply his wisdom outside of games or no?
This take limits you; you must interrogate wisdom to see if it is applicable in other domains, rather than having the stance that wisdom found in one domain cannot immediately be applied to another - innocent until proven guilty. Most of the time when insights are 'scoped' to the domain, it is abundantly obvious that is the case.
@@seannewell397 Most wisdom *is* scoped to a certain domain. The point is to not just take his words as gospel and do what he says without prior thought.
If you try to use wisdom from basketball and try to apply it to tennis just because both are ball sports, you'll have a hard time. Some things can be similar, but you need to consider everything on a case by case basis regardless. At that point your prior wisdom boils down to a checklist of things you could try, rather than things to take for granted.
Somehow I get the feeling this breaks accessibility unintentionally. It's interesting that it fired on a touch event, maybe mobile logic lets it work if it's the only event specified. Keyboard navigation is another concern.
That said, it does feel nicer to fire on a down event.
If one has jittery hands wouldn't that mean they would fail to click more often if it's not on touch event? A lot more chances to move out of bounds.
@@Utrilus @Utrilus The click event doesn't just mean a literal click. It can also be triggered using the keyboard, and when browsers are used with screen readers they will read clickable events out as clickable like links. If you use a different event like onMouseDown then I think that can only be triggered by an actual pointing device of some sort.
The cost of a mistake can sometimes be too high. The ability to undo an action is a crucial aspect of good user experience, allowing users to correct their errors.
The phone VR issue (applied to early VR headsets too) was not about fps (though 90+ fps is best for a feeling of presence within the virtual world), but about latency. The world catching up slowly when you turned your head was due to latency between you moving and the system detecting the movement, updating coordinates for the renderer, and sending the image to the screen so that photons could hit your retina. That is a chain of events where latency adds up at each step. It would not matter if you had a 200 fps renderer if the other steps had significant latency.
This might be okay for states that are temporary, however, I see a lot of alternate considerations like WCAG.
I certainly want the ability to opt out of buying something after mouse press by moving away from the button as many others would.
So we get the best of both worlds, how about we load the state during on mouse down but don’t commit the change until the click is registered on the element; this would be especially useful for making fetching data from the server more seamless.
Seems like a good idea! Feel like it would need a better implemtation by frameworks to make it easier/feasible to do.
I think that for sensitive or potentially destructive operations, buttons should act on release. Many times I have pressed a button to later think again and move the mouse over to cancel :p
True, but in that case they should trigger a confirm box imo
@@pianochess1882 Great, now you have two "responsive" buttons and confirmation boxes everywhere. Either way for sensitive or destructive operation there should always be a confirmation box. This is just a bad idea.
@@pianochess1882 every submit button, every link click, every next button now comes with an "are you sure". You have now doubled the amount of clicks to navigate the internet, and caused people to reposition their mouse / hand / keyboard focus / etc, to the center of the page, to click ok... which had better be a browser standard, because it would be an easy dark pattern for a company to swap them around from page to page, and 99% of the population would just not notice.
@@Xomps might as well make the one in the confirm box "mouse-up", confirm boxes everywhere probably are a good idea though anyways, right?
If buttons _sometimes_ act on release (and I agree that they should), they should _always_ act on release so as not to baffle the user with their inconsistent behaviour. There can be exceptions if they are clearly conveyed, but it's rarely necessary that buttons trigger some action on the down press, whereas sensitive and/or destructive operations happen all the time.
I use "release miss" many times to cancel click, probably some "unrecoverable" operations should use "up" event but all simple operations should use "down"
An important piece to the post I believe is building things to be “undoable” as much as possible. This helps negate accidental clicks, and if something is truly destructive a confirmation modal should probably be required
I'm more in the on-release camp. So many times I do that "click - oops! - drag off the control - release" thing. For FPS games, sure, act upon press. For most UIs, nope.
Thx for reading "less" as "fewer" correctly 🤓👌
This should be a user-specified OS setting. Users should be able to choose which event they prefer to count as "click", and applications (including browsers) should hook-in to this OS setting.
Many game UIs also act on button press and not release. Also, holding a button down will fire multiple press events in quick succession. Xbox guide uses tabs and if I want to go from tab 1 (games and pins) to tab 6 (settings and power) I press shoulder button once instead of five. It feels fast and indeed snappy, and I love it since I tinker with settings a lot. And here's the kicker: keyboard navigation in the browser works exactly this way too. I think UI designers would level up if they learn the intricacies of other input methods and not only mouse and touch
A me problem: I like to have to option to cancel my click action, if I have the reflex to do so.
A "web" problem: Buttons on the web have this really cool feature that when you focus on it with your keyboard and press spacebar (Enter too, I think), it calls the onclick function. The onmousedown event, on the other hand, it's not called. So you would either be sacrificing accessibility by doing nothing, or give you more work to do to ensure the native behavior still works (listening to keyboard events, checking if it's "spacebar" and if it is, call the onclick function, etc etc etc).
When the page layout changes from a result of a keydown, I imagine could cause problems. And if we want to mix different types of buttons, we really should have a visual language to tell the user what to expect. In VR UIs I can fully see how keydown events are better, I dont how many times I've missed a button because my hand shaked a bit before I was able to let go of the trigger. Hmm... Why are we using the pistol grip for VR cursor actions anyway? Is that really the most stable way? Could a reverse-flashlight-like grip be more stable than a pistol grip to avoid click-shaking?
Having to hold your hand PERFECTLY STILL while you click on UI elements in VR is the worst
Wish you explored this in a mobile context, because as a mobile user I would HATE if I mis-tapped a button while I was trying to scroll the screen and activate something I didn't want to.
OnPress by default. On release for heavy, possibly destructive, hard to cancel/undo operations.
Can’t tell you how many times I’ve regret-clicked and click cancellation has saved my bacon. 🥓
I don't know about other areas, but on the web, "Act on Release" is a must, users must be able to cancel a click or a touch on a button or a link.
I think it would be great to do mouse down for interactive within the page, where the risk of it not being able to be cancelled is a lot smaller that doing it with navigations
On press for navigation and other non-sending actions, on click for submitting should be a fine compromise.
Do all mobile browsers handle onmousedown like that? Firefox, Safari, and Chrome?
moving forward with this is a good idea, it will teach people to watch what they click before they click it
Oh no, please don’t! On release is a learned behavior, people know they can still drag their mouse away from the button before releasing to prevent the action to be called. Please don’t take this away from us! For most interactions people wouldn’t even notice, because when working in an app you don’t look at the counter. Most people here are probably working on apps where there are much bigger performance improvements can be made without breaking user expectations. Most people aren’t working on a keyboard on a VR headset, they are working on some random banking app that talks to some legacy backend and every page takes 15 seconds to load. But now they saved two full frames and all they had to do was to break 40 years of learned behavior!
Those performance improvements will never be made. We have years of history now to see that websites get SLOWER, not faster.
Been fixing games like this with steam input forever. Makes Dark Souls rolls actually usable
Huh, this disproved a couple of doubts I had, sounds like the better way. But I will add that apps/websites should provide a way to undo any accidental click, unfortunately many aren't designed for that.
It depends on what is more important, speed (or time accuracy) or avoiding accidental clicks. You can see this with Google's stopwatch. The start/stop button acts on press, the reset button acts on release. And it makes sense. That's also probably the reason why act on press is more common in games.
How can i incorporate this in shadcn style buttons?
I just went and onMouseDown'd something and it felt like precognition.
What about sliders? When I use sliders I always expect the release to be when the value is captured.
one recent trend in gaming is to use hold actions and i can only guess it originated from consoles where you dont have a 10x keyboard to choose your inputs from. so most recent games i played have a double, sometimes tripple assignment for buttons if you count double tapping as well. i would say generally double assignments are to be avoided and maybe a more context sensitive action would be preferable and also better suited for act on press. it all depends on the context of course, but i agree for most cases act on press would be better. but more as a guideline than a rule
How does this influence the accessibility of the application?
Please don't. There's a reason why he can't get traction for his idea. UI developers know more about how UI works than a game developer.
Users do not perceive acting on mouse up to be slow. And the only users who unintentionally move the mouse between mousedown and mouseup are the ones with a shake and thus are more likely to misclick. They need the option to opt out of a click.
Then there's mobile, where touching the screen can be a scroll attempt or even an errant press.
People have learned that you move the mouse to cancel a click, and slide your finger to cancel a touch. Don't mess with people's intuition just to create a more "responsive" experience they won't notice.
There's a reason why UI developers will intentionally add delay. Outside of games or other fast action situations, people want a more deliberate pace. They don't even perceive it as lag.
I mostly agree with you but did you even watch the video? He did mention all these points
@@Logan9312 Well, no. I watched the video long enough to hear the original idea, and thought it didn't make sense, and thus had no reason to continue. I was going to say nothing, but then I saw the description where he says he genuinely plans to change how he codes.
I then responded to that.
I think for desktop it makes sense but for phones what counts as a press if it isn't firm? If I am scrolling on the phone and touch pixels while doing so should that count as a press?
I feel like press and releasing while still on the same element helps ensure you had an intentional click as opposed to something like Instagram where you can very often accidentally like a bunch of comments.
I'm not much of a UI guy so I probably got something wrong. Would be nice if someone has another opinion.
edit - Just watched the video and guess you bought up the same issues. Thought I'd comment before just to unbias my thoughts first.
I went through this conundrum recently when building some TV apps (with a Netflix-like 'Row/List' UI) where the interaction is using a remote control. I went with the 'press' in the end. It just felt better.
Worth also looking at using the 'PointerEvent' (onpointerdown) rather than the mouse events. To help cater for touch and stylus etc. controls, where onclick would have handled them.
But a button that has to do something on onMouseDown will do that even if we do a right click on that button. So how would you fix that?
should you try touchstart instead of mousedown?
Or even 'onpointerdown' which should cover mouse and touch.
Except you need to decide if user pressed the button or started the scrolling/gesture. So if you act on the press, how would you decide?
The underlying input subsystem decides, and it decides so by triggering the touch event after the deadline during which you are allowed to start scrolling, and dropping it otherwise.
Let's say the gesture deadline is 30ms, but you hold the button for 200ms, then the release event comes 170ms later than touch. No, it comes much later yet, because there is release debounce which lasts another 30-100ms until the input device driver is certain that a release has occurred. 200 to 300ms of time added.
If you were implementing an input layer yourself, you would launch a timer on hardware reported touch and then cancel it if gesture movement has started.
For the car example at 8:00. Modern cars don't use a turn key anymore, it's a push button to start the car, and that push button is not "on press", it's "on hold" along with other "if" conditions like holding down the brake pedal, or checking some sensors if mechanical damage consequences are detected.
The handbrake on modern cars is no longer a lever pulling a steel cable, it's an electronic button that you hold for a few seconds while the computer go through its "if" conditions, like the brake will act differently depending if parked or driving at high speed. If you mess too much with the electronic parking brake in a too short period of time, it will fall into standby mode.
The turn signal lever is a toggle switch letting electric current through, so you'd think that's kind of "on press", well not always since some drivers are complaining of few seconds delay before the light bulb start blinking on a faulty wiring curcuit that needs maintenance.
Modern cars also have some of the worst UI design ever and many don't incorporate it. The push to start buttons are largely disliked by most people with them as it causes the person to have to go through unnecessarily extra effort.
The electronic emergency brake is also rather inconvenient because in an event where you actually need it (let's say an engine fire occurs while driving and burns up the brake resivour placed on the firewall [where all the heat goes]), the electronic systems may not be functional. Not to mention having a computer decide whether or not to engage the emergency brake in an emergency is kind of sketchy.
Faulty wiring generally doesn't cause the delay, that's either the flasher module or body control module in newer vehicles. Faulty wiring usually is indicated by a rapid flash.
We really shouldn't be basic website UI off car UI, nor should we be basing car UI off web UI. Otherwise we'll be living in a world where you need to "start" your website and subscribe to a newsletter in order to use your brakes.
I'm going to have to try this out and see if I like it. I suspect I will. Just don't know if it will create a bunch of other issues. But it is a good idea to experiment with for sure. For draggable elements I probably just use onClick, but for non draggable elements I think that is fine. The fact that users can "cancel" a click by dragging afterwards is probably not a use case I would sacrifice better responsiveness for.
wow how did you get it working like that on your phone with just a local network, whenever I tried that it never worked
You need to configure your computer to allow access from the network inside the network settings. On windows you for instance need to mark the network as a trusted network. Also the other device must be connected to the same network (obviously)
I agree with Carmack with this one and the solution about canceling a misclick (which also was something I was worried about) is quite simple in most cases.
- If it is a destructive and irreversible action (or long computation/animation): confirmation dialog (which should already be used in such cases)
- Otherwise, allow undo-ing. Could be an actual undo-state that works with ctrl+z, or another button or whatever makes sense. Like a "close" button, a "back", a "decrement" or an "undo".
Besides, how frequently do we even misclick? It might be so rare that it makes no sense to not improve everywhere else just for this.
It's funny because this issue comes up in all the Fromsoft games with how they handle both rolling and running with one button. Once you're familiar with that negative edge/on-release behavior, it's not too problematic. But to new players, it simply makes for countless moments where they say "i pressed roll!" but they didn't roll and they died.
This causes a similar issue in Escape From Tarkov, where the reload key has multiple uses.. one function is the normal reload action, the other is a quick reload that drops your magazine when you double-tap the reload key. This seems fine on the surface, except that it mandates waiting for a set amount of time on each press of that key to see if another keypress comes in. So even if you have it set to on-press, which you can thankfully in Tarkov, it still waits 300ms by default to see if you press it again to make it a double-press. So, one of the common tips is to either swap the bindings, and mash R any time you want to do a normal reload, because it'll start as soon as that double-press is confirmed.. or rebind the Quick Reload action to some other key, and keep the double-press binding empty (which thankfully does allow the game to do the single-press action instantly -- it only invokes the double-press timer for a given key if that key has a double-press action.. kudos to them for this, or the game would literally be unplayable because *every* action would have a 300ms delay).
Azure uses it and it always surprises me when it triggers sooner than I expected
What is the react native equivalent to onmousedown?
Don't forget that onpress is also used for selecting and dragging and dropping! A user might want to do that instead of clicking!
How do we end up with these paradigms in the first place? Is the name of the method, "onClick()", deceptively intuitive sounding? Like a dev sees "onClick" and is like, "oh that means when I click the button it'll do something", which then causes debate over what a "click" is... So I wonder if the naming of the methods is partly at fault. Not to mention the literal tons of guides out there that mention onClick over onMouseDown.
I really like the insight you gave here as I've never even thought about it. Thanks!
PS I'm totally one of those people that didn't think further about "onClick()" until now, so this is great insight lol
06:52 there are virtual keyboards like onboard that are on release (by default) because you can hold to open character selector
Are there any on-screen phone keyboards that operate on touch rather than release? I don't think I've ever used any that do.
What about hyperlink anchors ?
tempting but would pass it for the web, I have a feeling that the annoyance of not being able to cancel when I want to would outweigh the micro win between the mouse down and up, it doesn't happen that often so I am not sure how exactly would you do a research that would take that in account
also when I mouse down a link and hold it, it becomes draggable and I can drag it to a new tab, that wouldn't work with the act on press pattern
Getting this sort of info at me, which I'd likely otherwise miss, is why I watch Theo's videos.
The Google store uses on mouse down and I had it miss fire twice when I scroll. So on heavier pages it does interfere with scrolling.
“Act On Press” - John Carmack's Bad Take On UI
And it's good to see this whole comment section go against that guys's bad take on a11y and UX
It’s a tool to keep in your pocket and be aware of, but… I remember 10-15yrs ago also going “Oh wow, way faster on mouse down vs. click” and then also pretty swiftly learning my lesson about why it doesn’t make sense to always do it all the time.
And under what context is that? everyone acts as if he was suggesting it to be implemented on every kind of UI yet his beef might be the exact same issue but inverse, that nowadays everyone and their mom implement press and release because programming for mobile rotted their brain.
@@teresashinkansen9402came here to say this.
John mentions all the use cases and arguments for when to use it and when not to use it.
It's also funny to see people disagree with someone who is infinitely better at programming and user experience than most of is will ever be without fully listening to his argument.
It's just "guy on internet said something that goes against what I learn in school." And they flip out
I don't get the argument that onPress feels more responsive. I'm sitting here pressing random onRelease buttons right now and they feel perfectly responsive to me.
I much prefer being able to correct my mistake by moving my pointer/finger away before releasing than the extra 10ms or whatever that I don't even notice.
Maybe in a high-paced game I would notice. For example, I can see the argument that a gun should shoot on the _press_ of a mouse click, rather than on release. But for most regular 2D UI buttons I think onRelease is the way to go.
If you want to support dragging (e.g. clicking and holding on a link in your browser lets you drag it to the desktop to make a shortcut) then you can't have the action occur on mouse down. But for simple buttons, e.g. in a game UI, I definitely prefer to do the action on mouse down. It depends on the context. Both are useful.
Another example of something that feels really nice when it happens on press, is pulling items from a stack in Terraria. Right-click takes 1 item from the stack, then if you continue holding right-click you get a very satisfying repeat action that slowly increases in speed and pulls more items out of the stack until you release the button. Perfect example of something that would be impossible to do on mouse up.
is it impossible to just make it a toggleable option and everyone wins?
I would hazard a guess that the reason we do it this way (on release) is because that's already how it works on Windows (maybe OSs in general, but I wouldn't know). I don't think most people would notice the difference between the two, and I'm curious to see how it'd change the experience in each setting.
P.s.: Going on mouse down some stuff sounds like my new favorite euphemism.
Using SDL you can capture button down / up events.. makes things super flexible
I think it's not black or white, this just depends on the part of the UI you are making.
For almost all of it, it's great! But when it comes to dangerous actions like "Factory Reset", "Delete" and "Cancel" buttons, it's better to use OnClick.
An example is Firefox, most of its menus are OnMouseDown, including changing tabs, but the close button on the tabs is OnClick due to it being an action that is "dangerous" in the sense that it can make you lose work depending on the page you are in.
Was just thinking about this pattern because of several unintentional interactions with the like button on LinkedIn. Was able to cancel with the decades-old OS-level UX of moving away. Act on mouse up is a very good default. For the love of good UX do NOT just started making ALL interactions act on press because you treat the internet as gospel truth.
does onMouseDown do anything if you're not using a mouse?
This is why I think using a 'PointerEvent' (onpointerdown) might be better.
4:19 that’s my only concern, but I guess you can always use other design standards like confirmation modals or something similar perhaps?
Emergency button stops only works on click off
I love being able to click and move way to stop the event from triggering on things like a crafting button in a game, this type of feature saved me many times with important things, everything else should be on mousedown lol
Sometimes I accidentally press something and I hate not being able to drag away while holding to prevent it from happening
Can't fully agree on this one. Way too many times I find myself changing my mind as I'm clicking on something and glad I'm able to slide off before releasing. I definitely prefer that chance to correct a mistake over some perceived "snappiness" to programs.
yeah I came across this issue not too long ago where I had a modal with a form in it and a background overlay that would listen to a click event to close it - what used to happen was if you selected some text in an input and end up on the overlay while doing it the modal would close on release, because the user released when on the overlay
I hadn’t thought much about it after that but this makes total sense, especially the real world examples
I’m gonna be advocating for this at work now for most of the use cases
So I changed it to mouse down and problem solved
This comes down to expected behavior of the type of UI element.
Act on release might be best for things that commit the changes but for everything else it is fair game.
In my own web development, I separated out the functions for touch and click. I separated out mobile and desktop versions. If I, a nothing early js developer could figure it out, massive conglomerates can too. Make responsive ui when you can.
Hmm I don’t think your mousedown scroll example would work on a native app. I think the only reason why it DID work was due to the built-in 300ms touch delay that is always there on all mobile browsers (to be able to differenciate between an actual tap and a native scroll/zoom event)
THIS IS THE TYPE OF CONTENT I SUBSCRIBE FOR! Easy to implement super helpful tips that I cant get anywhere else!
Just depends on what is being clicked for me. If I have a form with checkboxes and a submit button. The checkboxes/other inputs should be on press, and the submit should be on release. Tons of different cases, But I think the mindset of using on press as the default choice is correct.
I have ADHD. I have to select the text I use multiple times by double click or triple clicking it. When there's a button, I keep pressing the button, then drag away to not trigger it. I keep doing that with all the buttons. Or a press it, then check if the form is filled correctly, then mouse up when I verified.
I guess form buttons are different than a menu button or something like that?
Glad it’s not just me doing that 😂
The very thing you showed: Pressing a button and mentioning I didn't want that so I move away from the button and release then to cancel the action is something I think John did not consider. I was implementing for a pure C++ OpenGL GUI some widget logic and that part: Remembering what was pressed and released on to trigger correct "click" events was the greatest pain I ever had implementing some rudiementary logic and I still feel I missed corner cases where some kind of interaction should happen but I do not handle correctly.
Honestly I don't know how this isn't obvious. Other than sliding off of an accidental click, the whole thing just feels worse, and undoing accidental clicks is very unusual. If there's anything that's really bad to do by accident, you typically put a second "confirm" after it, and you can do On Release for that button if you want.
I also like to be able to cancel the action by moving away and release it, I guess for critical parts where you need that functionality, keeping track of the last action and undoing it could solve the problem, it makes the code way more complex though, but it's worth it if you already have the framework to undo it.
3:00 I believe it's a problem called magic mouse for macs (as you use mac and that mouse). Sensor is quite in front of the mouse so when you want to click on things, it was to drag that thing as mouse moves a bit on click. Super annoying
I always do this if I can, but you can't always do it. Accessibility is a huge reason (mainly on the web), as is avoiding potentially destructive unintentional actions from mistaken clicks. A good middle ground is to have some effect or work start on mouse down, but not "commit" until release.
Okay. Okay John, big respect. But what about PRESS AND HOLD? You won't be able to press and hold an icon on a touchscreen to move it around or to display more options or whatever. What about press and hold on a link to open it in a new tab or copy the URL? Or what about drag and drop? I get that it's about immediacy but please don't take this as a "this applies universally and everywhere!" kind of deal. Because it doesn't or we'll have to seriously think out of the box to come up with brand new paradigms that basically end up finding a problem where there's none.
on-press should be used when you’re imitating a real button, like with the virtual keyboard example, or maybe when the result of your button press is elsewhere than the button, and you expect the user to be watching for it (idle clicker game, games in general, etc). it is paramount that you can undo the action and avoid accessibility/scrolling nightmares by using on-release. if you rest your finger for too long on the button before scrolling, it’s going to register a press, but that’s just an example
How did it happen that we did not adopt act on press for mouse buttons when that is what we have always expected for keyboards ?
If you have a more than minimal move event before the release, you could abort / undo the action.