Incorrect mouse movement in 3D / 2D views on Plasma Wayland

[jonri]

14 hours ago, MirceaKitsune said:

I think there would be one way actually: What if DR didn't require cursor wrapping at all? Just translate mouse movement as it happens and don't care where the mouse pointer is located!

The problem comes when the mouse leaves the window.  For example, if you start dragging the 3D view to turn left and your mouse gets all the way to the left side of the window, you won't be able to keep turning left.  In order to turn left infinitely, the mouse pointer has to get snapped back to give you room to keep moving the cursor left.

[MirceaKitsune]

8 hours ago, jonri said:

The problem comes when the mouse leaves the window.  For example, if you start dragging the 3D view to turn left and your mouse gets all the way to the left side of the window, you won't be able to keep turning left.  In order to turn left infinitely, the mouse pointer has to get snapped back to give you room to keep moving the cursor left.

Right. And if the pointer reaches a screen edge or corner and can't go any further, the view wouldn't be able to keep moving / turning and get stuck... you'd need to disengage then click again to grab it which would be very annoying.

I'm well out of ideas. If only the "export GDK_BACKEND=x11" hack wasn't partial: It fixes it for the main views only leaving the model viewer broken, but for some reason activating the clipper to cut brushes breaks that as well and I need to restart DR every time after using it. Maybe we can see what the clipper does to trigger it and patch at least that bit out?

One last thing comes to mind: Is there a build flag to force DR to compile without Wayland support? Maybe I can try that to force the whole software to be x11 exclusive which should resolve this, granted wxWidgets allows it.

[MirceaKitsune]

I'd like to ask a (hopefully) easier favor from the DR devs on this, considering it's such an issue and making me struggle to keep mapping normally. I know the root issue seems to rely on WxWidgets and may not be solvable on our end, but at least one part of it I consistently run into should be.

Like I said I can initially avoid the problem by exporting the "GDK_BACKEND=x11" variable in my shortcut, this makes it go away for the 2D and 3D viewports and only persist in the model viewers which I can care less about. There's just one exception: The moment I press X to use the clipper tool and cut a brush, the 2D view starts experiencing the issue and becomes impossible to drag normally. Because of this I need to avoid using the clipper as much as possible, and whenever I have to in order to get a triangle brush I must restart Radiant immediately after.

Would it be possible to figure out what the clipper does that's so unique to triggers this? Does using it reset the 2D views and somehow gets them reloaded by Wayland? If yes could this be avoided so in the meantime it doesn't break that workaround? If at least this can be solved and the GDK_BACKEND workaround keeps otherwise working, it will be a lot easier to wait for the proper full fix. Sorry for the annoyance with this... just want to make some good FM's and using DR this way is running on a handicap that makes the already complex process of mapping even slower.

[7318]

I can confirm the problem in Ubuntu 23.10, and the solution proposed "GDK_BACKEND=x11" i confirm that to solves it partially, the clipper does bring back the problem.

Edited January 28 by 7318

[MirceaKitsune]

51 minutes ago, 7318 said:

I can confirm the problem in Ubuntu 23.10, and the solution proposed "GDK_BACKEND=x11" i confirm that to solves it partially, the clipper does bring back the problem.

Thanks. If you press X to activate the clipper tool and cut a brush, can you confirm that breaks even this solution until Radiant is restarted?

[7318]

yes, i confirm it.

[jonri]

I revisited this issue and might have had a breakthrough.  I noticed that when using the clipper my cursor wasn't changing, and afterwards when the cursor was no longer hidden when the dragging problem starts happening.

If I take out the code that is supposed to change the mouse cursor when activating the clipper, the drag behavior remains correct during and after using the clipper tool.

If I remember correctly from last time I went down this rabbit hole, Wayland has restrictions on pointer locking and I think it was only allowed while the pointer is hidden.  I'm guessing something with changing the cursor is failing and then it doesn't get hidden before the pointer lock happens, causing Wayland to reject the pointer lock request.  And when the pointer doesn't get locked, the math to calculate the drag amount goes all wonky.

@MirceaKitsuneor others, if you'd like to try a quick and dirty workaround, find the

void OrthoView::setCursorType(CursorType type)

function and put a return statement on the first line so it doesn't actually change the cursor.

I'll see if I can figure out what's actually going wrong when setting the cursors so we can make a proper fix.

[MirceaKitsune]

10 minutes ago, jonri said:

I revisited this issue and might have had a breakthrough.  I noticed that when using the clipper my cursor wasn't changing, and afterwards when the cursor was no longer hidden when the dragging problem starts happening.

If I take out the code that is supposed to change the mouse cursor when activating the clipper, the drag behavior remains correct during and after using the clipper tool.

If I remember correctly from last time I went down this rabbit hole, Wayland has restrictions on pointer locking and I think it was only allowed while the pointer is hidden.  I'm guessing something with changing the cursor is failing and then it doesn't get hidden before the pointer lock happens, causing Wayland to reject the pointer lock request.  And when the pointer doesn't get locked, the math to calculate the drag amount goes all wonky.

@MirceaKitsuneor others, if you'd like to try a quick and dirty workaround, find the

void OrthoView::setCursorType(CursorType type)

function and put a return statement on the first line so it doesn't actually change the cursor.

I'll see if I can figure out what's actually going wrong when setting the cursors so we can make a proper fix.

Oh my gosh, thank you for this! After reading what you just said, I realized I've in fact noticed the same thing in another program, but assumed it must be something completely unrelated: I've been working on creasing a CPU based voxel raytracing engine in Python, for which I settled with using PyGame. When the time came to implement first person camera control using the mouse, I noticed it did not work initially... however if I told my code to hide the mouse pointer first, locking it in the center of the window started working. It must be the same thing here in some form, Wayland likely has some very particular expectation on how the mouse pointer must be set in order to be locked.

In fact just a few days ago, I tried a virtual worlds software called Vircadia / Overte again to see how it's been progressing. I activated mouse look mode and guess what happened: The exact same behavior of the view spinning endlessly as the cursor drifts away from the center due to lack of locking. So it's clearly happening to many things and not just GTK / WxWidgets related... but no all of them, most games (including TDM itself) work perfectly fine and mouse control never breaks.

I'll try your solution later and mention the results, likely tomorrow as it's late now and I need to head off. If it works that would be incredible! At least as a workaround, we could simply not hide the cursor on Wayland, which will look ugly but at least allow using DR without needing hacks until they can solve whatever is happening on their end.

Also Plasma 6 was just released, my distribution (Manjaro) will likely be getting it a couple of weeks or at most months from now. I'm curious how that will affect it: Hopefully it won't make the issue worse, but just in case I'd be happy to have a solution in Radiant before then.

[jonri]

I've figured it out!  There are 2 parts:

1. The FreezePointer class has a mismatch, it blanks the cursor on the top-level window but does the pointer locking on whatever particular sub-window (i.e. the 2D view widget) calls for it.  The cursor has to be explicitly blanked on the same widget that locks the pointer.

2. The clipper tool updates the cursor whenever the mouse moves, even if it's in the middle of dragging and should be hidden.  It was easy enough to guard against changing the cursor while the mouse capture is active.

This also fixes the same issue that was happening in the 3D view of the model viewer (but not on the main 3D view).

I'll submit a PR shortly, @greebo or others will need to test this change on Windows to make sure it doesn't do any harm there.

EDIT: The PR is here: https://github.com/codereader/DarkRadiant/pull/37

Edited March 22 by jonri
Added PR link

[MirceaKitsune]

I manually integrated and tested your PR. There is good news and bad news: The good news is that when running with the "GDK_BACKEND=x11" flag, using the clipper tool no longer breaks the window forcing me to restart DR afterward, the model / entity viewer no longer experiences the issue either. The bad news is that flag is still required, every viewport retains the problem if running Radiant in Wayland mode, we still have a big problem Linux users will increasingly run into as distros adopt it (KDE Plasma 6 now uses Wayland by default).

So please integrate the PR if it doesn't break anything, it makes life much easier in the meantime! But this should remain open until either DR or WxWidgets or Wayland solve the core issue that exists when running in native mode.

[jonri]

Yes, I probably should have been more clear.  This fixes everything when running on XWayland (which emulates X the best it can but must still ultimately play by Wayland's rules).

Last time I checked, GTK3 was not planning to support mouse pointer warping at all on Wayland and left the function as a no-op in the Wayland version.  It might be simplest for us to figure out an easy way to automatically force the x11 backend to be used.  Wayland is increasingly becoming the default, but XWayland is not going away anytime soon.

[MirceaKitsune]

23 minutes ago, jonri said:

It might be simplest for us to figure out an easy way to automatically force the x11 backend to be used.

There might be another way, or at least it's what I thought of as a non-developer: Use a different way to transform mouse movement into camera rotation or viewport offset. Is there no alternative to calculating the distance from the pointer to the window center before resetting the pointer to the middle? There must be other mouse look implementations that could work.

Most obvious alternative: We can detect how much the pointer moved on the screen compared not to the center, but to its previous position wherever that may be. While of course still trying to lock it in the middle, but if that fails at least it doesn't cause the view to go crazy: The only issue then will be the cursor reaching the screen edge and having to be re-engaged so you can keep scrolling in that direction, which is also a big annoyance but comparatively less bad and not as noticeable unless you do long-range movements in one go. Any reason why we couldn't give this option a try?

[OrbWeaver]

I notice that what Blender does is not lock or hide the mouse cursor, but allows it to move, then snaps it back to the other side of the window so you can drag infinitely in one direction. I.e. if you are dragging to the left, when the cursor reaches the left-hand window edge, it immediately re-appears at the right-hand window edge and continues moving to the left.

I wonder if re-writing the code to use the Blender approach would solve the problem? Perhaps the back-end code can handle instantaneously changing the mouse pointer position more reliably than trying to lock it in place.

[jonri]

10 minutes ago, OrbWeaver said:

Perhaps the back-end code can handle instantaneously changing the mouse pointer position more reliably than trying to lock it in place.

It doesn't matter if you want to warp the pointer every frame or only once, in both cases GTK3 chose not to support it on Wayland.

Here's the hack that Blender came up with: https://projects.blender.org/blender/blender/issues/77311

They found they had no choice but to render the cursor internally to get the intended behavior.  Probably easier to do when you built your own widget toolkit like they did, rather than having to go through wxwidgets and GTK.

[OrbWeaver]

Oh well, I guess that rules out any kind of mouse grabbing behaviour that will work on Wayland desktops.

I'm not sure what the alternative is. Some kind of on-screen widget that you can click and drag to rotate the camera, like a sort of circular scroll-bar?

[MirceaKitsune]

13 minutes ago, OrbWeaver said:

Oh well, I guess that rules out any kind of mouse grabbing behaviour that will work on Wayland desktops.

I'm not sure what the alternative is. Some kind of on-screen widget that you can click and drag to rotate the camera, like a sort of circular scroll-bar?

Mouse look is essential so no. We should use whatever behind-the-scenes solution of messing with the pointer will work... once Wayland or WxWidgets or GTK3 will offer one, it's kind of on them that they haven't up to this point. Until then Jonri's PR works by forcing the X11 backend, which isn't a real solution but in practice solves everything for now.

Here's a modified snippet of what I did in PyGame which works. Obviously this code has no effect, just an example to show what works in my project, just in case it's a similar situation here: Maybe Wayland expects us to hide the mouse pointer at a core level before it allows locking, and the reason it doesn't work is DR only hides it visually at top level?

pygame.mouse.set_visible(False)
x, y = pygame.mouse.get_pos()
pygame.mouse.set_pos((x, y))
pygame.mouse.set_visible(True)

 

Edited March 23 by MirceaKitsune

[jonri]

In case anyone decides to pursue this further, here are a couple more references:
https://discourse.gnome.org/t/pointer-warping-on-wayland/9197

https://gitlab.freedesktop.org/wayland/wayland-protocols/-/issues/158

Seems like this feature is also common in PCB design software.  The "correct" solution would be to use Wayland's pointer locking interface and relative mouse events, which is essentially what the FreezePointer class accomplishes by continuously warping the pointer back to its original location.

The complication is that the way it works now all goes through wxwidgets which then goes through gtk3 on Linux.  To fully implement this, I think someone would have to:

• Create a platform-specific alternative to the cross-platform FreezePointer that we have now
• Make sure it's only called on Wayland, and only if the pointer locking API is available
• Work your way through the wxwidgets / gtk3 layers to get hold of the native wayland surface and pointer handles
• Reconcile the Wayland mouse events with the mouse events received by wxwidgets

If someone was motivated enough to pick this up, it might make sense to try to build this in at the wxwidgets level and try to get it accepted there.  It would help keep platform-specific code in the right place and also make a couple other projects happy.