Scroll and other mode for pointing devices

[Jaraxzus]

Now there are a few modes for working with the trackball
I only use scroll mode, but there is also a high-precision mode, etc.
I searched through the source code and pool requests, and I liked the idea of an abstract PointingProcessor from PMW3360 and PMW3389 input devices
It seemed to me a good idea to make such modes options of the contact Pointing device, and not a specific implementation, since the algorithms are most likely universal for all devices, at least software implementations
I made one for myself

struct ScrollConfig {
    delta_per_tick: u8,
}

impl Default for ScrollConfig {
    fn default() -> Self {
        Self { delta_per_tick: 25 }
    }
}

struct MotionAccumulator {
    acc_x: i32,
    acc_y: i32,
}

impl MotionAccumulator {
    const fn new() -> Self {
        Self { acc_x: 0, acc_y: 0 }
    }

    fn process(&mut self, dx: i16, dy: i16, divisor: u8) -> (i16, i16) {
        self.acc_x += dx as i32;
        self.acc_y += dy as i32;

        let out_x = (self.acc_x / divisor as i32) as i16;
        let out_y = (self.acc_y / divisor as i32) as i16;

        self.acc_x -= out_x as i32 * divisor as i32;
        self.acc_y -= out_y as i32 * divisor as i32;

        (out_x, out_y)
    }

    fn reset(&mut self) {
        self.acc_x = 0;
        self.acc_y = 0;
    }
}

enum PointingMode {
    Cursor,
    Scroll {
        config: ScrollConfig,
        accumulator: MotionAccumulator,
    },
}

impl PointingMode {
    fn generate_report(&mut self, x: i16, y: i16) -> MouseReport {
        match self {
            Self::Cursor => MouseReport {
                buttons: 0,
                x: x.clamp(i8::MIN as i16, i8::MAX as i16) as i8,
                y: y.clamp(i8::MIN as i16, i8::MAX as i16) as i8,
                wheel: 0,
                pan: 0,
            },
            Self::Scroll { config, accumulator } => {
                let (pan, wheel) = accumulator.process(x, y, config.delta_per_tick);
                MouseReport {
                    buttons: 0,
                    x: 0,
                    y: 0,
                    wheel: wheel.clamp(i8::MIN as i16, i8::MAX as i16) as i8,
                    pan: pan.clamp(i8::MIN as i16, i8::MAX as i16) as i8,
                }
            }
        }
    }
}

But I haven't figured out how to push values from configs here, so I've hardcoded them at the initialization stage so far.

Next I found ScrollWheelProcessor in Add config for input devices
so far, I don't understand how this will work, with the current settings
in the presence of processor_chain, the current code will be ignored, for example in Pmw3610Processor::process? Since it also processes the delta arrival event and moves the cursor.
or you will also have to add scroll_mode: bool to Pmw3610Processor so that the main and auxiliary processors do not work together. But then there will be a strong connectivity of components, which doesn't look very good.

In general, I have some doubts about how this will work, which I decided to share with you.

[HaoboGu]

Thanks very much for sharing! The design has not been decided yet, any feedback is welcome!

But I haven't figured out how to push values from configs here, so I've hardcoded them at the initialization stage so far.

Just wanna check, what you mean is to change the processor's mode at the runtime, am I right? Why do you think with processor_chain, the current code will be ignored? This only happens when there are two processors processing the same event, which is not common?

[HaoboGu]

tbh, I don't like the current implementation in #685 either, I'm currently looking for a better approach.

[Schievel1]

Just a quick note: I added the high precision mode for my board by having a controller that listens for an event and if it happens it sets a lower cpi for the sensor.
This works, however if those events happen in too short succession, some sensors are too slow to react and you end up in the wrong setting.

Therefore I wanted to make a pointing processor that does it in software. E.g. it has three different modes, for normal, precision and dragscroll. And do the whole thing in software without touching the sensors settings.

That pointing processor should get what events it listens to (and all the other settings) as parameter for its new()? But how would that work with #685 and its automatic calling when you call the processor in toml? Iirc the only argument is &keymap.

Also this processor should handle the invert_x, invert_y and swap_xy functions imo. As this is not always implemented on the sensor level and having such a functionality in the driver is the wrong place. (Driver is just delivering raw data not altering it)

However, I think we should first take a step back and think about if this should be in RMK or if keyboards should implement this themselves (once custom processors arrive it should also be possible to combine with toml configuration)

[HaoboGu]

This works, however if those events happen in too short succession, some sensors are too slow to react and you end up in the wrong setting.

I don't really get the issue, what you mean is that when lots of controller events are sent, the sensor fails to receive(because it's pub/sub channel, and most publisher use publish_immediate)

That pointing processor should get what events it listens to (and all the other settings) as parameter for its new()? But how would that work with #685 and its automatic calling when you call the processor in toml? Iirc the only argument is &keymap.

It should not be an issue for built-in processors, for the custom processor,

#[rmk::keyboard]
mod my_keyboard {
    // Custom processor - name matches function name
    #[processor]
    fn scroll_processor() -> ScrollWheelProcessor {
        ScrollWheelProcessor::new(&keymap, ... any_other_params)
    }
}

this way can be used for initializing the processor. It cannot be configured in keyboard.toml, but it can be referenced as scroll_processor in keyboard.toml.

However, I think we should first take a step back and think about if this should be in RMK or if keyboards should implement this themselves (once custom processors arrive it should also be possible to combine with toml configuration)

Yes, I'm considering other approaches, the current impl #685 is not very good, it's just a try. Ideas are welcome!

[Jaraxzus]

Just wanna check, what you mean is to change the processor's mode at the runtime, am I right?

Yes. Something like the `PointingProcessor ' extension added a state to it

/// PointingProcessor that converts motion events to mouse reports
pub struct PointingProcessor<'a, const ROW: usize, const COL: usize, const NUM_LAYER: usize, const NUM_ENCODER: usize> {
    /// Reference to the keymap
    keymap: &'a RefCell<KeyMap<'a, ROW, COL, NUM_LAYER, NUM_ENCODER>>,
    /// Working mode
    current_mode: PointingMode,
}

But I didn't figure out how to push the configs to PointingMode so as not to change a lot of code

In fact, we have a task that can be common to all Pointing devices, these are some basic options for transforming the results obtained from sensors. I think it would be a good idea to make a general implementation of these transformations.

For example, the high-precision mode can be made soft, by dividing, or as we wrote above, by actually lowering the cpi. I think the soft version can be delivered out of the box, and the hard version if possible.

Why do you think with processor_chain, the current code will be ignored? This only happens when there are two processors processing the same event, which is not common?

As I understand it, now when the system receives an event, it is passed through the processor chain.
For example, the event passed through Pmw3610Processor|PointingProcessor or any other default processor
where a MouseReport was generated for cursor movement, because the default processor does not know that the same event in ScrollWheelProcessor will generate another MouseReport.
It seems to me that it is easier to solve all this in one processor, which encapsulates all the logic of processing a single signal. At least for input devices. If it needs to use any other processors, it will notify them via the same bus, but with a different specific event.
This is the first thing that comes to mind, I'm not sure that it will really work.

[HaoboGu]

OK I got what you mean.

Yeah the event processing of multiple processors is hard. For each processor in the chain, there are 3 options:

1. The processor processes and consumes the Event
2. The processor transforms the Event to another Event and pass to the next processor
3. The processor ignores the Event and pass the Event to the next processor as is

If I understand you correctly, you have multiple processors which are interested in a same Event type, so the first processor just has no idea about whether to consume or ignore the Event, right?

There are two solutions from me:

1. Ensure that one Event type can only processed by only one processor
2. Adding something like event id. The processor only consumes the interested Event type with correct id

Any other ideas?

[Jaraxzus]

1. Ensure that one Event type can only processed by only one processor

it seems to me that this is not very flexible, such a restriction makes sense only for processors that look at the same event and perform conflicting actions. but they can perform disjointed logic based on a single event, then this restriction will add inconvenience.

2. Adding something like event id. The processor only consumes the interested Event type with correct id

Who will give idetntifiers, another processor, or the one who initially generates the event? If and how to determine the correct ID, it will be another nested enumeration?

I still like the option when it is unnecessary to limit anything, and give any identifiers to events, in fact, the event type, this is already like a special identifier, adding some other key to it, we will get a complex marshutization scheme, I'm not sure that this is necessary
it seems that processors should still be registered only for the event type
and do something like this

Each processor, depending on internal checks, will generate the necessary event if it needs to delegate part of the work to another processor, for example, 'Scoll Event, Cursor Event' etc.
And already the handlers of these events will generate and send MouseReport, so they will be independent of who generates the event.
This way we will have a clearer understanding of who is responsible for what, and there are no identical checks in processors that would exclude double processing.
This does not protect against the fact that someone can insert two identical processors into the chain, but this is already a user error.

P.S. embedded systems programming is not my main specialty, so I am not sure that my ideas will be valid within the limits set by the capacities of the target systems. But maybe they'll be useful)

[kot149]

I believe ZMK's input processor approach is the best. Jaraxzus' proposal represents one aspect of ZMK's approach:

• Processors handle common event types independent of sensor type (i.e., processors handle Cursor Event, Scroll Event, etc., not Pmw3610Event, JoystickEvent, etc.)
• Processors basically do not consume events, but pass them (with modifications) to the next processor. Finally, events are converted to HID reports in a separate location from the processors.

With this design, operations like scaling, inverting, swapping, and converting to scroll can be provided as independent processors, allowing users to freely combine them.

Also, in ZMK,

• The processor chain is independent for each device instance. This allows, for example, having two identical sensors installed, using one for cursor movement and the other for scrolling.
• Different processor chains can be configured per layer. This enables dynamic changes to parameters like CPI through layer switching.

[HaoboGu]

The current implementation is very similar with zmk's chained processor system, so we can learn how zmk solves the issue maybe?

[Schievel1]

I don't really get the issue, what you mean is that when lots of controller events are sent, the sensor fails to receive(because it's pub/sub channel, and most publisher use publish_immediate)

No, it's not an issue of RMK and its channels, it's just the sensor not accepting the changes in a too short succession.
This happens with qmk as well for example.

So an alternative to setting the actual cpi on the sensor is to crank up the cpi and then scale down the movement in software to whatever suits you.

[Schievel1]

• Processors handle common event types independent of sensor type (i.e., processors handle Cursor Event, Scroll Event, etc., not Pmw3610Event, JoystickEvent, etc.)

Also, in ZMK,

• The processor chain is independent for each device instance. This allows, for example, having two identical sensors installed, using one for cursor movement and the other for scrolling.

I don't understand. Doesn't that contradict each other? They handle common event types, so how do they distinguish two sensors that emit the same event type?

/ I understand now. So basically sensors are grouped into events they cause, like pointing event, key events, scrolling events, etc. and each sensors gets its own processor chain.
You have a toolset of processors which you can plug into the chain of sensors to alter the events.

Yeah, that's a reasonable approach. We already did a similar thing by having the pointing sensors also emit joystick events. In the end our joystick events became common pointing events, only the name is wrong. We could keep doing that and in the processors we identify the sensor and then the processor decides whether it needs to do something if the event is coming from "its" sensor.
Or we have separate channels for each sensor. (Kind of what we already have with the key matrix?)
Is ist reasonable to have separate channels or would that exhaust resources too much?