Toyota: Prevent EPS fault during unwind after sharp turn

[wtogami]

• 2021-2023 RAV4 Prime TOYOTA_RAV4_PRIME (logs below)
• 2022 RAV4 TOYOTA_RAV4_TSS2_2022 (logs below)
• 2021 RAV4 TOYOTA_RAV4_TSS2 (3 reports, no logs)
• 2021-2023 Sienna TOYOTA_SIENNA_4TH_GEN (2 reports, no logs)

These Toyota models can experience a rare EPS fault.

After a sharp, low-speed manual turn the human lets go of the wheel. During or shortly after the unwind the EPS faults. It can be difficult to reproduce.

PR #24067 "Toyota: remove 100°/sec steering lockout" attempted to prevent this high angle rate EPS fault by cutting torque if more than 18 frames. I found reducing this threshold by just one frame fixes the problem.

RAV4 Prime Routes

acd9ae4383134531/000000eb--b5fef978c7/488/503
acd9ae4383134531/000000d1--e4cb3cf682/484/498
acd9ae4383134531/000000d3--8495477bc1/567/584

RAV4 2022 Routes

8c3afba96b7a5db6/00000592--d968bfd1bf/11
timestamp: 19:22:40
EPS fault during manual right turn
acd9ae4383134531/00000155--d6bbd238ba/18
timestamp: 13:54:45
EPS fault during manual left turn

[github-actions]

Thanks for contributing to opendbc! In order for us to review your PR as quickly as possible, check the following:

• Convert your PR to a draft unless it's ready to review
• Read the contributing docs
• Before marking as "ready for review", ensure:
  • the goal is clearly stated in the description
  • all the tests are passing
  • include a route or your device' dongle ID if relevant

[wtogami]

@sshane back when you analyzed PR #24067 you had a lot more data to check against to find the ideal threshold. I'm curious if the Connect/Firehose corpus from recent months shows similar EPS faults happening on Toyotas at 18 frames?

[wtogami]

I'm convinced this is correct. Totally solves the problem for my Toyota. But it can be exceedingly difficult to reproduce.

I think you folks have access to all the logs to be able to verify this EPS fault is happening to other Toyotas and what threshold they are experiencing?

Please let me know if you really need me to get logs from the other Toyota models who reported the same issue. It's been difficult to get them to cooperate because they don't currently understand Connect, are not subscribed so they lack history older than three days, or it's difficult to reproduce the fault. The combination of these factors has made it difficult to get logs from the other users.

[jyoung8607]

For each route, it looks like the original mitigation fired, but slightly too late for the EPS's liking.

The original mitigation from commaai/openpilot#24067 was done over three years ago, and predates the breakup of controlsd into three separate services. As a consequence of that breakup, there's an extra frame of reaction time added to events like this. If Shane found the exact threshold back then, it logically follows we need to take out another frame here.

@wtogami: Thanks for diagnosing this!

@sshane: LGTM, but touches to release safety modes need comma review.

Representative snapshot of the event -- all three were about the same.

[wtogami]

Update: TOYOTA_RAV4_TSS2_2022 now has logs of the same EPS fault

RAV4 2022 Routes

8c3afba96b7a5db6/00000592--d968bfd1bf/11
timestamp: 19:22:40
EPS fault during manual right turn
acd9ae4383134531/00000155--d6bbd238ba/18
timestamp: 13:54:45
EPS fault during manual left turn