AP_RCProtocol: Serial port SBUS input erratic

[JacksonUAS]

Bug report

Issue details

Putting inverted SBUS onto a serial port as per multiple Rx install and enabling FS_THR_ENABLE results in looping failsafe alarm. FS_THR_ENABLE disabled results in no failsafe.

Version
AC4.0.3

Platform
[ ] All
[ ] AntennaTracker
[ X ] Copter
[ ] Plane
[ ] Rover
[ ] Submarine

Airframe type
X8

Hardware type
Cube Black
Herelink (inverted SBUS onto Serial 5/CONS)

Logs
https://discuss.ardupilot.org/t/dual-rx-input-insanity-herelink-and-rfd900x-sbus-passthrough/54620/8

[JacksonUAS]

Transposed SBUS inputs, can confirm there is a bug with the handling of failsafe condition on inverted serial receiver.

[JacksonUAS]

Issue exists on cube orange without external SBUS inversion as well.

[Pedals2Paddles]

Based on the log provided, the problem appears to be AP processing the RC input from the serial port. The log shows the channels all bouncing between good and no signal. This of course is making the failsafe kick on and off. It is NOT a failsafe problem. The failsafe is just a symptom of the RC handling problem. The user's log is attached.

He has already tested using multiple different inverters and difference receivers. Everything looks fine on a scope. But AP shows the input bouncing all over the place. Given his multiple methods of testing to rule out a hardware issue, I'm going to tag this as a bug. Feature originally added here last year #12152.

Log attached from @JacksonUAS
serialRCinput.zip

More discussion here: https://discuss.ardupilot.org/t/dual-rx-input-insanity-herelink-and-rfd900x-sbus-passthrough/54620

[Pedals2Paddles]

@JacksonUAS I changed the title since this is a core RC protocol library problem that impacts all vehicles and builds, not just a copter failsafe issue.

[JacksonUAS]

Any updates on this? Is there any timeline I can convey to operators about when this will be investigated.

[sachman]

Any update on this? This seems like a very serious bug,

[sachman]

@bugobliterator , Could it be a problem of SBUS data capture itself (due to CPU overload)? Or of processing?

I tried putting the RCin port read in Timer thread instead of RCin thread - it gave me a lot of continuous LATE_FRAME errors with this change. Very strange.

[sachman]

I have increased the RCIN logging rate to 400 Hz (in code) keeping rcin port read in rcin thread itself (as it was). Below is a screenshot of what is typically happening to RC signals on my side. Might be helpful in debugging. The frequency of these glitches are quite high.

Also, I have seen glitches, although really rarely, on Copter 3.5.5.

[sachman]

The problems seems to be of data capture:
With enabling DMA for the RX of the uart port capturing SBUS, the glitches reduces by around 20 times (and hence so does desired attitude and other relevant aspects.)

Attaching here logs for the two setups from my test bench with RCin logging at 400 Hz and Attitude Logging at 400 Hz:
DMA enabled and DMA disabled on RCin UART port.

https://drive.google.com/file/d/1RgvqAzRRvqQUCrWIuBeJ17MWSuLqQQvb/view?usp=sharing

I don't see these kind of glitches on Copter 3.5.5 as significant as this.

Have the CPU processing requirements considerably increased from 3.5.5 to 4.0.3??

[sachman]

With DMA enabled, I tried increasing the DMA stream for the relevant USART to highest (from 0 to 3). However, that also did not completely remove the RCin glitches. I am super confused right now. Can somebody point me any further?

[sachman]

Hi,
The problems seems to be of capture and reading pipelines - the capture (when DMA is not enabled) is poll based in UART thread which has priority of 60 (low priority), and rcin reading in RC thread (which has lower priority than the main ap thread) are too slow.
If you move the reading part of RC from RC thread to TImer thread, and if you enable DMA on RX or capture byte in interrupt callback for the uart, the problem goes away.

No glitch at all!!!!! :D

[JacksonUAS]

Any updates on this one? I have the aircraft back here to try sort out for customer.

[andyp1per]

Use a DMA enabled serial port seems like the safest option. Changing the thread priority of the RCIN or UARTs is a more intrusive change. I suspect this is true for a lot of RC protocols - we should probably add DMA to the decoding report so that people know.

[peterbarker]

@JacksonUAS this one seems to have dropped off the radar for 3 years. Is this actually still a problem?!

[peterbarker]

No answer in half a year.

We've completely rewritten that handling several times since 2020. I don't think its realistic we'll trace this problem now as it has almost certainly been replaced with better bugs.

Closing this now.