data drop sending long strings

[craiq]

Hi,
according to several issues adressing data drop, my problem could not be solved by those fixes yet.

Setup:
Win10 x64
Arduino IDE 1.8.6
bleuart sketch in full debug mode
Android Bluefruit LE App

Sending string from Bluefruit LE App via Uart to the Board.

Example String to send:

{
    "id": 1,
    "name": "Leanne Graham",
    "username": "Bret",
    "email": "Sincere@april.biz",
    "address": {
      "street": "Kulas Light",
      "suite": "Apt. 556",
      "city": "Gwenborough",
      "zipcode": "92998-3874",
      "geo": {
        "lat": "-37.3159",
        "lng": "81.1496"
      }
    },
    "phone": "1-770-736-8031 x56442",
    "website": "hildegard.org",
    "company": {
      "name": "Romaguera-Crona",
      "catchPhrase": "Multi-layered client-server neural-net",
      "bs": "harness real-time e-markets"
    }
  }

What I reveived:

{ "id": 1, "name": "Leanne Graham", "use{ "lat": "-37.3159",na", "catchPhrase": t-server neural-net"l-time e-markets" } }

I tried the pull request #178 , but it could not solve this problem.

Is there any other boundry, why this might not work?

What I realised is that up to 20 bytes it's safe to send data. Bigger junks are very likely to drop data.

Full Serial Log:

Bluefruit52 BLEUART Example
---------------------------

[CFG   ] SoftDevice's RAM requires: 0x20002CE8
Please use Adafruit's Bluefruit LE app to connect in UART mode
Once connected, enter character(s) that you wish to send

BSP Library     : 0.8.5
Firmware        : S132 5.1.0, 5.1.0 dual banks
Serial No       : 6F3Fxxxxx

--------- SoftDevice Config ---------
Max UUID128     : 10
ATTR Table Size : 2048
Service Changed : 0
Peripheral Connect Setting
  - Max MTU         : 247
  - Event Length    : 6
  - HVN Queue Size  : 3
  - WrCmd Queue Size: 1

--------- BLE Settings ---------
Name            : Bluefruit52
Max Connections : Peripheral = 1, Central = 0 
Address         : FD:9D:0C:BA:D2:AF (Static)
TX Power        : 4 dBm
Conn Intervals  : min = 20.00 ms, max = 30.00 ms
Peripheral Paired Devices: 

Central Paired Devices: 


[BLE   ] BLE_GAP_EVT_CONNECTED : Conn Handle = 0
[GAP   ] Conn Interval= 48.750000
[BLE   ] BLE_GATTC_EVT_CHAR_VAL_BY_UUID_READ_RSP : Conn Handle = 0
[GATTS ] Confirm received handle = 0x0000
Connected to Galaxy S7
[BLE   ] BLE_GAP_EVT_CONN_PARAM_UPDATE : Conn Handle = 0
[GAP   ] Conn Interval= 30.000000
[BLE   ] BLE_GAP_EVT_CONN_PARAM_UPDATE : Conn Handle = 0
[GAP   ] Conn Interval= 7.500000
[BLE   ] BLE_GAP_EVT_CONN_PARAM_UPDATE : Conn Handle = 0
[GAP   ] Conn Interval= 30.000000
[BLE   ] BLE_GATTS_EVT_WRITE : Conn Handle = 0
[GATTS ] attr's cccd
00:  01 00
[BLE   ] BLE_GATTS_EVT_WRITE : Conn Handle = 0
[GATTS ] attr's cccd
00:  00 00
[BLE   ] BLE_GATTS_EVT_SYS_ATTR_MISSING : Conn Handle = 0
[BLE   ] BLE_GATTS_EVT_WRITE : Conn Handle = 0
[GATTS ] attr's cccd
00:  01 00
[BLE   ] BLE_GATTS_EVT_WRITE : Conn Handle = 0
[GATTS ] attr's value, uuid = 0x0002
00:  7B 20 22 69 64 22 3A 20 31 2C 20 22 6E 61 6D 65
10:  22 3A 20 22
[BLEUART] RX: 
data: 7b 20 22 69 64 22 3a 20 31 2c 20 22 6e 61 6d 65 22 3a 20 22 
[BLE   ] BLE_GATTS_EVT_WRITE : Conn Handle = 0
[GATTS ] attr's value, uuid = 0x0002
00:  4C 65 61 6E 6E 65 20 47 72 61 68 61 6D 22 2C 20
10:  22 75 73 65
[BLEUART] RX: 
data: 4c 65 61 6e 6e 65 20 47 72 61 68 61 6d 22 2c 20 22 75 73 65 
[BLE   ] BLE_GATTS_EVT_WRITE : Conn Handle = 0
[GATTS ] attr's value, uuid = 0x0002
00:  7B 20 22 6C 61 74 22 3A 20 22 2D 33 37 2E 33 31
10:  35 39 22 2C
[BLEUART] RX: 
data: 7b 20 22 6c 61 74 22 3a 20 22 2d 33 37 2e 33 31 35 39 22 2c 
[BLE   ] BLE_GATTS_EVT_WRITE : Conn Handle = 0
[GATTS ] attr's value, uuid = 0x0002
00:  6E 61 22 2C 20 22 63 61 74 63 68 50 68 72 61 73
10:  65 22 3A 20
[BLEUART] RX: 
data: 6e 61 22 2c 20 22 63 61 74 63 68 50 68 72 61 73 65 22 3a 20 
[BLE   ] BLE_GATTS_EVT_WRITE : Conn Handle = 0
[GATTS ] attr's value, uuid = 0x0002
00:  74 2D 73 65 72 76 65 72 20 6E 65 75 72 61 6C 2D
10:  6E 65 74 22
[BLEUART] RX: 
data: 74 2d 73 65 72 76 65 72 20 6e 65 75 72 61 6c 2d 6e 65 74 22 
[BLE   ] BLE_GATTS_EVT_WRITE : Conn Handle = 0
[GATTS ] attr's value, uuid = 0x0002
00:  6C 2D 74 69 6D 65 20 65 2D 6D 61 72 6B 65 74 73
10:  22 20 7D 20
[BLEUART] RX: 
data: 6c 2d 74 69 6d 65 20 65 2d 6d 61 72 6b 65 74 73 22 20 7d 20 
[BLE   ] BLE_GATTS_EVT_WRITE : Conn Handle = 0
[GATTS ] attr's value, uuid = 0x0002
00:  7D 0A
[BLEUART] RX: 
data: 7d 0a 
{ "id": 1, "name": "Leanne Graham", "use{ "lat": "-37.3159",na", "catchPhrase": t-server neural-net"l-time e-markets" } }
[BLE   ] BLE_GATTS_EVT_WRITE : Conn Handle = 0
[GATTS ] attr's cccd
00:  01 00
[BLE   ] BLE_GATTS_EVT_WRITE : Conn Handle = 0
[GATTS ] attr's cccd
00:  00 00
[BLE   ] BLE_GATTS_EVT_WRITE : Conn Handle = 0
[GATTS ] attr's cccd
00:  00 00
[BLE   ] BLE_GAP_EVT_DISCONNECTED : Conn Handle = 0
[GAP   ] Disconnect Reason 0x13

Disconnected
[BLE   ] BLE_GAP_EVT_TIMEOUT : Conn Handle = 65535

[hathach]

• 20 bytes is the largest MTU previously on SD v2
• Starting from SD v5: MTU can be configured to max ~240 bytes and negotiated with the host. iOS normally accept MTU around 100 bytes.
• sending more than an MTU require LONG WRITE sequence

Thanks for filing the issue with a specific test case. I couldn't investigate now, since I am too busy with nrf52840. It may take a bit of time, please patient !!!

[mkirkhart]

@craiq

I found these same problems, and submitted a pull request (#147) some time ago which fixes them.

If you want, feel free to try them out. For convenience sake, I have created a new branch (https://github.com/mkirkhart/Adafruit_nRF52_Arduino/tree/mk_BLE_UART_updates_with_upstream_updates) which has these fixes merged with the current state of the master branch. I ran a quick test of this with my end project, and it works correctly.

To send data longer than 20 bytes, you will need to enable transmit buffering using the BLEUart::bufferTXD() function. The prototype for this function is as follows:

void bufferTXD (uint8_t enable, uint16_t fifo_depth = BLE_UART_DEFAULT_FIFO_DEPTH);

where enable = 1 to enable buffering, or enable = 0 to disable buffering (if already enabled). If not specified, fifo_depth defaults to BLE_UART_DEFAULT_FIFO_DEPTH, or 256 bytes. In my end project, I use a fifo_depth of 2014.

@hathach

Any reason you have not responded on my pull request (#147)? Even with all the changes since the pull request, it should merge cleanly. If needed, I can submit a new pull request.

Michael Kirkhart

[hathach]

@mkirkhart I am really sorry, I am much appreciated for anyone's PR or constructive comment on any issues in general. Your PR is really relevant and affect many users. So I would like to sit down and test it properly. Unfortunately I am too busy with nrf52840 prep which will be launched soon enough. nrf52840 has exciting USB but It also come with lots of works. I need to get it running with both CircuitPython and Arduino at launch as well

To sum up, my hands are tied down to nrf52840, I can only look at simple and easy issue without much testing for now. You can find a long queue of waiting issues and PR as well as forum support .

[craiq]

Thank you very much @mkirkhart
Thanks for the hint with the buffer. Without it didn't work 😅
Will try it soon!

[craiq]

hi @mkirkhart
still have trouble with this problem.
Using your branch and bleuart.bufferTXD(1,1023); before or after bleuart.begin(); dosn't fix it for me.
Higher buffer depth makes also no difference.
Note I use the bleuart sketch for this testing.

Do you have some suggestions where the problem is?
I'm testing with a 65 char json string.

[khawajamechatronics]

Hi,

I am having same issue.

As mentioned
'Starting from SD v5: MTU can be configured to max ~240 bytes and negotiated with the host. iOS normally accept MTU around 100 bytes'

From where I can modify? for BLE Uart example. My program crashed when I send more than 20 bytes

[hathach]

probably related to Long Write support, which is implemented in master, would you mind pulling master code and give it a try.

[craiq]

@hathach thx for your effort!
sending long stirngs from the board to the adafruit app 3.0.9 works fine now!
Conversely sending long strings from the app to the board still does not transmit the full string.

[hathach]

@craiq have you tried with the latest master code, if you still have issue with bluefruit app, try to use nrfconnect app from Nordic. Long write is just implemented, it should be fixed now.

[craiq]

@hathach latest commit on my local copy is 3c27998
(first updated Board in the IDE to 0.9.3 and then recreated to folder with git clone ... , restarted the IDE and upload the bleuart sketch)
Nordic app: (new to me and don't know if I used it properly)
with the nordic app I can send 256 characters to the board. stable without drops
receiving characters with the app is about 20 characters. Everything above will just show the last package.
Adafruit App:
app receives much more bytes now per transfer, but still with dropped packages.
conversely the same.
amount of bytes is quite random

[craiq]

@hathach tried between two nrf52832 now
they send max 320 chars
first 128 chars look stable, afterwards it's unstable until it reaches the 320 chars.
It looks like it gets aborted one package before 320 chars are transmitted, make a break and transmit one more package. Last package is different from transmission to transmission.

[hathach]

That is at least long write is working, seems like it is buffer related issue. Can you provide detail instructions to reproduce the above missing package: your phone hw, os, app you use to test, sketch that running on the feather board. I will try to troubleshoot that

[craiq]

Phone:

• Samsung S7 with modded original rom
  PC: Win 10 x64 Pro 1803
  App:
• nRF Connect for Mobile
• Adafruit Bluefruit LE Connect

Arduino IDE 1.8.6 - used integrated serial monitor
Board Lib: 0.9.3 (updated to 3c27998)

Sketches: (example sketches)

• central > central_bleuart (cleaned void bleuart_rx_callback - deleted first and last Serial.print)
• peripheral > bleuart

Thats my setup. Happy trouble shooting :)

[hathach]

thanks, but I am still not clear how you send 320 chars to feather that causes the issue. Can you elaborate how to reproduce that

[craiq]

ah ok. as I wrote in the comment this information is from.
This was with two nrf52832 connected to one PC, running two serial monitors.
One was running as central bleuart, the other as peripheral bleuart.

[hathach]

thanks @craiq, that is a different issue, most likely the txbuf packe count is running out since bleuart central doesn't do LONG WRITE sequence. I will investigate this later on.

[connelly38]

Is this still an issue with the latest (as of Dec 2020) released software? I'm trying to get more than 20 bytes sent from my nrf52840 bluefruit sense to the Bluefruit Connect app. I tried setting bufferTXD(true) (which seems to be the latest api) but I still can't get anything above 20 bytes per packet. Is this still in the TODO list, or is there a better recommended way?
Thanks!

[eulphean]

Hi @connelly38, I'm stuck at the same issue. Did you ever figure out how to get around this issue? Maybe, I need to write a new algorithm that chunks the data that I send from the arduino.

[CarbonNeuron]

I am also experiencing this, is there any solutions available?