Cannot control pulse mode by arduino code in RTPP

[SallyWang234]

Hello,

I am new to bonsai and programming. I am trying to do an RTPP experiment using bonsai combined with arduino, where the laser is on when the camera captures and identifies the mouse on the designated side of the area. The page of the bonsai workflow and the arduino sketch is attached.
Arduino sketch for RTPP.txt

However, although the mouse's position can be identified and the laser is given, the laser does not flash at a certain frequency and interval as written in the code uploaded to the arduino, but rather always flashes erratically and irregularly.

I guess the possible reasons are as follows: 1) The Firmata protocolis not integrated with arduino, so the instructions in the code about laser mode cannot be executed; 2) Some delay settings in bonsai or arduino's loop affect the laser mode. But I have no ideas how to solve these problems by myself.

Thank you in advance for your help!

[NeuroThom]

Hi Sally, I think your issue is more likely to do with the workflow than the Arduino. I'll try to explain: the sequence at the bottom is subscribed to the "Tracking" `Subject, which will produce a new value or 'event' for every camera frame. This means when your animal is in the right section of the arena, the condition on the bottom will be passing the `True` result of the GreaterThan operator at the same rate as your camera. So rather than activating a single pulse train when the animal goes into that section, you are activating many pulse trains, which likely makes it look like the pulses are happening randomly. If you visualise the 'TriggerOpto' `Subject` that is feeding the `DigitalOutput`, or the output of your `Condition`, you should be able to confirm this. Depending on your specific requirements and setup, a `DistinctUntilChanged` node after your `Condition` on the bottom sequence will solve most cases. This node will only pass 'True' if the previous event was a 'False', so you'll just get one when the animal enters the area, and then a 'False' which will be stopped by the `Condition` when the animal leaves. Then you still have to think about whether you want to abort the pulses in case the mouse leaves the trigger area. Bear in mind as well that when the tracking centroid is at the edge of the cutoff area set in `GreaterThan`, whether the animal is in that area or not is ambiguous: Consider if the centroid is exactly on the threshold in `GreaterThan`, but the estimation is subpixel and of course has some jitter. So from frame to frame the centroid can jump from one side of the line to the other, meaning you may get a rapid burst of switching between 'True' and 'False'. One way to solve this is to have a timer or a counter than only activates the lower sequence when you have had a set number of observations (say 10 frames) or a set time (say 100ms) for which the `GreaterThan` has reported consecutive 'True' values, so you can be sure (or more sure at least) that the animal is in fact on that side of your arena. Another slightly more involved approach is to check that all (or a proportion of) the pixels detected in the `LargestBinaryRegion` operator are above your threshold. Hope that helps! Best, Tom

…

________________________________ From: SallyWang234 ***@***.***> Sent: Tuesday, February 18, 2025 1:46:58 PM To: bonsai-rx/bonsai ***@***.***> Cc: Subscribed ***@***.***> Subject: [Bonsai] Cannot control pulse mode by arduino code in RTPP (Discussion #2146) I am new to bonsai and programming. I am trying to do an RTPP experiment using bonsai combined with arduino, where the laser emits a laser when the camera captures and identifies the mouse on the designated side of the area. The page of the bonsai node and the arduino sketch is attached. Arduino sketch for RTPP.txt<https://github.com/user-attachments/files/18846109/Arduino.sketch.for.RTPP.txt> bonsai.workflow.for.RTPP.PNG (view on web)<https://github.com/user-attachments/assets/45761ee7-3b3a-4553-aa48-de43315f46e2> However, although the mouse's position can be identified and the laser is given, the laser does not flash at a certain frequency and interval as written in the code uploaded to the arduino, but rather always flashes erratically and irregularly. I guess the possible reasons are as follows: 1) The Firmata protocolis not integrated with arduino, so the instructions in the code about laser mode cannot be executed; 2) Some delay settings in bonsai or arduino's loop affect the laser mode. But I have no ideas how to solve these problems by myself. Thank you in advance for your help! - Reply to this email directly, view it on GitHub<#2146>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AI6XU4JQRYA6KKEK63Q6DY32QM2VFAVCNFSM6AAAAABXLV74TCVHI2DSMVQWIX3LMV43ERDJONRXK43TNFXW4OZXHE4DIOJTHA>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[SallyWang234]

Hi Tom,

Thank you very much for your answer! I think it is reasonable that the signal transmission is related to the camera sampling rate.

I have tried adding a "DistinctUntilChanged" node after the “condition" node. However, no laser is open whether the mouse is in the specified area or not. I tried to insert 'DistinctUntilChanged' in any other location, but it didn't work.

Is it possible that mice stay in the same area all the time, and the DistinctUntilChanged node will assume that there is no signal change, so it won't trigger the laser control instruction?

(Also, I don't understand which node in Bonsai the signal received by Arduino comes from. Is it from ”Boolean“ and “BitwiseNot”? When I double-click on the “GreaterThan” node, I can see that the value is 1 when the mouse is in the designated area, and 0 when the mouse is outside the area. What do these 1 and 0 values mean? Can they be sent to Arduino? I'm sorry that I don't quite understand the specific meaning of each node.)

Thank you in advance for your help!

Sally

[NeuroThom]

Morning Sally, No problem for the answer. Regarding your questions: Is it possible that mice stay in the same area all the time, and the DistinctUntilChanged node will assume that there is no signal change, so it won't trigger the laser control instruction?: Yes, this is completely possible, if the animal stays in the area you'll only get one 'True' value from the DistinctUntilChanged. If you initialised the TriggerOpto as 'True' at the start of the workflow and the animal starts in the area and never leaves, then you'll get a trigger when you start the workflow and never again. (Also, I don't understand which node in Bonsai the signal received by Arduino comes from. Is it from ”Boolean“ and “BitwiseNot”? Yes, the first one (the Boolean) is presumably set to 'True', and the BitwiseNot will convert this to a 'False', completing a 'NOT' operation on the 'True' value it receives. So your Arduino will set the pin HIGH, then after a delay (for a time set in your Delay operator), then set it to LOW after the BitwiseNot is triggered and sends a 'False' What do these 1 and 0 values mean? Boolean is a type of data that can take 'True' or 'False' values (or HIGH / LOW in an Arduino sketch). In a time series visualiser, these are interpreted as 1 (True) and 0 (False) to plot on a graph for visualisation. Boolean values are what is sent to the Arduino digital pins in your workflow: Firmata will interpret this as setting a pin to HIGH if you send a 'True', and LOW if 'False'. However, the value from the GreaterThan isn't the signal sent in your current workflow; as you've summised this is only reporting whether the mouse is in the area or not. You have other trigger logic and timing that is triggered by this event on the bottom sequence. If you haven't already, I think you may benefit from a look at the documentation and tutorials on the website: https://bonsai-rx.org/learn. The articles in the 'docs' section will help you get to grips with the different types, the operators, how observables work and how data flows through a Bonsai workflow. There are worksheets and tutorials that will also help. Couple of other tips: You can double-click virtually any node in Bonsai to understand how signals are moving and being operated on. If you right click the node you can also access the text visualiser instead of the time series, which is often more informative about the number of elements and their type. This will show you the True and False values, for example Finally, you can press 'F1' with any node selected to access the documentation for that operator from the editor itself. If you are still stuck, then come back and please send the actual .bonsai workflow and we can check the details of what properties you have set and so on. Best, Tom

…

________________________________ From: SallyWang234 ***@***.***> Sent: Wednesday, February 19, 2025 6:32 AM To: bonsai-rx/bonsai ***@***.***> Cc: TomRyan ***@***.***>; Comment ***@***.***> Subject: Re: [bonsai-rx/bonsai] Cannot control pulse mode by arduino code in RTPP (Discussion #2146) Hi Tom, Thank you very much for your answer! I think it is reasonable that the signal transmission is related to the camera sampling rate. I have tried adding a "DistinctUntilChanged" node after the “condition" node. However, no laser is open whether the mouse is in the specified area or not. I tried to insert 'DistinctUntilChanged' in any other location, but it didn't work. Is it possible that mice stay in the same area all the time, and the DistinctUntilChanged node will assume that there is no signal change, so it won't trigger the laser control instruction? (Also, I don't understand which node in Bonsai the signal received by Arduino comes from. Is it from ”Boolean“ and “BitwiseNot”? When I double-click on the “GreaterThan” node, I can see that the value is 1 when the mouse is in the designated area, and 0 when the mouse is outside the area. What do these 1 and 0 values mean? Can they be sent to Arduino? I'm sorry that I don't quite understand the specific meaning of each node.) Thank you in advance for your help! Sally bonsai.workflow.for.RTPP.new.PNG (view on web)<https://github.com/user-attachments/assets/576a2535-f98c-470f-badd-860fdf4934d2> — Reply to this email directly, view it on GitHub<#2146 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AI6XU4MWQ5P62LEXO4JMYKD2QQQOFAVCNFSM6AAAAABXLV74TCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEMRUGU3TKNA>. You are receiving this because you commented.