Testing

[juxibuz]

@konsumer I had to set the midi device manually with "-d 2", which took me a while to figure out. in the original js version this was default. you should include it in the options list in the next update for those who don't know...

after that all the commands worked. I only tried it with one pad, but I don't see any reason why it wouldn't work on other controls. the preset was saved and loaded back (again I only changed the value of one pad in the .beatstep file)

thanks! I'll fiddle around with the code and keep on posting issues and suggestions as they come along

[konsumer]

Glad it worked!

I had to set the midi device manually with "-d 2", which took me a while to figure out. in the original js version this was default. you should include it in the options list in the next update for those who don't know...

In the original JS, it would filter looking for a correctly named device for beatstep, which can have issues (multiple beatsteps, or non-beatstep arturia devices, that should work ok, for example.)

I realized this could be expanded to the full line of controllers that Arturia supports in the Midi Controll app, which would be cool, but not if it's hard-coded to search for a beatstep. On this version it defaults to the first device, but you can use list to find the device you want to target.

You can see the options for each command with beatstep COMMAND --help, but maybe the top-level help could explain that more (not sure how to set it up with the arg-parser I'm using, but it might be helpful.)

That seems ok to me, at least for now. Another idea might be to make device mandatory, and show an error "Please use list and use the device-number like this...." I'm not opposed to the search through the list in code for a default, but I'm not quite sure of the best way to do it in C++ for multiple devices and stuff, so this will at least get people started (since you can list and choose the correct device.) Maybe initially it would just be helpful to put that in the README, as well.

[juxibuz]

@konsumer I'm writing a UI with ncurses. it's working fine, but if I use the BeatStep at the same time, I start getting the message: MidiInAlsa: message queue limit reached!!

do you know how to deal with this?

there is some info about it here: thestk/rtmidi#236

it says it happens if MidiInApi::getMessage is not called often enough, but that's already too advanced programming for my skills

[konsumer]

I am not sure. I have been talking over here about possibility of using portmidi instead of rtmidi, and async callbacks, which I think would fix that problem. I'm not sure how you are using it, but I will try to look into it, soon.

[juxibuz]

stripped from any functionality the code is

  else if (app.got_subcommand(subCurses)) {
    initscr();
    int k = 0;
    device = 2;

    bs->openPort(device - 1);
 
    while (k != 'x') {
      k = getch();
    
      // do stuff

    }
  endwin();
  delete bs;
  }

basically using the BeatStep within any loop between bs->openPort(device - 1); and delete bs; will give the error

[konsumer]

Great information.

So, I tried putting it all in an async loop (so it grabs all the messages in the polling loop) last night, and it might fix this, but not the original thing I was trying to fix. sleeps. I wanted to remove them, but the beatstep can't handle getting a bunch of messages without waiting between them, and to get/set anything you have to send it messages. I think the sleeps will cause similar problems because it basically pauses the program while it's waiting for the beatstep to be ready for the next sent message. It also complicates the code to read things, because you have to keep a separate chunk of memory to hold all the results, and manage that, which is a bit clunky (instead of just asking the device for the value and waiting for the response.) I might be able to do something with threads, but that is like a whole other ball of code-worms, and would definitely have to be reworked completely on things like the teensy.

I think maybe we should take a step back, though. So, you get this error when you are playing it and also trying to load/save presets at the same time? Is this a feature you really need? I don't think the official software does this either, really. I can do some testing, but as far as I can tell, it does a big push/pull of gets when you press "sync", and puts it all in memory, then you make changes in the UI (while it's only listening to regular midi messages) and it doesn't send anything unless you hit the sync button again. If I understood your usecase a bit better (can you explain what you are trying to do?) I could probly make it do closer to what you expect, but as it is now, running the CLI should be similar to pressing "sync".

I can also take a look at using another MIDI library (portmidi) but the core problem of "the beatstep is kinda slow at responding to requests" isn't really solved, so it will need some management, I think, and there is no guarantee a new midi lib would fix it.

[konsumer]

As far as ncurses goes, is it meant to be like a TUI version of official software? Can you share a link? I think a really simple/jenky solution would be to turn off your midi and spawn the program (or library routine) for sync, but otherwise use an async callback in your thing. Another thing is we could link it a bit deeper, so it does my stuff in your async loop, too.

In rtmidi, a minimal program that shares an async callback would look something like this (untested):

#include "RtMidi.h"
#include <iostream>
#include <vector>

RtMidiOut *midiout;
RtMidiIn *midiin;
std::vector<unsigned char> message;
std::vector<unsigned char> version = {0,0,0,0};

// Platform-dependent sleep routines.
#if defined(WIN32)
  #include <windows.h>
  #define SLEEP( milliseconds ) Sleep( (DWORD) milliseconds ) 
#else // Unix variants
  #include <unistd.h>
  #define SLEEP( milliseconds ) usleep( (unsigned long) (milliseconds * 1000.0) )
#endif

void midi_callback( double deltatime, std::vector< unsigned char > *message, void *userData ){
  // process any incoming messages you care about here, here is an example that handles version-response:
  unsigned int nBytes = message.size();
      if (
        nBytes == 17 &&
        message[0] == 0xF0 &&
        message[1] == 0x7E &&
        message[2] == 0x00 &&
        message[3] == 0x06 &&
        message[4] == 0x02 &&
        message[5] == 0x00 &&
        message[6] == 0x20 &&
        message[7] == 0x6B &&
        message[8] == 0x02 &&
        message[9] == 0x00 &&
        message[10] == 0x06 &&
        message[11] == 0x00 &&
        message[16] == 0xF7
      ) {
        version[0] = message[15];
        version[1] = message[14];
        version[2] = message[13];
        version[3] = message[12];
      }
     // messages could be split up more, too, like sysex is 0xF0 ...  0xF7
     // notes & controls are described here:
     // https://www.cs.cmu.edu/~music/cmsip/readings/MIDI%20tutorial%20for%20programmers.html
}

try {
  midiout = new RtMidiOut();
  midiin = new RtMidiIn();
} catch (RtMidiError &error) {
  error.printMessage();
  return 1;
}

int device = 1;
midiin->openPort(device - 1);
midiout->openPort(device - 1);

midiin->setCallback( &midi_callback );
midiin->ignoreTypes(false, true, true);

// send request for current version
message = { 0xF0, 0x7E, 0x7F, 0x06, 0x01, 0xF7 };
midiout->sendMessage(&message);

// sleep before sending more messages
// but also, if you are not sending more messages right away, this could be left out
SLEEP(10);

Does that make sense? I'm happy to help with the real thing, as a curses-based CLI thing sounds pretty cool.

[juxibuz]

https://github.com/juxibuz/beatstep-cli-curses

I only made changes in main.cpp, CmakeLists.txt and README.md

I've only just started, so it just changes pad behavior and note, rest will follow...

for now I just want to play my synth via the MIDI jack while I'm making changes, to actually hear them. it all works fine, only this message starts flooding the screen after a while. it doesn't crash the program, so one could almost live with it - but rather not...

thanks for the info on rtmidi, I'll take a closer look once I have time again, i.e. next year

[konsumer]

This is a simple example that I actually tested, that shares async callback:
main.cpp.zip

I think keeping a big object in memory to hold the JSON settings, and just setting them when they come in will work, but you will still need to throttle your message-sending (as I am doing with SLEEP) somehow to ensure the device gets the message & has time to respond. In my example, I just log all messages that come back, but you could test them like I do in Beatstep.hpp for what sort of response they are (get or version.)

Here is my output:

Got bytes: f0, 7e, 0, 6, 2, 0, 20, 6b, 2, 0, 6, 0, 3, 0, 2, 1, f7
Got bytes: f0, 0, 20, 6b, 7f, 42, 2, 0, 1, 20, 1, f7

the first is version (reverse: 1.2.0.3)
and the other is the value of get 20:01: 1

the last piece is a set function, which basically looks like this:

void beatstep_set (unsigned char cc, unsigned char pp, unsigned char vv) {
  std::vector<unsigned char> message = {0xF0, 0x00, 0x20, 0x6B, 0x7F, 0x42, 0x02, 0x00, pp, cc, vv, 0xF7};
  midiout->sendMessage(&message);
}

which also needs sleeps (if you are doing a bunch on a loop) to wait for the beatstep to get the message.

[doctea]

I think maybe we should take a step back, though. So, you get this error when you are playing it and also trying to load/save presets at the same time? Is this a feature you really need? I don't think the official software does this either, really.
[..]
I can also take a look at using another MIDI library (portmidi) but the core problem of "the beatstep is kinda slow at responding to requests" isn't really solved, so it will need some management, I think, and there is no guarantee a new midi lib would fix it.

Hi again! Just wanted to chime in here with my latest findings from my Teensy project, where I've implemented async send and receive queues, and have just added the ability to retrieve/push the step sequencer data. Obviously this is using a different OS, MIDI and USB stack on the host side, but I'm also finding that messages sent to the Beatstep get missed easily. This can be improved somewhat adding delays between requests, but then the Beatstep loses sync pretty badly with the clock when retrieving/sending step sequencer data, seemingly regardless of any delays between the requests. I wonder if the Beatstep doesn't have a very large MIDI input buffer and loses bytes while processing sysex?

Since we all seem to be seeing these problems regardless of platform, it seems to confirm that this is a problem with the Beatstep itself. A shame, but I doubt that Arturia will be much interested in patching this old device since they have the BSP to sell.

Maybe there is still some trick to be found, some way to trick the Beatstep into reading all its messages correctly (maybe its expecting to get requests in a certain order, or expecting padding bytes between messages...?).. but I'm at a loss for now.

If I come across anything further that might help then I'll report back!