Extra Transform options (NRPN/SysEx)?

Would be really great to be able to transform CC to NRPN or SysEx commands.

NRPN might not be too much of a departure from the existing Transform pipe functionality but since it requires MSB and LSB to be defined maybe it would need a new pipe.

SysEx can get really complicated, for example some of the old Yamaha FM synth parameters are quite wild. Often multiple parameters are combined with bitwise operations and then sent as MSB/LSB. The configuration options for this might be too difficult to implement in a user friendly way, though it would be so useful and unique!

Hey, weā€™ll likely add the NRPN transforms in future software releases for Midihub.

The SysEx is definitely tricky to do in a generic and easy to use way. Could you elaborate what operations would you like to perform with example input and output messages?

For synths like Yamaha DX7, Iā€™m considering adding a special ā€˜interfaceā€™ pipe to which one could MIDI map CCs to its parameters that would then get translated into the required SysEx messages for the synth.

6 Likes

there are two ways mainly that Iā€™d like it implemented:
cc to nrpn (Iā€™d be controlling dave smithā€™s tetra)
and cc to MSB-LSB distinct values (for nord drum pitch parameters)
OSC would be amazing too :wink:

2 Likes

The most interesting opportunities are probably in the land of Universal SysEx messages. For example the ability to transform to and from basic MMC transport commands would be one useful thing to have - mapping MIDI start/continue/stop controls is one fairly obvious use-case, another is translating Mackie to/from control transport commands (ie note on -events).

2 Likes

Are the universal start / continue / stop commands intended to be sent to particular devices in the chain, or itā€™s just a different way of sending start/continue/stop messages? Could you come up with some example how this would be used in practice? Personally I havenā€™t yet used the universal SysEx messages yet.

Universal SysEx can be directed to either a specific machine or to all of them and itā€™s best to not make assumptions about usage, so the Device ID should be a user-settable argument.

MMC is used by things like hardware multitrack recorders and DAW software, which may not respond to MIDI clock messages at all. Also MIDI start/continue/stop is expected to be accompanied by the actual clock messages so itā€™s kinda ā€œillegalā€ for a plain controller to send them, whereas MMC is just the transport control, itā€™s not concerned with sync at all, and ditto with Mackie.

As a concrete example from my home studio:

  • Fostex D2424LV multitrack recorder - receives MMC and MTC, can send MIDI clock
  • Pyramid sequencer, only talks MIDI clock
  • M-Audio Code controller, whose transport controls send Mackie control (note on) messages

So basically three devices all talking MIDI but dialects that the others donā€™t understand (much simplified here, the reality is far more convoluted and bizarre) :joy:

Much of the time the Pyramid is the clock master, but when recording audio the Fostex has to be the master, Iā€™m imagining using two separate MidiHub configurations for these situations. In both cases, Iā€™d like to be able to use the transport controls on the Code controller for, well, transport control of the clock master. As MidiHub can already do note -> start/stop transformations, the former case should work (once the MH eventually arrives), but not the latter.

The ability to translate between these transport-related ā€œdialectsā€ would seem a like a generally useful thing to have in the toolbox and fitting to the MidiHub role.

As for other MMC commands, at least MMC GOTO 0 as an always applicable special case of that command would be useful.

4 Likes

So youā€™d like transformations like these, where ā€˜device idā€™ is configurable for MMC messages:

  • MIDI Start -> MMC Play (possibly MMC Rewind to be sent first, then MMC Play?)
  • MIDI Stop -> MMC Stop
  • MIDI Continue -> MMC Play (no rewind message)

As well as equivalent transformation in the other direction, from MMC to regular MIDI.

Are there more transformations that should be done?

Is MMC Goto 0 equivalent to MMC Rewind?

Roughly yeah. Devilā€™s in the details as usual I guess.

Good point about rewinding on MIDI Start, although MMC Rewind is only the opposite of Fast Forward. So itā€™d need to be MMC GOTO 0 instead to go to start, GOTO 0 being equivalent of MIDI SPP 0.

The far more straightforward part would be arbitrary Note on ā†’ MMC op transformations which donā€™t need wacky semantic translations: all the MMC ops that donā€™t take complicated arguments, ie Sub-ID#2 from 01 to 0D.

Just as an example, Mackie control messages are something like:

  • MIDI Note on A#6: Play
  • MIDI Note on A6: Stop
  • MIDI Note on G#6: Fast forward
  • MIDI Note on G6: Rewind
  • MIDI Note on G5: Record

ā€¦with a straightforward mapping to MMC counterparts, but varying dialects of Mackie/HUI exist so these shouldnā€™t be hardcoded, just Note on ā†’ something transformations (and optimally, vice versa).

Just to get this straight, Iā€™m not trying to convince you these are Highly Important Must Have Things (they would be useful to me, but then my setup is probably pretty far on the weird side :smile: ). I mainly wanted to point out in the context of this topic that there are truly generic SysEx messages with meaningful transformations to and from, ones that competition cannot do (well, BomeBox probably does, but only through separately sold OSX/Win editor software)

Iā€™ll be happy to dig out and think/discuss further details if you decide to actually implement some of this of course.

1 Like

Thank you for the information. This really helps. Thereā€™s been quite a few requests for translating to SysEx, but rarely anyone provides the details of what is meant by that. :slight_smile:

So far these translations seem possible to implement, we are tracking suggestions like these, so we should get around to implementing them in a future firmware upgrade. :slight_smile:

4 Likes

Honestly thrills me that sysex parsing/ manipulation is being considered.

Itā€™s something Iā€™ve desperately been wanting to get more into via Max MSP, but I havenā€™t been able to find a good resource or entry point. If anyone would care to share any resources or insight, Iā€™d be immensely grateful.

Additional info: My current gear-list includes the Elektron Rytm, Four, and Heat. One example would be for pattern pushing (ie. shifting the phase of the trig lane per track), but also populating tracks with trigs is another. I know Jakob Penca (aka. Void on elektronauts) was able to do this and more with his Strom iPad app, but heā€™s gone silent and I havenā€™t been able to get ahold of him.

Okay, cool. From that perspectiveā€¦

Like said, there are a bunch of potential transformations beyond MMC in the Universal SysEx group (such as master volume/balance MSB mapped to some CC) but of course all the really interesting stuff is in the proprietary SysEx world. Which is of course also hopeless to generally transform.

The most simple possible approach would be just allowing ā€œarbitraryā€ raw hex data to be sent, but thatā€™s of very limited use. Youā€™d want to be able to map eg CC value to a parameter in the SysEx format for it to be useful, but even at its simplest form such a thing runs into the fact that many (maybe even most) manufacturers use a checksum on the SysEx data - checksum whose algorithm is naturally manufacturer/device specific. Which is game over for any generic SysEx parameter mapping attempt.

I think the best hope to do something beyond Universal SysEx would be implementing manufacturer-specific SysEx message ā€œtemplatesā€. Manufacturers tend to stick to similar systems in their products, for example my four Roland synths from as many decades, all follow a common <ids> <cmd> <address> = <value>, <checksum> scheme for their SysEx parameters. If such a format could somehow be supported even in limited manner, it would cover a lot of parameters in lot of synths.

More specifically, the format to set parameters inside F0 - F7 is:

  • Roland ID (42)
  • One byte device ID (similar to Universal SysEx device ID)
  • One byte model ID
  • One byte command ID (12 for ā€œdata setā€)
  • 3-4 byte address (three on oldest, four on newer)
  • One or more bytes of data
  • One byte checksum

In the address part, the first word is ā€œbankā€ and the second one address within, bank 0 being system parameters and others being parameters of patches and performance sets.

So entertaining the idea of limiting it to just simple 0-127 valued system parameters (for simplicitys sake), an alleged Roland-specific ā€œset system parameterā€ transformation from a CC/Note on would require the following arguments as the rest are known/calculated:

  • device id byte
  • model id byte
  • address MSB/LSB bytes

Considering what it unlocks, that doesnā€™t look so bad at all. If bank MSB/LSB were to be added, it would unleash pretty much every parameter there is in Roland synths AIUI and still not look too bad. Which would be huge, but whether it makes sense for MidiHub to implement is an entirely different question.

AIUI similar schemes can be found in other manufacturers SysEx (obviously nobody will want to design such a system from scratch for every device you make), so probably quite a lot of ground could be covered by covering a few major manufacturersā€™ SysEx schemes. How feasible any of this would be in reality, I wont try to even guess.

Oh and obligatory disclaimer: the above is just gathered from looking at a bunch of manuals on a lazy Saturday morning, I havenā€™t actually tried it at all, or double checked for mistakes. So caveat emptor etc, itā€™s just a study to what might be possible in this space :slight_smile:

6 Likes

Iā€™ve gathered some specific SysEx message examples for controlling Yamaha FB01. It has become a bit of a long post but I wanted to try and be thorough.

These examples are specific to Yamaha FB01 which seems to have a fairly weird MIDI implementation. Iā€™ve imagined how some simple Pipes could potentially be implemented as well.

First, some easy examples, where a SysEx Command ID number and Parameter Value maps directly to a single Device Parameter.

Each message uses the following pattern (below). The symbol _ preceding a letter represents a number that needs to be configurable but can be static, * preceding a letter represents a number that is dynamic and would be mapped from a CC, for example but any incoming value as long as it has been filtered/translated accordingly first. Iā€™ve made these bold in the examples because they are really the only thing that needs to change from one message to the next once the Pipe has been configured.

Numbers preceded by x are in hexadecimal and padded with a leading 0.

i.e. x0F == 15, xF0 == 240

xF0 [ Start ]
_M  [ Manufacturer ID ]
x75 [ Sub Status ]
_S  [ System Channel Number ]
_I  [ Instrument ID ]
_C  [ Command ID ]
*V  [ Parameter Value ]
xF7 [ End ]

So for example the Command ID for setting Polyphony is x00. A message formed with the following bytes would change Instrument 1 Polyphony to 8:

xF0 x43 x75 x00 x18 x00 x08 xF7

The following would set Instrument 3 polyphony to 1:

xF0 x43 x75 x00 x1A x00 x01 xF7

The Parameter Value can be mapped directly from the CC number, assuming the CC number can be clamped to a certain range, 1 - 8 for example here. Range Filter looks like it would achieve that.

Something to note about the Instrument ID here is that it is actually calculated from Instrument Id + 23. Who knows why? In an ideal world we would be able to set Instrument ID as 1, 2, 3 for example have Midi Hub figure out what the numbers to be sent are (18, 19, 1A) combined number is but for now a single user configurable value would suffice. Of course it would be fun to be able to control at least Instrument ID as well even if that is within a predefined offset range.

So far so simple, here are some other easy examples.

LFO On/Off Command ID is x0A. For some reason On is 0 and Off is 1ā€¦ Some way of mapping a CC to the appropriate value here would be good.

Set Instrument 1 LFO On:

xF0 x43 x75 x00 x18 x0A x00 xF7

Set Instrument 1 LFO Off:

xF0 x43 x75 x00 x18 x0A x01 xF7

Examples for Octave Transpose, which is Command ID x07.

The range is from -2 to +2, each of the values is sent as 0 to 4. Straight mapping of CC to value with the Range Filter seems fine here.

Set Instrument 1 Octave Transpose to -2 (minimum value):

xF0 x43 x75 x00 x18 x07 x00 xF7

Set Instrument 1 Octave Transpose to 0:

xF0 x43 x75 x00 x18 x07 x02 xF7

Set Instrument 1 Octave Transpose to +2 (maximum value):

xF0 x43 x75 x00 x18 x07 x04 xF7

Potential Pipe Idea
A simple Pipe implementation that would facilitate the above message style in is described below. I havenā€™t explored the Hub Editor very thoroughly but I think this would be achievable.

The Pipe would alow definition of the message structure and take a single incoming value. Start and End of SysEx messge will never change so they could be appended automatically. By having the message structure configurable it will support any given Manufacturer ID etc.

For example letā€™s Name the Property in the Pipe ā€˜SysEx Bytesā€™ and the Value would be a text box allowing entry of comma separated list.

The example of setting Instrument 1 Polyphony would look like this; 67, 117, 0, 24, 0

SysEx Bytes: 67, 117, 0, 24, 0

Those decimals are the same as the hex in the example (x43, x75, x00, x18, x00), some syntax for optionally handling hex numbers in the Property Value field would be nice here.

The incoming value would be used to form the final byte before the SysEx end byte, so the same example would look like this (input -> [output array])

 8 -> [240, 67, 117, 0, 24, 0, 7, 247]

Hopefully what Iā€™ve written so far makes sense, Below Iā€™ll describe some of the more difficult cases.

For the most part, a similar Pipe implementation as described above would work, with some additional configuration for the Paramater Value Bytes.

This time there are two Parameter Value bytes that are sent, the SysEx structure looks as follows.

xF0 [ Start ]
_M  [ Manufacturer ID ]
x75 [ Sub Status ]
_S  [ System Channel Number ]
_I  [ Instrument ID ]
_C  [ Command ID ]
*V  [ Parameter Value Low ]
*V  [ Parameter Value High ]
xF7 [ End ]

First, a simple example, all on Instrument 1, controlling LFO Speed which is Command ID x48. The actual Command ID as listed in the manual x0C but it needs to be offset when sending as SysEx, I think itā€™s reasonable for the user to specify the offset value rather than require Midi Hub to work this out as things are likely complicated enough already.

Set LFO Speed to 1:

xF0 x43 x75 x00 x18 x48 x01 x00 xF7

Set LFO Speed to 5:

xF0 x43 x75 x00 x18 x48 x05 x00 xF7

Set LFO Speed to 16:

xF0 x43 x75 x00 x18 x48 x00 x01 xF7

Set LFO Speed to 32:

xF0 x43 x75 x00 x18 x48 x00 x02 xF7

Set LFO Speed to 123:

xF0 x43 x75 x00 x18 x48 x0B x07 xF7

Set LFO Speed to 128:

xF0 x43 x75 x00 x18 x48 x00 x08 xF7

Set LFO Speed to 255:

xF0 x43 x75 x00 x18 x48 x0F x0F xF7

For each of the above, the following bitwise operations are used, where parameterValue is the LFO Speed

Parameter Value Low; parameterValue & 15
Parameter Value High; parameterValue >> 4

Potential Pipe Idea
The Pipe implementation for this could look similar to the previous implementation example with an additional two Properties ā€˜Data Low Functionā€™ and ā€˜Data High Functionā€™ where the Value would be a text box where the bitwise operations can be defined.

For the two Parameter Values above a Pipe would have the following Properties (values are in decimal);

SysEx Bytes: 67, 117, 0, 24, 72
Data High Function: & 15
Data High Function: >> 4

or as hexadecimal

SysEx Bytes: x43, x75, x00, x18, x48
Data High Function: & x0F
Data High Function: >> x04

Results for the examples as defined above (input -> [output array])

1 ā†’ [240, 67, 117, 0, 24, 72, 1, 0, 247]
5 ā†’ [240, 67, 117, 0, 24, 72, 5, 0, 247]
16 ā†’ [240, 67, 117, 0, 24, 72, 0, 1, 247]
32 ā†’ [240, 67, 117, 0, 24, 72, 0, 2, 247]
123 ā†’ [240, 67, 117, 0, 24, 72, 11, 7, 247]
128 ā†’ [240, 67, 117, 0, 24, 72, 0, 8, 247]
255 ā†’ [240, 67, 117, 0, 24, 72, 15, 15, 247]

Iā€™m wondering if some syntax in the ā€˜SysEx Bytesā€™ property to express that the bytes refer to another Property in the Pipe might be useful. For example if in one of the above we wanted to change the Instrument ID byte the Properties in the Pipe might look like this. It would mean the Pipe needs to take multiple input values (referred to as arguments below).

SysEx Bytes: 67, 117, 0, 24, {Instrument ID}, {Data Low}, {Data High}
Data Low: {$argument1} & 15
Data High: {$argument1} >> 4
Instrument ID: {$argument2}

This would look something like this in practice

123, 72 ā†’ [240, 67, 117, 0, 24, 72, 11, 7, 247]

OK probably not so difficult after allā€¦ so far! The Command for controlling LFO Speed maps directly from a single Device Parameter so itā€™s actually straightforward.

There are also what Iā€™ll call a ā€˜compoundā€™ commands, which map a Command Parameter Value from two or more Device Parameters sent as a single message with one Command ID.

An example of a compound command is one relating to Feedback Level and Algorithm Number. In this case both parameter values are combined and then split again into the Value Low and Value High before being sent with the Command ID of the compound command.

There would need to be a way of keeping track of the ā€˜last sent parameter valuesā€™ because the Low and High bytes for changing the Feedback Level with Algorithm 1 set will be different to bytes for changing Feedback Level with Algorithm 5 set.

I will try to add some examples of this too. Perhaps youā€™re already familiar with the problem and something like the proposed DX7 interface pipe would already cover it at least the tracking of the values. Could you describe a bit more detail of what youā€™re thinking with the interface pipe? Iā€™m not familiar with DX7 SysEx and would hope itā€™s not as crazy as FB01 but I would not be too surprised if it was! :slight_smile:

Potential Pipe Idea
Another Pipe that would be useful is one that just sends a pre defined SysEx message. For example you could store a list of the bytes that make up the message in the Pipe and have it after some Pipes that filter out channel, program message or CC number for example, the sent message need not have any dynamic content at all. This would be the most simple way of providing at least some basic SysEx control.

5 Likes

Thank you for the detailed write up, this will be definitely helpful when working on this.

For DX7 pipe, I thought simply having a ā€˜generatorā€™ pipe (one that you place as the leftmost pipe in the editor), and it would simply have a large number of MIDI mappable parameters in its properties. Youā€™d then map some MIDI controller to its controls, and itā€™d be generating appropriate MIDI messages for DX7 synths.

Such pipes would mostly be meant for particular hardware devices and probably provide little reusability for other kind of synths, but theyā€™d be convenient for interfacing with the devices theyā€™re meant for.

The pipes you describe sound like would be useful in general cases.

3 Likes

Hi everybody,

Yes, a generic object, like an editor container, for creating sysex streams and then execute them with either a CC and itā€™s parameters, or in direct simple cases, a Program Change, could be one way of doing things,

I am a live keyboard player, and would love to use something like the UNI CC version of Synth Controller ( https://www.stereoping.com/synth-controller/?lang=en ), or any other standard CC hardware transmitter, and map the CC setup for the controller, to something more interesting, and by this be able to control things in deep in my rack synth modules, simple using the controller or my master keyboard workstation.

I just downloaded the app some 15 minutes ago here, so I have only snoozed through things, but is there already a way to create bank select messages for program change, and trigger these with an incoming message in some way?

I wasnā€™t in the kickstarter, downloaded the app to fiddle around, but canā€™t wait for you to start selling the units to the rest of us!

KUTGW!!!

Cheers
JF

Are you thinking of doing the same for any of the other FM synths? TG77 / SY77 would be an incredibly powerful one for this kind of thing, Iā€™m a newb in this area but maybe there could be a Pipe for CC > Sysex that had something like instrument definitions that many sequencers have where we can specify the messages that our particular synth uses? I have no clue here but would be very interested to see where all this ends up! :slight_smile:

2 Likes

A CC to Sysex string pipe would be awesome. Right now, I use Bome MIDI Translator to do exactly that, to convert CC to Sysex values on my Korg EX-8000 and DSS-1, which only support basic CC natively.

So I run my Cirklon through USB to my Mac running Bome, which then spits out Sysex and merges it into the MIDI stream to the EX-8000. Itā€™s clunky but it works. I thought of buying a Bome Box to do it, but the Midihub has more DIN pairs and flexibility.

3 Likes

Likely yes, depending on how well DX7 goes :slight_smile:

Could you provide some cc->sysex message examples of how this works? This would help make sure we get it right for more use cases when designing the feature. :slight_smile:

2 Likes

Have a look at page 35 in the manual for the Novation SL MKII controller, thatā€™s a good start.

And overall, that series controllers supports quite a variety of options for sending cc/rpn/nrpn/msb/lsb etc.

1 Like

Alright, so it appears a fellow Elektronaut successfully decoded the A4ā€™s sysex format, and I figured Iā€™d share it here for posterity. :slightly_smiling_face:

Edit: Also just found an even more thorough decoding of the Elektron Analog Rytm on github. :slight_smile:

3 Likes

Apologies if this was already covered - I searched but couldnā€™t turn anything up.

I would find it very useful to be able to convert messages like Start or CC values into MIDI Notes. I have a (very annoying) piece of gear that canā€™t receive Start messages but can be controlled using MIDI notes, and a piece of gear that can send everything BUT MIDI notes. Iā€™d like to get them talking to each other if possibleā€¦