EVOLUTION-MANAGER

Edit File: README_NETJACK2

------------------------------- NetJack2 for Jack2 ------------------------------- This release includes a version of netjack designed for jack2. Indeed, the original concept has been completely redesigned to better fit to the Jack2 architecture, but also in order to provide additional capabilities, and ultimately a greater robustness. This document describes the major changes between those two systems, then a simple how-to for setting up a basic usage of 'netjack2'. ------------------------------- Major changes and architecture ------------------------------- The biggest difference between netjack1 and netjack2 is the way of slicing audio and midi streams into network packets. For one audio cycle, netjack1 used to take all audio and midi buffers (one per channel), put butt all of them, then send it over the network. The problem is that a network packet has a fixed maximum size, depending on the network infrastructure (for 100mb, it reaches 1500bytes - MTU of the network). The solution is then to slice those buffers into smaller ones, and then send as many packets as we need. This cutting up can be done by network equipments, but it's more efficient and secure to include it in the software data management. Still this slicing brings another issue : all the packets are not pleased with any emission order and are unfortunately received in a random order, thanks to UDP. So we can't deal with data as it comes, we need to re-bufferize incoming streams in order to rebuild complete audio buffers. In netjack2, the main idea is to make this slicing depending on the network capabilities. If we can put only 128 complete audio frames (128 samples for all audio channels) in a network packet, the elementary packet will so carry 128 frames, and in one cycle, we will transmit as many packet as we need. We take the example of 128 frames because it's the current value for 2 channels. This value is determined by taking the maximum 'power of 2' frames we can put in a packet. If we take 2 channels, 4 bytes per sample (float values), we get 8 bytes per frame, with 128 frames, we now have 1024 bytes, so we can put these 1024 bytes in one packet, and add a small header which identify the packet. This technique allows to separate the packets (in time) so they can be received in the order they have been emitted. If the master is running at 512 frames per second, four audio packets are sent per cycle and the slave deals with them as they arrive. With gigabytes networks, the MTU is larger, so we can put more data in one packet (in this example, we can even put the complete cycle in one packet). For midi data, netjack1 used to send the whole buffer, in this example, 512 frames * 4 bytes per sample and per midi port. Those 2048 bytes are in 99% of the time filled to a few bytes, but rarely more. This means that if we have 2 audio and 2 midi channels to transmit, everything happens as if we had 4 audio channels, which is quite a waste of bandwidth. In netjack2, the idea is to take into account that fact, by sending only the useful bytes, and not more. It's completely inappropriate to overload the network with useless data. So we now have : 99% of the time one midi packet (of a few dozen of bytes), followed by four audio packets (in this example). This way of separating audio and midi is quite important. We deal here with network transmissions, and also need to be 'realtime'. We need a system which allow to carry as many audio and midi data streams as we need and can, as if the distant computer was in fact a simple jack client. With all those constraints, we can't avoid packets loss. The better thing to do is to deal with it. But to loose an audio packet is different from skipping a midi one. Indeed, an audio loss leads to audio click, or undesirable, but very short side effect. Whereas a midi data loss can be completely disastrous. Imagine that we play some notes, with sustain, and we loose the sustain 0 value, which stops the effect. The sustain keeps going on on all following notes until the next 'sustain off' event. A simple missing byte can put all the midi system offside (that's the purpose of all the big PANIC buttons on midi softwares...). That's why we need to separate audio (more than one per cycle) from midi (one packet at 99% of the time). If we loose an audio packet, we probably still have an available midi packet, so we can use what we received, even if some audio is missing. Those audio and midi packets are preceded by a synchronization packet, which will make the slave directly synchronized on the master's cycle rhythm. This packet also carries transport data. Thus it's actually possible to synchronize also transport. This feature goes a little further than in netjack1. The idea here is to make every computer of the network fully synchronized on the master's transport. That means the master needs to know the state of every slow sync clients of each of its slaves. The master can now manage the transport state (especially the 'rolling' state) of each slave thus the main transport waits for the last slow sync client before turning 'rolling'. By doing this, the transport can start (roll) in the same cycle for every computers managed by the master. The second main difference between netjack1 and netjack2 is the way the two computers (master and slave) synchronize their parameters and launch. In netjack1, once the slave configured (by the command line) and launched, it was waiting for the first incoming packet to synchronize (launch its first audio cycle) then run. The two computers needed to be configured separately but with the same parameters to run correctly. In netjack2, the only thing you have to set for the slave is its number of in/out midi and audio channels. No more need to choose and set parameters depending on the master, they are automatically determined and communicated to the slave. This first synchronization step uses a multicast communication, no more need to know by advance all the IP addresses. The slave says on a multicast address "hey, I'm available". A master get the message, and communicate parameterers to the slave. Once synchronization done, data transfers can start. Moreover, the master being still listening on the multicast address, it can catch other slaves and manage them (create a jack client to communicate with the slave, and neatly close everything when the slave is gone). The loaded internal client is no longer only an interface for the slave, like in netjack1. It's now called 'network manager', it doesn't deal with audio or midi, just with some kind of 'network logistical messages'. The manager automatically create a new internal client as soon as a new slave is seen on the network (by sending messages on the multicast address the manager is listening on). This manager is also able to remove one of its internal client as soon as a slave has left the network. This conception allow a complete separation of audio exchanges from parameterers and management. The 'unloading' of the internal client (the manager) will cause a full cleaning of the infrastructure. The jack clients are all removed from the server, the slave are all turned available again, ready to be caught by another master etc. When a slave quits, it's also automatically removed from the manager's slaves list. ------------------------------- How-to use this ? ------------------------------- Netjackmp is very simple to use. On the master's side, an internal client deals with the slaves, and the slaves themselves are classical jack servers running under a 'network audio driver'. The difference between the two versions is that the master now has a manager, which takes care of the slaves, while listening on the multicast address and create a new master as soon as a slave is available. But everything is transparent to the user, that's why it uses multicast (someone says "hello", and anyone who wants to hear it just has to listen). So, just compile and install Jack2 as you are used to, on linux, using './waf configure', './waf' and './waf install' as root. On macosx, you can use the xcode project. On Windows, you can use the Code::Blocks workspace (you also have a small script to make an all in one installer). On the master, just launch a classical jack server, the period size doesn't matter. Then, load the network manager using jack_load : 'jack_load netmanager' This will load the internal client, which will wait for an available slave (see the message window on QjackCtl - or the console output). If you want to listen to a specific multicast socket, you can add some options. To specify a complete command line, you can use : 'jack_load netmanager -i"-a xxx.xxx.xxx.xxx -p udp_port"' If you set another multicast address or port, you have to set the same on the slave's side. The default value should be working in many cases. On the slave, just launch a new jack server using : 'jackd -R -d net' As in a standard backend in Jack2, you can use '-S' (synchronous mode). The asynchronous mode (without '-S') allows to send the computed data during the next cycle. In synchronous mode, data are sent back at the end of the cycle, that means after the process. You can specify some options, like '-n name' (will give a name to the slave, default is the network hostname), '-C input_ports' (the number of master-->slave channels), '-P output_ports' (the number of slave-->master channels), default is 2 ; or '-i midi_in_ports' and '-o midi_out_ports', default is 0. If you set multicast address or port on the master, you can add '-a xxx.xxx.xxx.xxx' and '-p udp_port'. You can also use others network mode, with '-m' option (fast, normal or slow). Fast mode allow a zero latency transmission. This mode means the master waits for its returned data from the slave in the current cycle. This mode is appropriated for 'small' transmissions (only a few channels with a light process on the slave). Normal mode brings one cycle latency. It allow a normal use of the network. Slow mode (default) allows a full usage of the whole system. Slow mode brings a two cycles additional latency. This mode allows to send a lot of data on the network, but it also takes into account some process time on the slave, thus the data aren't expected on the master before two cycles, so it's not necessary to wait for them. For additional informations, you can go to the NetJack2 Wiki at : http://trac.jackaudio.org/wiki/WalkThrough/User/NetJack2. ------------------------------- What's next ? ------------------------------- The development of netjack2 continues and some things are always moving... If you use it, please report encountered bugs, ideas or anything you think about. If you have any question, you can subscribe the jackaudio developers mailing list at http://www.jackaudio.org/ or join the IRC channel '#jack' on FreeNode.