Controlling a live production: the 4 layers of AV control

Control is often the blind spot of audiovisual projects. Cameras, the switcher and the production software get chosen first, and only at the end does anyone ask who presses what and how the command travels to the machines. Yet it is this control layer that decides, in operation, whether a production is comfortable or stressful. The underlying trend is clear: point-to-point physical control (RS-232, RS-422, GPI) is giving way to IP systems that are bidirectional and able to report device state back.

One idea runs through this whole article: the right control system is not the one that drives the most equipment, but the one that lets the operator trigger the right action, at the right moment, with reliable state feedback.

What are we actually talking about?

The word “control” covers three different things: the surface the operator touches, the logic that links the machines together, and the language those machines speak. Confusing these levels leads to comparisons that make no sense. A Stream Deck, NDI®, OSC and VISCA cannot be compared directly, because they do not belong to the same layer. To reason cleanly, it helps to separate four layers.

The 4 layers of control

The operator surface layer is what you physically handle. It includes programmable button surfaces, touch interfaces, PTZ joysticks or simple control keypads. Among the tools commonly seen: Stream Deck, X-Keys, Loupedeck or TouchOSC. Their role is not to talk to the machines directly, but to offer a command that stays readable in a live situation.

The middleware layer makes the link. It is what turns a button into a sequence of actions across heterogeneous equipment: change an input, recall a preset, start a recording, switch a router. Among the automation tools commonly seen: Bitfocus Companion, Central Control, Chataigne, Node-RED or QLab. This layer is often invisible to the operator, but it is what makes the system coherent.

The generic protocol layer gathers the languages that are not specific to a trade: OSC, MIDI, HTTP, WebSocket, TCP/UDP. They are flexible and widely used, but they define no universal vocabulary: the same command means nothing from one application to another until the mapping has been documented.

The AV-specific protocol layer is the one that controls precise functions of the equipment: pan, tilt, zoom of a camera, tally of a switcher, connection of an ST 2110 router. This is where VISCA, ONVIF, TSL UMD, NMOS, Ember+ or GPI/GPO live. Surfaces and automation tools may change, these protocols remain the real point of contact with the hardware.

An example makes this logic concrete. In a training room, a single button can recall a camera preset, switch a scene in the live software, start a recording and turn on a tally. The operator sees only one button. Behind it, several layers coexist: the surface that receives the press, the automation that orchestrates the sequence, the control protocols that carry each order, and the actual functions triggered on the machines.

On the camera side, some modern PTZ cameras, including BirdDog solutions, can depending on the model be controlled via NDI® PTZ, VISCA over IP or third-party software interfaces.

The underlying trend: everything becomes IP-controllable

The general move is the shift to IP. Where control used to run over a dedicated serial link and a dry contact, it now relies on the network: bidirectional commands, state feedback, automatic source discovery. This changes how an installation is designed, because the network carries the control as much as the video.

NDI® illustrates this evolution well, provided it is not mislabelled. NDI® should not be reduced to a control protocol: it is first an audio and video transport ecosystem over IP, also able to carry metadata useful for control, tally or PTZ steering depending on the equipment. Distinguishing it from OSC, VISCA or ONVIF, which are command protocols, avoids a common confusion.

This is the moment to set out a distinction that structures the whole discussion: transport describes how the video travels, control describes how the machines are commanded. They are two separate subjects. For transport fundamentals, see the Understanding NDI guide and the NDI vs SDI article; for transport protocols in the broad sense, the NDI, SRT, RTSP, RTMP guide. This article deals only with the control layer.

The trap: “it responds” is not “usable live”

This is the most useful point for a decision-maker. The fact that a device responds to a command in a test does not guarantee it is usable in production. Several checks separate the demo from real operation.

State feedback comes first. A button should ideally reflect the real state of the machine, not just send an order into the void. Without feedback, the operator works blind. Then come latency, redundancy, behaviour after a reboot (do sessions and connections re-establish on their own?) and the consistency of presets across equipment. Network security closes the list, with a control network kept separate from the video network. A system that passes these points is usable; a system that merely responds is not necessarily.

A guide by need

The right starting point is not the technology but the need. The table below gives an orientation by family of solution, to be adapted to the equipment in place and the operator’s skills.

Field recommendation

For a modern demonstration environment, a simple architecture is enough, without locking in a single combination: a button surface (such as a Stream Deck) connected to an automation tool (like Companion), on a dedicated IP network, with PTZ presets in VISCA or NDI® and control of the live software or the switcher as needed. To go further, add a web tablet or an OSC interface, a PTZ joystick if the camera is used for real, a control VLAN separate from the video network, fixed IP addresses or DHCP reservations, and clear documentation of ports and protocols.

The security of this layer is often forgotten. Control interfaces are protected like the rest: a managed network, limited access, passwords when the equipment allows it, and no direct exposure to the Internet. For remote access, go through a dedicated VLAN or a VPN.

In short

There is no best control system in absolute terms, only a system suited to a context. The useful reasoning starts by separating the four layers: the interface you touch, the automation that orchestrates, the generic protocols and the AV-specific protocols. It continues by starting from the field need, then choosing technologies as examples, never the other way round. This first article sets out the general framework. Control surfaces and automation are the subject of the second part; the control protocols are detailed in a dedicated reference guide.

In this series

1. Controlling a live production: the 4 layers of AV control
2. Control surfaces and automation: from button to action in live production
3. AV control protocols: VISCA, OSC, NDI® PTZ, ONVIF and the rest

To connect this framework to a production type, see the Solutions pages (broadcast, corporate, sport, education). To identify the right configuration for a project and the matching equipment, the Where to Buy page helps find a reseller, or get in touch for a technical discussion.