AutoPilot + Unreal Engine 5: Letting the DJ Direct the Camera

The DJ Is Already Directing

A DJ playing a peak-time set is making directorial decisions every eight bars: hold here, lift here, cut on the one. Those decisions are legible in the audio — the build is a build, the drop is a drop — but on most rigs they only reach the visuals if a second human is watching the first one and reacting.

Unreal Engine 5 makes that gap conspicuous. You have a virtual stage with real cameras in it, and the camera is doing nothing until somebody presses something.

AutoPilot closes it by treating the track's structure as the shot list. When CueSync arms a track it builds a fire plan from the analysed structure — where the intro ends, where the build starts, where each drop lands — and then fires those moments as they arrive, without an operator on GO. Point that plan at Unreal instead of at a lighting console and the DJ is directing the camera.

Two Channels, Not One

CueSync talks to UE5 through the Remote Control API — the same OSC surface Unreal exposes for any external controller. Everything arrives on a preset named CueSyncBus, on port 8001 by default, and it arrives on two channels that behave very differently.

The continuous stream is state. Beat position within the bar, tempo, phrase ID, an energy value from 0 to 1, an energy trend of up, flat or down, and a hue shift derived from the track's musical key. This is what you bind to material parameters and Blueprint variables — the thing that makes an environment breathe with the music rather than cut with it. It updates constantly and nothing bad happens if a single packet is lost, because the next one is already on its way.

The accents are events: CutToCamera, PlayCameraMove, TriggerDrop, BurstFX, SetScreenMedia, level load and unload. These are the directorial decisions, and losing one is not survivable in the same way — a missed cut is a shot that never happened. So they do not ride the same best-effort path as the stream: each accent carries a sequence number, is sent redundantly, and is de-duplicated at the Unreal end, so a dropped UDP packet costs nothing and a duplicate does not fire the cue twice.

Getting that split right is most of the design. Bind the wrong thing to the wrong channel and you either get a camera that stutters or an environment that only moves on drops.

The Virtual Stage Profile, Read Out Loud

CueSync ships an AutoPilot profile called Unreal Virtual Stage, built for the reference show level and centred on 128 BPM. It is worth reading as a piece of direction rather than a config:

• Intro ends → cut to camera 0, the wide. This is the frame the audience returns to all night, so it gets established first.
• Build starts → play the CranePush move. A build is euphoric and it travels; if the camera cut here instead, the drop would have nothing left to do.
• Drop → TriggerDrop plus a ten-unit BurstFX. The drop cue snaps from the wide into the beams, and the burst travels the truss rather than flashing everything at once.
• Final drop → the same, harder, then cut to camera 5.
• Fill section → cut to the crowd camera, no burst. A fill is punctuation, not a section.
• Outro starts → home to the wide and unload the beams world, so the next track begins clean.

And one entry that matters by being absent: breakdowns are not mapped at all. A breakdown is where the DJ is doing the work, and the correct camera decision is usually to leave the shot alone. A profile that fires on every structural moment it can detect is a profile that never stops talking.

Why Camera Cues Need Pre-Roll

A lighting cue can be reactive. You detect the moment, you fire, the fixture moves, and a few milliseconds of lateness reads as nothing at all.

A camera move cannot. A crane push that starts on the downbeat arrives at its framing a second later, which means the climax of the move lands after the climax of the music. Purely reactive camera cues read a beat late on screen even when the trigger itself was on time.

So the Unreal Virtual Stage profile arms its moments with four beats of pre-roll. The move begins before the drop so that it arrives on the drop. This is the single most important setting in the profile and the one most likely to be wrong in a profile you build yourself — if your camera work feels consistently slightly behind the music, this is why.

Setting It Up

1. In UE5, enable the Remote Control API and Remote Control Protocol OSC plugins, plus the CueSync for Unreal plugin, and create a Remote Control Preset named CueSyncBus.
2. In CueSync, add an Unreal Engine 5 output pointing at the machine running UE5 (port 8001 by default).
3. Bind the continuous addresses — Beat, Energy, EnergyTrend, HueShift, PhraseId — to the Blueprint variables and material parameters that should move with the music.
4. Expose your camera and effect functions on the preset so the accents have something to call.
5. Select the Unreal Virtual Stage AutoPilot profile, or copy it and edit the moment map for your own level.
6. Play a track and watch it in read-only mode first. The OSC monitor shows exactly what is being sent, which is much faster to debug than watching for a camera that did not move.

Where the Operator Still Sits

Hands-free is not unattended, and AutoPilot is built on that assumption.

You can pause it for a set number of phrases when you want the next section by hand, and it resumes on its own afterwards. You can fire any moment manually at full energy at any time, in any state. And because AutoPilot and manual cues share a coordination bus with asymmetric suppression windows, firing something yourself does not produce a double-hit when the automated moment arrives a beat later.

Rewinding is handled too: seeking back through a track clears the fired state ahead of the playhead, so replayed sections re-fire correctly rather than sitting silent because they already ran once.

What This Is Not

It is not timecode, and it is not frame-accurate. Position is being derived from a live performance, which is precisely why it works on an improvised set — but if you have pre-rendered content that must land on an exact frame, you want a clock, not a beat detector. Where Should Your Show's Timecode Come From? covers that decision.

It is also not a replacement for a VJ on a show that has one. It is a floor: the camera is always doing something defensible, so the operator's attention can go to the passages that actually need a person.

On timing: cue evaluation is held to a sub-5 ms budget on the cue-evaluation path, with the OSC send budget under 2 ms. What you see on the LED wall also includes Unreal's own render and output path, so end-to-end depends on your machine and your network.

One edition note: every CueSync edition includes all 15 output protocols, so Theatre can drive Unreal from cue lists — but the AutoPilot engine itself ships in the DJ and Production editions.

Further Reading

• Unreal Engine 5 Integration — the full OSC surface and setup
• Pioneer DJ Auto-Sync for Lighting — where the beat data comes from
• How to Sync Resolume to a DJ Set in Real Time — the same idea, media-server side
• How CueSync Detects Beats — what the structure analysis is doing

Unreal® and Unreal Engine® are trademarks of Epic Games, Inc. CueSync is an independent product and is not affiliated with or endorsed by Epic Games.