Colleagues from ±«Óătv R&D and ±«Óătv Sport, along with our outside broadcast suppliers Arena-TV, have been working hard to bring the FA Cup to our viewers in Ultra HD, with high dynamic range and wide colour gamut. In this blog, we aim to explain how our workflows have progressed since last year, what went well, what has proved difficult and what our future roadmap is.
Last year.
Large scale trials of large production started last year with the Royal Wedding of TRH The Duke and Duchess of Sussex and carried on throughout the summer with the FIFA World Cup and Wimbledon. In all cases, the Ultra HD production was a separate, parallel workflow requiring two vision mixers working from a single control signal. Cameras either worked in a dual HD/UHD HDR output mode or HD-only cameras were upconverted to UHD HDR to provide the second signal. In order to ensure that our HD signal (the signal with the most viewers) was correct, the vision engineers adjusted the cameras whilst looking at this HD signal, with only one engineer checking that the UHD HDR output was acceptable.
The camera control is split between the camera-person who controls the framing and focus and the vision engineers who monitor the signal and adjust the controls which affect the “look” of the image. Traditionally this has meant mainly the exposure control and black level of the signal to maintain a consistent brightness but, as HDR starts to reduce the need for exposure control, they now have more ability to control hue, saturation and other controls to enable them to craft a specific look which remains constant as ambient lighting conditions change. Some controls apply equally to both HDR and SDR output signals (e.g. iris) and some are different (e.g. HDR black gamma vs. SDR black stretch).
The effectiveness of this solution depends on how well the camera shadows and midtones match in the HD and UHD HDR signal outputs and on how well the camera controls track between the two output signals.
Now.
We’ve now moved to the next stage – to produce the UHD HDR image as the primary image and create a downmapped HD SDR signal from the UHD HDR. There are still limitations on what can be done, HDR reference monitors are still very expensive and so they are not yet commonplace in outside broadcast scanners, so we still need to control the output of cameras whilst looking at an HD SDR signal. This ensures the very best pictures for our HD SDR audience. A single HDR monitor is then used to check we’re happy with the HDR output.
For the three-match series, we planned with ±«Óătv sport and our OB provider to test early stages of the roadmap we have to move to full UHD HDR production. We decided to use the first match as a non-televised test event (Millwall FC vs. Brighton and Hove Albion FC) and planned to use a hybrid of what worked last year – racking on the camera SDR output using a fixed HDR signal offset. The HD pictures on ±«Óătv One were, for the first time, created directly from the Ultra HD HDR output of the vision mixer using a new downconversion lookup table (LUT) that we designed, implemented in third-party hardware.
The new downconverter we trialled is designed to be hue invariant – that means that the colours on the HD picture match the hue colours on the UHD HDR picture – so the wide colour gamut is compressed into the limited HD gamut without a loss in hue accuracy. However, the vision engineers were uncomfortable with the transmission output being a different signal to the one they were monitoring.
For the second trial match (Manchester City FC vs Brighton and Hove Albion FC) a change was made so that the camera operators were looking at the HLG Compatible SDR Image. The operation performed by the downconverter in use is very similar to the HLG signal being fed in to a BT.2020 colour space non-HDR monitor with a gamma setting of 2.2. This means that the vision engineers were watching the HLG signal which was being used to create the ±«Óătv One programme processed in a way that closely matches the output of the downconverter – there were some differences between what the vision engineers were viewing and the HD output, but small enough to allow confidence that the output signal for both UHD HDR and HD versions were looking good.
This is quite a change to the traditional workflow – and after some puzzled looks at a test day with Arena-TV at their Redhill Aerodrome base, we got to a point where we were happy the system would work and be much easier than that used at Millwall.
For the FA Cup final (Manchester City FC vs Watford FC), a further improvement was made – using a small hardware LUT box behind each reference monitor, the engineers were able to look at the exact same downconverted image that was being sent to ±«Óătv One, so the slight discrepancy between the downconversion and the compatible image was eradicated. Thus, by monitoring the actual HD output with no fixed offsets in the signals, the operators could use the full range of UHD HDR painting controls confident that what they were seeing was going to air on ±«Óătv One. This allowed the vision engineers to push the camera saturation and colour controls to allow the HD output to match current sport production norms, whilst simultaneously producing great UHD HDR content for distribution by iPlayer. The vision engineers commented that the larger dynamic range of the camera outputs allowed them to spend more time crafting the look they were after as there was less need to react quickly to lighting changes.
As the FA Cup host broadcaster, we also had to provide match feeds and Iso feeds of pitch-facing cameras to other broadcasters, to the venue for use on stadium screens and to the Football Association for use with the Video Assistant Referee (VAR) and goal-line systems. (“Iso” a broadcast term shortened from “Isochronous” – “At the same time”.) The Iso feeds need to be in the HD format and need to be converted in a way which allows the other broadcasters to mix in their own pitch-side camera feeds. To do this, we converted all the UHD HDR match cameras to HD using a “scene-referred” converter, ensuring that the downconverted camera output matched what would be output by an HD camera pointing at the same scene. This is a slightly different requirement to the “display-referred” conversion used to generate the ±«Óătv One video signal, which matches, as closely as possible given the limitations of HD technology, what the vision engineers see on their monitors to what ends up being displayed on TVs in the home.
The Future.
There’s still some way to go before we get to the point where vision engineers will be using HDR reference monitors at all points in the vision processing chain and all cameras will be able to produce HDR images. Until then we are progressing along a roadmap and the FA Cup trials have been a fantastic opportunity to refine our ideas for Ultra HD production using currently available equipment on a large scale.
There are still some improvements to be investigated:
Replay servers are usually run in a 1080i25 or 1080p50 8-bit mode to maximise storage rather than the 1080p50 or 2160p50 10-bit modes that we’d prefer. Until this can be increased, replays will need to be recorded in non-HDR BT.709 video and upconverted to UHD HDR for the programme.
Camera controls are not necessarily consistent between the HD and UHD HDR output channels and work differently between manufacturers. For large scale productions which are required to provide both UHD HDR and HD camera signals to clients and use cameras from multiple manufacturers, it is important that the camera controls work consistently for both.
Hue preserving downconverters are good, but further work is needed as some very saturated, very bright colours are changed by a large amount to bring them back in to the ITU-R BT.709 colour gamut. Further work is required to see if a minimal colour distortion model can be found.
We’re still learning about how to get the very best out of the new production technology; and hopeful that further trials will simplify and improve the production workflow. We also plan to work with a range of camera manufacturers to ensure that this new workflow is viable for all outside broadcasts. Whatever happens, we can expect some great content over the next year or so for our viewing public.