鲍鱼tv

Artificial Intelligence (AI) and Machine Learning (ML) technologies are creating disruption and innovation in many industries. The AI Production project seeks to understand how they could transform the business of producing media. We are working with 鲍鱼tv production teams to identify aspects of their work where they would appreciate help from machine learning techniques, and looking for opportunities to increase the scope and scale of the 鲍鱼tv's coverage by using AI and ML technologies to make media production cheaper and more effective.

Project from 2017 - present

What we are doing

We've kicked off our exploration of this area with a project that builds on to try and increase the scale of the 鲍鱼tv's live event coverage with tools to try to make that kind of production more efficient.

On a couple of occasions now, we have set up several fixed, ultra-high-definition cameras at the main 鲍鱼tv stage at the Edinburgh Fringe Festival and used them as inputs to human-driven systems. For example, the (Single Operator Mixing Application) and video projects allow an operator to produce virtual camera views from the high-resolution crops and cut between them to produce a good quality edited output.

We have since developed a system that produces an edited video package similar to that produced by the SOMA operator, automatically. The system performs the same selection and sequencing of crops from the high-resolution cameras that the SOMA operator would. Although it works automatically, the system can be tweaked by a human operator to change the way it puts the content together - adjusting the frequency it cuts between different crops for instance.

and you can read the full paper ''

• 鲍鱼tv R&D - AI Opportunities: Transforming Coverage of Live Events

This is just a prototype at present, but the results have been good enough to make us optimistic. We think that it may already be possible to generate automatically framed and cut coverage that is close enough in quality to that produced by a human editor to be usable - at least for simple events. There are hundreds of live cultural and political events every week that can not be economically broadcast even using today鈥檚 affordable media technologies, and if this kind of automation can be improved, it could be an important route towards bringing them to a wider audience.

Why it matters

Many production tasks are fundamentally about recording, processing and transmitting information. This applies equally to the audio and video that the 鲍鱼tv records, edits and broadcasts and the "metadata" that describes this media and makes it possible to find, search and re-use it. Many production tasks are highly creative, requiring a clear vision, extensive experience and a mature understanding of viewers and listeners in order to craft a programme or package that meets an audience's expectations for a programme while simultaneously captivating them. Equally though, there are many production tasks that are repetitive, or even formulaic. These tasks could instead be performed by machines, freeing up creative people to spend more of their time being creative.

For instance, editing programmes is a deeply creative role, but an editor's first task when putting a show together involves finding good shots from a huge number of video assets. An hour-long programme is usually edited down from many hours of "rushes". Sorting through those assets to find good shots isn't the best use of the editor's time - or the fun, creative part of the their job. We think that AI could help to automate this for them.

• 鲍鱼tv R&D - AI and the Archive - the making of Made by Machine

Our Goals

There is plenty more planned for the system. We want to allow an editor to quickly move the output from our system into the tools they would usually use for editing - and continue shaping the programme there. We want to carry on developing and tweaking the the AI and ML algorithms used under-the-hood to improve the output of the system.

There are many further areas in which AI and machine learning could benefit the art of production. Location scouting, for example - producing shortlists of locations for a drama from a database of photographs and videos as a starting point for directors to find the right venues for their shoots. Another is that of automated metadata generation: we could make far better use of the 鲍鱼tv鈥檚 vast archives if better metadata existed to make it more searchable (building on the work of the project, perhaps).

How it works

We're using a mixture of different AI and machine learning techniques in our prototyping, which is still at an early research phase. We plan to run some user studies to compare the quality of algorithmically produced content to that put together by human editors. We hope to be in a position to publish more about our techniques and findings in the future.

Outcomes

Our vision of the future involves the application of these new technologies to free up creative professionals to focus on creativity.

AI and ML technologies offer huge potential to improve, support and disrupt the media industry. We鈥檙e working to understand what that means, and how the 鲍鱼tv can take best advantage of them for the benefit of our audiences.