The Quality of Live Subtitling:

Acknowledgements

I owe an immeasurable intellectual and personal debt to Professor Agnieszka Szarkowska, who initiated my interest in audiovisual translation back in 2010 as I was setting sail on my academic voyage in her BA seminar. It is only thanks to her unwavering support and eternal patience that this research is finally seeing light. It is a light at the end of what was a very long tunnel but hopefully is just a beginning of our journey.

I would also like to thank Prof. Małgorzata Tryuk for her support as my first doctoral supervisor and all the intellectual guidance and inspiration I benefitted from as a participant of her PhD seminar.

My journey into media accessibility in general and live subtitling in particular would have been impossible without many years of great collaboration with Monika Szczygielska, who gave me an opportunity to get trained as one of the first Polish respeakers in 2013 and offered me support and inspiration ever since in many accessibility projects we embarked upon together.

My deep appreciation goes to my intellectual Muses: Prof. Pablo Romero Fresco, whose many works have served as milestones on my path and have heavily influenced my thinking (but also led me to spend many rather tedious hours on performing the NER assessment); Dr Gian Maria Greco, for our discussions on quality and accessibility that allowed me to gain new perspectives on many of the issues I discuss in this book; and Dr Breno Silva for his many attempts at making me appreciate the beauty of inferential statistics and for his invaluable help in analysing the results of this research.

I would also like to thank Joanna Pietrulewicz and Maria Łogińska, who helped initiate me in television subtitling back in 2011; Katarzyna Lalik, with whom I had the pleasure to take my first steps as a respeaker; my wonderful team of colleagues from Dostępni.eu, whom I had the great privilege to train and collaborate with and who helped me fulfil my personal dream of working again as a live news subtitler; live subtitlers and my dear colleagues who assisted me in recording, evaluating and re-thinking the quality of TV shows: Karolina Szeląg, Aleksandra Dobrowolska, Aleksandra Urban, Tomas Senda and Agnieszka Grabska; my colleagues from the Global Alliance of Speech-to-Text Captioning: Chris Ales, Jen Shuck and Kimberly Shea; as well as Zoe Moores and Natalie Fresno, for sharing their experience and expertise; my academic role models Dr Wojciech Figiel, Dr David Orrego Carmona and Dr Agnieszka Walczak, who guided me inside and outside of the academia; and Dorota Mierzejewska, for always sharpening my institutional intelligence and helping me navigate the meanders of administrative procedures.

My special thanks go again to Karolina Szeląg and Aleksandra Urban as well as to Eduardo Medina and Jazmín Granell. Without their help and assistance in juggling other projects, finishing this book would have been impossible.

Last but not least, I wish to thank my family, my partner Tomek, my mom Mirka and especially my dad Heniek, who did not live to see this finalized and whom I greatly miss. Thanks to them and to all my friends for putting up with me while I was working on this book.

Needless to say, I am solely responsible for any errors or inaccuracies.

Introduction

Let me remind you of a quote from “Alice in Wonderland”:

Anything is possible. You just have to know the ways.

(Live subtitling at the inauguration of the academic year at the Adam Mickiewicz University in Poznań, Poland. October 2021. Speech by the Rector, Prof. Bogumiła Kaniewska.)

We live in the age of information and access to the media is an important prerequisite for full participation in social life (Greco, 2018; Jankowska, 2020; Neves, 2005). For many years there have been numerous efforts to make the mass media more accessible to disadvantaged groups either by the use of audio description, which helps blind people enjoy audiovisual materials, or by the use of subtitles, which allow the Deaf and the hard of hearing to have improved access to audiovisual programs. Subtitling for the Deaf and the hard of hearing (SDH) now plays an important role in the education and entertainment of people with hearing loss (Greco & Jankowska, 2020; Neves, 2005; Zarate, 2021).

In many countries, the quantity of SDH available on TV has increased dramatically over the years, both for pre-recorded and live TV content; however, the quality of subtitling is not always satisfactory (Romero Fresco, 2021). For example, in Poland, the EU and the national legislation and regulations have led to a relatively large rise in the quantity of pre-recorded audiovisual content that is subtitled on Polish television. And while the Deaf and the hard of hearing viewers now enjoy a far greater access to the media, the users have long been advocating for more access to live TV content, especially news and current affairs programmes (Sacha & Kasperkowiak, 2012; Szarkowska, 2010; Szczygielska, 2019).

The provision of subtitling for live TV content has been hindered by a long-held belief among practitioners and broadcasters that live subtitling in Polish is impossible due to technical and organizational challenges involved (Künstler, 2008; Szczygielska, 2019), the belief that was even echoed by the Polish media regulator, the National Broadcasting Council (KRRiT, 2016). In this context, let us remember a scene from a Disney adaptation of “Alice in Wonderland” from 1951. In the film, Alice encounters a small door and shouts in exasperation, “I simply must get through!”, to which Doorknob responds, “Sorry, you’re much too big. Simply impassable”. “You mean impossible,” Alice asks. “No, impassable. Nothing’s impossible.”

When I first embarked on this research journey, my main goal was to show that live subtitling in Polish is not impossible; indeed, this door is very much “passable” with the right technology, methods and workflows. In the spirit of action research (Cravo & Neves, 2007), my objective was to prove that live subtitling in Polish can and should be applied in broadcasting settings to increase the accessibility of TV for the hearing impaired audiences. My plan was to review the different methods used to create live subtitling in an effort of identifying the ones that could be used in Polish and then testing them to find the best ones, i.e. the ones that can produce the best quality and the most value for the end user. However, in the early stages of working on this book, a twist of fate took me from attempting live subtitling in the lab and training respeakers in the classroom to doing the same in broadcasting studios as I participated in implementing live subtitling for one of the Polish broadcasters.

As of 2020, when I began this research, all the three main broadcasters in Poland (TVP, Polsat and TVN) had already introduced live subtitling for some of its programming. And due to legal requirements, they will need to keep subtitling more and more shows in the coming years. While there is still a way to go in terms of quantity, I believe it is finally time to look into the quality of live subtitling.

Now that we have passed through the first door, a new “impassable” challenge arises: that of guaranteeing reasonable, fit-for-purpose quality which would mean that subtitles not only serve to fulfil legal obligations but also fulfil the needs of the viewers and allow hearing impaired audiences to have effective access to live TV content. This led to a turn in my research as I switched the main focus to examining the quality of live subtitles.

In view of the above, the present work aims to address the challenge of ensuring quality in live subtitling by examining the available technology, methods and workflows, as well as studying user needs, reviewing and perfecting quality assessment metrics and monitoring the current quality of live subtitles. While the book primarily draws examples from Polish news television channels, the quality issues and metrics it discusses are relevant for viewers, broadcasters and researchers worldwide. The focus on live subtitling quality is essential for enhancing standards in areas and among broadcasters where quality currently falls short. Equally, it ensures that those who have achieved acceptable levels of quality, particularly as they shift towards more automated subtitling methods involving AI, continue to meet these standards. Therefore, establishing clear quality standards, rigorously testing metrics, and consistently monitoring quality are essential steps for advancing live subtitling practices.

To provide the necessary context for understanding the complexities of live subtitling quality, the first chapter begins with a definition of live subtitling as differentiated from semi-live subtitling and pre-recorded subtitling. Following this, it situates the concept of live subtitling within the theoretical framework of Translation Studies – particularly in the realm of audiovisual translation – and Accessibility Studies.

In the second chapter, I discuss the history of live subtitling. I look at the roots of subtitling and the innovations that brought about live subtitling and then led to the invention of respeaking as one of the main methods of creating live subtitles. As this book concerns live subtitling in Polish, I pay particular attention to the beginnings of SDH in Poland, and the development of live subtitling in Polish in various settings: at live events, in online streaming and on TV. As progress in accessibility in general and live subtitling in particular is inextricably linked with legislation and regulatory efforts, I present the legal context of accessibility in Poland within a wider context of accessibility regulations in the EU and the US. Finally, I discuss the current state of live subtitling in Poland and the challenges for the future.

The focus then switches to technology that allows for the creation of live subtitling. The third chapter introduces all the major methods of producing live subtitles, which are then critically examined to identify their potential as well as limitations. Special emphasis is placed on finding out which of these methods can be applied to create live subtitles in a wider array of languages, including Polish. Once speech recognition (SR) in general and respeaking in particular are established as most promising in terms of producing live subtitling, the technology behind SR is discussed in detail to explain what it can and cannot do and why. Again, special emphasis is given to the performance of SR in Polish, the comparisons of systems available for this language and the potential of future development. Finally, this chapter discussed the new generation of SR systems that was made possible by leveraging large language models.

The fourth chapter switches the focus from technology to people and discusses the users of subtitling, their needs and expectations, as well as their role as one of various stakeholders in live subtitling. As all users agree in expecting quality subtitling, in the fifth chapter I discuss what quality means, and how it can be conceptualized as well as measured. After identifying and analysing various dimensions of live subtitling quality, both linguistic and technical ones, the focus moves to describing existing metrics of live subtitling quality such as accuracy, latency, subtitle speed and reduction rate. Then a new metric is proposed to measure text segmentation quality. This is followed by a discussion of the differences between subjective perceptions and objective measurements. The chapter closes by summarising the current knowledge on the impact of different live subtitling workflows on the quality of live subtitles.

The sixth chapter presents the details of the study on the quality dimensions of live and semi-live subtitling on three Polish news channels: TVP Info, Polsat News and TVN24. After discussing the method, the materials and the procedure, I present and critically examine the results on various quality dimensions; most notably: accuracy, latency, subtitle speed, reduction rate and segmentation score. In the case of each of these metrics, the chapter then discusses the statistical methods which were used to examine the data and confirm the statistical significance of the results. The chapter closes by discussing major differences between live subtitles provided by the three Polish news broadcasters, as well as the interaction between various quality dimensions.

The conclusions drawn from the analysed data hold significant implications for the study of live subtitling and respeaking. They also offer practical and conceptual insights for broadcasters, live subtitlers and media regulators. The book concludes with recommendations for broadcasters and media regulators, aimed at enhancing the quality and effectiveness of live subtitling practices.