A study by the Baltic Centre for Media Excellence has just been presented. It examines trends in the quantitative evaluation of the sustainability and impact of the media literacy projects and their creators' understanding of the sustainability, effectiveness, and impact of the projects in Latvia. Its purpose is to collect publicly available, and as objective as possible, indicators of the 27 projects (the results of which were made public in the three years until July 2023) to obtain an overview of the media literacy field, thus contributing to the further improvement of the sustainability and impact of media literacy activities. The results are helpful in planning and implementing educational activities related to media literacy.

The methodology of the quantitative analysis of the projects was created anew, taking into account the Latvian context, but the ideas are based on theoreticians' proposals for measuring the impact of media literacy activities and practically carried out studies [1].

What are the insights and lessons learned?

More long-term measures are needed

Of the 27 analyzed media literacy projects and initiatives, most (20) are one-time events. Material is created and presented, or a seminar is conducted, but no further activities follow. Only three projects/initiatives were regularly active for at least one year.

Of course, some projects are objectively impossible to repeat; for example, a book is published, or a conference is held. However, in general, there is a tendency to create individual one-time events (discussion, lesson), which can be accomplished in a shorter time and with potentially less funding than planning successive activities during the year within the framework of a single project. This creates fragmentation and negatively affects the sustainability of projects since also communication activities for a one-time event take place in a potentially shorter time and with a smaller budget than regular promotion of a long-term initiative.

If media literacy activities are concluded soon after the start and their creators reorient themselves to new projects, this also hinders the development and in-depth consideration of media literacy topics and knowledge transfer. Therefore, it would be desirable to have projects that provide for more prolonged/regular media education - at least one year long- to ensure more stable project results.

Reusability of results

Although most of the analyzed projects are one-time activities, their strength is the reusability of the developed materials. Only in one case, in the end, apart from the knowledge and skills provided to a limited number of participants, there are no other lasting results. For the rest, even if they were held in the format of "volatile" events, in the end, there is a publicly available and relatively easy-to-find video recording or developed materials – books, lesson plans for teachers, etc. It also benefits the community after the active phase of the project is over.

Educates theoretically, less practically

When evaluating the project's content in correlation with the potential goals of media literacy education, the majority (18 out of 27 projects) mainly promote knowledge. For example, they are lectures without practical tasks. Only a third of the projects aim to improve the participants' media literacy knowledge and skills, including practical tasks in the content. It is recommended that the theoretical lecture is supplemented with fact-checking training, misinformation recognition, or other thematically related practical tasks, not only to educate the audience theoretically but also to teach practical information evaluation techniques and steps.

Not enough novelty?

Only a few projects show a particular novelty regarding the chosen target audience, subject, or practical implementation methods. These innovative projects aim to reach an audience that was rarely / never addressed in Latvia or is challenging to engage (not the usual projects, for example, for young people or teachers). Thematically, new, relevant media literacy topics have been chosen, or an unusual combination of content contrary to generalized subjects regularly included in projects, such as how to distinguish disinformation. The slightest novelty is expressed in the methods used in the projects, where usual techniques, such as conducting a lecture, mostly prevail. It could also be related to project financing and a routinized, proven approach to project creation. The use of new methods also requires additional skills from the project staff.

Although "routine projects" that fulfill a specific function are also helpful, "flagship" projects that bring something new to the field of media literacy are significant. The fact that only a small number of projects have a high degree of novelty can potentially slow down the field's development because preference is given to already proven but perhaps not always the most effective ways of project implementation.

Communication strategy

It is recommended that the strategy of publishing media literacy materials is evaluated. The sustainability of media literacy projects is limited by the relatively short time in which some of them are available to the public. Although a separate project website allows the user to access all materials at once, its creation is also a potential risk - sometime after the end of the project, the specially created websites often cease to exist. Hence, the projects’ results are no longer available on the web.

Interviews with representatives of the potential target audience of the media literacy project also show how diverse the wishes and expectations are - in regards to the format they want to receive information about media literacy. It can be concluded that there is too little information for the potential target audience, especially for those people for whom media literacy issues are not part of their daily work, as it is for teachers and librarians. Therefore, more attention should also be paid to communication activities, to reach those not already among the regular interested parties.

A combination of original content and adaptations

Analyzing the origin of the projects in the research sample, the majority, or 20 of 27 projects, are original content created in Latvia. The origin of one is difficult to determine, while six projects are based on adapted content - materials developed in another country are translated and adapted to the Latvian context. For example, the adapted online games "Go Viral!" and “Harmony Square” or an adaptation of the media literacy breakout room game were developed at the University of Washington. Considering the large proportion of originally created projects, the part of adapted projects should be evaluated positively. Such an approach allows us to bring to life in Latvia measures already tested elsewhere. It is crucial that they are not translated mechanically but adapted to the local context.

The ability to bring about change in society

One of the project’s sustainability elements of a project is its ability to create change in society. In this study, it was defined that the maximum number of points (2) will be awarded to projects if they help or potentially claim to help people use media literacy in a way that expands their capabilities and opportunities in life (knowledge, ways in which something related to media use in life is done differently). No project received this rating. 18 projects have been found to have a partial ability to create societal changes, which is a good indicator. Meanwhile, nine projects (1/3) have minimal capacity to make a difference or it is difficult to be identified.

How can we promote projects' ability to contribute to positive changes? This is closely related to the project's novelty and regularity/durability over time, as one-off events have less potential to promote change.

Discussion on sustainability and efficiency criteria

The research shed light on the shortcomings of media literacy projects (not enough long-term measures; a relatively small proportion of projects that have novelty in terms of target audience, subject or practical methods; providing mostly theoretical knowledge; only a third beingoriented to develop media literacy skills, etc.). However, it also showed the projects' strengths - their results have a high reuse potential, and most of the project creators in the survey indicate that they measure the effectiveness of media literacy activities.

Projects' sustainability, efficiency, and impact are undeniably related to available funding. However, one step that can be implemented to promote general understanding is a meeting organized by the ministry responsible for media policy or international partners. It could be an exchange of experience with colleagues from other countries or a discussion on sustainability and effectiveness criteria of media literacy projects, creating guidelines applicable to Latvia’s situation that could serve as a reference point in planning new projects. The previous ideas of media literacy project implementers are different about what a sustainable and effective media literacy project is and how to evaluate the projects created in Latvia, attributing opposite characteristics to them, starting from "not very sustainable" to evaluations that "are moderately good" or "very sustainable”. Such a discussion could contribute to a more unified vision of what kind of project is a long-term activity and what are the sustainability criteria; and thus, future media literacy projects could be more sustainable, efficient, and influential.

Author: Dr. Klinta Ločmele, senior researcher at the Baltic Centre for Media Excellence, a docent at the Department of Communication Studies of the University of Latvia

Background illustration by: tippapatt

[1] Livingstone, S. (2022, July 5), The vital role of measuring impact in media literacy initiatives; McDougall, J. (project lead) [ Without date]. Evaluating Media Literacy with a Theory of Change: A guide to using a theory of change to design and evaluate media literacy projects and activities; Ofcom. (2023, November 10), A toolkit for evaluating media literacy interventions - Ofcom.