This chapter provides an overview of the "Diversity and Equal Opportunities in the Media" (DICUM) evaluation tool, which Prof. Christine Horz (TH Cologne) developed based on scientific literature and applied research. The tool enables a systematic and practical evaluation of diversity management in the media using concrete questionnaires and instructions. It can be adapted according to each media company’s specific requirements.

By Prof. Dr. Christine Horz-Ishak.

Participative top-down process

As a rule, diversity strategies are decided at management level (top-down) and should be based on analyses of the existing situation and the specific needs of the company. A sustainable diversity strategy encompasses the entire company with all its divisions and departments.

It is advisable to involve employees and all stakeholders in the change processes at the early stock-taking stage. This allows the creative innovation potential of employees to be activated (activity level) and ensures that the necessary support for the implementation of diversity measures can be achieved "from below" (bottom-up). In this way, all employees of the company can support diversity and are involved in the learning and reflection processes (Stuber 2004).

Potentially important stakeholders in the change process:

• Management/Directorate
• Committees(if any)
• Editors-in-chief
• Editors and journalists (especially those who are potentially disadvantaged, such as PoC, journalists with a migration background, etc.)
• Works council members/staff representatives
• Equality/Diversity/CSR Officer
• Human Resources/Organizational Development
• Internal and external critics of diversity and equal opportunities

Selbstkritik statt nur Diversity-PR

Der Erfolg des Diversity-Konzepts hängt von einer objektiven und damit auch selbstkritischen Analyse bisheriger Maßnahmen und Integrationsberichte ab, sofern es sie gibt. Integrationsberichte wie sie beispielsweise öffentlich-rechtliche Sender derzeit für ihre Geschäftsleitungen im Rahmen von Geschäftsberichten veröffentlichen, erreichen dieses Ziel häufig nicht. Sie binden zwar viele Ressourcen, gehen aber letztlich unsystematisch vor und sind auf punktuelle Medieninhalte verengt. 

Die Personalebene klammern sie meist aus. Zudem erinnern sie mitunter an PR-Maßnahmen, die lediglich bestrebt sind, die Anstrengungen des Unternehmens hinsichtlich Vielfalt in ein positives Licht zu rücken.

Für eine nachhaltige Diversity-Strategie sollten stattdessen Aufwand, Zeit und Geld in systematische und nachvollziehbare Evaluierungen investiert werden. Erst dann lässt sich tatsächlich messen und nachvollziehen, inwiefern die Maßnahmen für die Zielgruppe, das Unternehmen sowie die Mediennutzenden lohnenswert sind.

Why diversity is not a cross-cutting issue

It is particularly important that all internal levels as well as external contacts of a media company are included in such an evaluation process because more diverse staff leads to more diversity in reporting (Napoli 1999). The fact that journalists and editors-in-chief with immigrant backgrounds have been severely underrepresented for decades shows that special efforts must be made to include and promote them.

But diversity should not be seen as a "cross-cutting issue”. With cross-cutting issues, responsibility is decentralized to all or as many areas (in this case, departments) as possible. But to implement a sustainable diversity strategy, it is important that a separate unit is designated and given the responsibility and the concrete instruments it needs to implement and monitor the strategy. And this unit must be supported by the chief editors and coordinated by one office. The strategy itself naturally affects everyone in the company.

The evaluation of measures taken to increase cultural diversity in media companies depends on the following prerequisites (cf. Wladasch & Liegl, n.d.).

1. Appoint a diversity officer

Appoint a person who is responsible for the development and implementation of the diversity strategy in the media company. That person should:

• ideally work exclusively and hold a permanent position, have a reasonable budget, and be authorized to issue directives
• be significantly involved in the development of the diversity strategy
• have a constant overview of diversity measures and their progress
• have access to relevant information
• have expertise on the topics of anti-bias, diversity management and equal opportunities, as well as monitoring and evaluation skills
• ideally, have professional experience in journalism
• To achieve an objective assessment of the measures, independent bodies should be responsible for evaluation, data collection and implementation of performance measures.

2. Assessment

Gather new information or pool existing data as part of the assessment process based on the following questions:

• How diverse is the total workforce (all levels), are the applicants, is the audience?
• Is the diversity strategy treated as a top priority or merely as a "cross-sectional task" without clear responsibilities?
• Are representatives with a migration history also present in other areas such as broadcasting councils and other bodies?
• How diverse are the service providers with whom the broadcaster or publisher works, such as freelance writers, production companies or other external service providers?
• Are the salaries of employees with and without an immigrant background the same for comparable jobs?
• What training measures are in place to increase diversity in the staff structure and to what extent are they accepted by employees?
• How satisfied are employees and the public with the station's or publisher's personnel structure and media products?
• How satisfied are applicants with the application process?
• How are complaints lodged - internally (e.g., employees) and externally (e.g., viewers, readers)? Are the relevant contact or ombudspersons publicly known? Are the complaints procedures transparent and open to public scrutiny?
• How prevalent is harassment and discrimination against people of colour, Blacks or other minority groups?
• How are employees' fears of change processes handled?

3. Assessment of Requirements

Assessing changing requirements serves to determine additional measures and projects that the respective media company may put in place for various departments and editorial offices in the course of the overall assessment process. For media companies, diversity measures that aim to create a more diverse workforce, a more inclusive working climate and content that reflects their audiences or readers diversity are a good way forward.

4. Defining Goals

It’s important to define measurable goals and to set priorities and timelines as part of the development of a diversity action plan.
 

5. Transparency

All phases of the change processes should be evaluated, and the results should be communicated to the public as transparently as possible. This is of decisive importance for acceptance within and outside the company, especially because of the political impact of topics like diversity, migration, and integration. It’s even conceivable for media companies to make their diversity measures a subject of their own journalistic reporting and to present their successes.

1. Analysis of the situation and requirements

Assessment of the current situation - The diversity of the workforce, the working climate and diversity in reporting should be considered and measured. The need for diversity measures can be determined based on the information obtained in this assessment. Where does a media institution see itself positioned? Asking the following questions can help to find an answer:

For example: Workforce:

• Do journalists with immigrant backgrounds apply for positions or does the recruitment process already make them feel excluded? Do they have equal access to journalistic positions or are they considered less qualified despite having the same educational background?
• Does the company collect (anonymous) data on the percentage of non-white journalists?
• Do projects for more diversity in editorial departments already exist?

For example: Work climate:

• Are journalists from immigrant families allowed to voice their opinions in editorial conferences? Are they considered on par with their colleagues - or are they seen as "quota migrants" who only got their jobs due to a quota system (Graf 2011)?
• Are there incentives that could intensify the commitment of employees to diversity?

For example: Content:

• Have all members of the editorial team acquired comprehensive knowledge of diversity-sensitive reporting, for example with the help of training courses? 
• Are people with a migrant background able to voice their opinions in talk shows and vox pops beyond subjects that deal with migration?

2. Ex-ante evaluation

This is conducted at the early stages of the policy-making exercise and serves to determine in advance which measures are relevant for the media company, whether they are coherent, and what rules should apply to implement them. A look at some of the best practice examples (for example, those in this handbook) can also help. Ex-ante evaluation can be used to define targets and milestones, baselines, outcomes, and processes for later evaluation.

Tools such as a Diversity Score Card (DSC) or a Diversity Guide can be helpful. They provide a basic and simple structure for planning and evaluating diversity management. They can also be used to link diversity goals with other goals of the broadcaster or publisher, and to communicate and evaluate diversity measures in a clear way (Hubbard 2004).

If possible, all stakeholders should be involved in the development of the DCS, because this can increase the acceptance of developing diversity measures.

In addition, the general strategic goals of the company should be in line with the diversity strategy and the advantages of having a diversity strategy should be communicated, e.g., addressing new audience groups, thereby increasing circulation of a newspaper (Hubbard 2004).

3. Interim Evaluation

This is where milestones can be evaluated in order to record progress and, if necessary, to readjust measures if the actual targets are not being met. The Diversity Score Card helps in this process, as it can be used to review basic goals and areas in a comprehensible manner. Such as:

 

• Has the recruitment strategy been adjusted with a view to diversifying the workforce structure?
• Has the training program been adapted to the needs and goals for more diversity - for example, do permanent staff and freelancers have access to training sessions in which they can learn about cultural sensitivity in reporting?
• To what extent have projects for more diversity been initiated at all levels of the broadcasting company or publishing house - including at the management level?
• Have the editorial departments analysed their reporting to determine how and where they can present more pluralistic perspectives in the future?
• Have the timetables for implementing the envisaged goals been met? If not, what are the reasons for the delay?
• Where is there a need to intensify efforts?

4. Final evaluation

This takes place at the end of each phase of the project and assess the extent to which the actual target has been met. The final evaluation focuses on the continuation of the diversity strategy with new goals. In addition, the final evaluation also acts as an institutional memory that keeps track of all information, qualitative and quantitative data, and people involved in the process.

For example: Workforce:

• Was it possible to increase the number of editorial staff members in line with the target? 

For example: Work climate:

• Did accompanying measures such as team building, or the improvement of feedback structures ensure an open, less hierarchical climate of communication?

For example: Reporting:

• Have training measures been put in place to ensure that staff can acquire more cultural sensitivity?
• Are the voices of people and experts with migrant backgrounds heard more often now than before the start of the diversity project?

5. Ex-post evaluation

This takes place up to three years after the measure has been completed to review final impacts and measure the sustainability of the diversity strategy.

For example: Workforce:

• Have recruited journalists from immigrant families, People of Colour, and Black people been retained at the company?

For example: Work climate:

• Has a sustainable improvement in the work climate been achieved, for example by reducing hierarchies?

For example: Reporting

• What reactions did the change in reporting to include more diversity elicit from the public, audiences, readership?
• Do the projects continue to meet the company’s needs, or have new social developments made it necessary to formulate new goals?

Involving as many stakeholders as possible to review the effectiveness of the implemented measures integrates different perspectives, areas, and hierarchical levels into the evaluation (Stakeholders page xx). It is helpful to consider which evaluation tools (e.g., survey) serve which focus of analysis (e.g., diverse staff) and for which target group (e.g., workforce diversity characteristics) they should be used (Wladasch & Liegl, n.d.). 

In summary, the following structure can be presented:

• A list of all measures to be implemented in the evaluation period should be compiled for those in positions of responsibility (management, editors-in-chief, diversity officers), so that they can take the first steps toward a diversity strategy. The target groups of the measures (PoC, journalists from immigrant families) can also be included to make them measures even more effective.

• To anchor the measures to become best practice, a survey can be conducted among CSR and diversity officers, staff representatives, editors, journalists, especially the potentially disadvantaged ones, middle management, and internal and external critics of the diversity strategy. This can be done using an interview guide, group discussions or similar methods to measure the level of satisfaction of these groups with the measures from their respective perspectives.

• Additionally, smaller focus groups can be formed from to obtain more detailed evaluation results. Again, interview guidelines that can measure (dis)satisfaction with the measures from the respective perspectives are helpful in identifying gaps. In addition, regular surveys of the public and staff can be used to gather very specific perspectives.

Several levels and dimensions must be considered in the development of a diversity and inclusion plan in the personnel structure and its evaluation:

Leadership/decision-making level

The NdM study „Viel Wille, kein Weg“ showed that the positions of chief editors in German media show very little diversity (NdM 2020). This is where specific measures and evaluations focusing on the management levels become necessary, since this is where decisions on diversity strategies are made.

Editorial offices

For years, the number of journalists with a migrant background working in German editorial offices has been very low (Pöttker 2016). Here, a start can be made on reviewing and changing current recruitment measures so that this group feels addressed.

Public Value

More inclusive and diverse editorial staff can have a positive external effect (role model function, media reflecting society). This also includes reflecting and presenting diverse perspectives, which is reflected in the trans culturalization of content (e.g., radio formats such as “Datteltäter”).

Financial Value

The cost of the diversity project and its benefits must be balanced, but this will almost always be the case in the form of new user groups and audiences. 

The BBC in the UK, for example, can prove that its diversity strategy has attracted new audiences.

There is no single method for evaluating diversity measures. Depending on the different status quo in each media company, tailor-made approaches to evaluation must also be applied, setting different focal points. The focus of an evaluation can depend on aspects such as the following:

• the respective goals of the diversity strategy (e.g., 50 percent of the next group of trainees should be recruited from marginalized groups)
• the resources involved (economic, personnel, time). In the above example, one would have to decide whether the measures for trainees should be cost-neutral and should involve the HR department or not. How long the project should run, interim evaluations and milestones must also be defined
• the level of progress in terms of diversification in the respective media institution

Important: regardless of the focus, representatives of the target groups of diversity strategies should always be involved in the conceptualization, implementation and analysis of measures and results. Based on previous surveys (NdM 2020, Horz 2016), we can assume that most broadcasting and publishing organizations in Germany have not implemented any concrete diversity measures.

Companies that want to take first steps towards more diversity:

Following an initial assessment of the situation, minimal evaluation standards are required. These include evaluations that help to assess the results of any previously defined measures making it possible to evaluate which measures were started on time and where there is room for improvement.

• Is the new wording for the announcement of the trainee programme on schedule?
• Have new training measures and workshops been planned to address cultural sensitivity in reporting and programming?
• Have any initial plans been made as to which employees from marginalized groups should be promoted?

Such data can easily be accessed using simple survey instruments, such as a milestone report, short staff surveys, or roundtable discussions to gain qualitative feedback from all stakeholders.

Companies that want to expand their diversity measures

In media companies that are already implementing selective diversity measures, more complex evaluation methods lend themselves to surveying diversity in their staff and trainee structures, in their programming and in their content. Qualitative studies offer tools to assess the effectiveness of the measures.

Through workshops and focus group discussions, you can get staff to commit to diversity measures at an early stage of implementation. For example, MDR-next the innovation development department of MDR broadcasting company, regularly conducts participatory co-creation processes in the form of living labs. These can also be jointly planned with organizations such as the NdM. During this phase, innovative forms of evaluating the diversity measures should already have developed. 

Dialogues with stakeholders can also be a further instrument to measure progress. These conversations should not only be held with experts but should include viewers and readers, as well as people who have experienced discrimination.

Companies that want to achieve Best Practices

Supplementary quantitative survey instruments are useful to ensure the success of comprehensive diversity measures based on objective and comparable data. Surveys targeting editors-in-chief, staff, and media consumers can be helpful in addition to qualitative methods. They can help to ascertain how, from the employees’ points of view, the workplace climate and communication culture has improved and where there may still be room for improvement. Such surveys can also show whether acceptance for implemented measures, such as the introduction of voluntary quotas; has changed, whether developments are seen as promising and whether the measures and strategies adopted are being transparently publicized.

Assessment: in-house and by external experts

Progress can essentially be evaluated in-house by initiating feedback loops, discussion forums, focus groups, idea competitions or similar methods. The best evaluations, however, are those conducted together with external, independent institutes and experts. These lead to more objective assessments and thus increase the quality of the evaluations. This in turn increases public acceptance of diversity measures.