Terms of reference: Final Evaluation of Suara Masyarakat (Community Voices) Project in Malaysia (January 2020 – December 2022)

TERMS OF REFERENCE

Final Evaluation of ‘Suara Masyarakat (Community Voices) project in Malaysia

INTERNEWS OVERVIEW

Internews is an international non-profit organization whose mission is to empower local civil society worldwide to give people the news and information they need, the ability to connect and the means to make their voices heard. For more information about the organization please visit: www.internews.com

PROJECT OVERVIEW

For the past 32 months, Internews in Malaysia with the support of the European has been implementing a 36- month action entitled Suara Masyarakat (Community Voices) that aims to build the capacity of journalists to produce high-quality, investigative content on reform process in Malaysia and create feedback loops between citizens, journalists and local authorities that are crucial for government accountability and transparency. Internews partners with Science of Life Studies 247 (SOLS 24/7) as co-applicant for the implementation of this action. Further, implementation has been undertaken with the support of strategically selected third parties, Sinar Project and the Centre for Independent Journalism (CIJ), who has been engaged through a sub-granting mechanism. Internews has also partnered with other NGOs and grassroots organisations such as Jaringan Kampung Orang Asli Kelantan, Faqcheck Lab and the Center for Orang Asli Concerns through microgrants. To implement this action, Internews has drawn on its robust experience supporting journalists and CSOs in the region.

Internews in Malaysia team through its local partners, has been delivering high quality investigative and data journalism on reform processes in the country and develop community information media and networks to improve governance by providing two-way communication between citizens and local authorities. This is done through workshops and trainings on investigative, data and mobile journalism which targets journalists, citizen journalists and information poor communities with a focus on women, youth, Indigenous and marginalised groups.

Overall objective: Strengthen the capacity of media as a key actor in promoting good governance, transparency, and accountability

Specific objectives (SO):

1. SO 1: Increase the capacity of Malaysian journalists to deliver high quality investigative and data journalism on reform processes in the country
2. SO 2: Develop community information media and networks to improve governance by providing two- way communication between citizens and local authorities

The Suara Masyarakat (Community Voices) project is designed based on the following methodology:

This action seeks to increase journalists’ capacity and skills through trainings and mentoring in investigative reporting and data journalism, with the goal of building the capacity of media to promote good governance, transparency and accountability. Internews’ training and mentoring programmes are built on creating an environment for independent media to safely grow and promoting local information networks for both citizen journalists and the general public. In line with this goal, Internews has designed comprehensive training and mentoring programmes that strengthen the capacity of journalists to report effectively and strengthen the

inclusion of communities and marginalised groups in democratic debate. Internews and its partners have also woven a sub-grants programme into the fabric of the programme, promoting sustainability and giving mentees real experience in reporting, publishing and sharing content. Our organisational Theory of Change frames all our work: to enable safe access to trusted information, to ensure the inclusion of systematically excluded groups, to increase quality content of local, verifiable and unbiased information; and to enhance the engagement of citizens with their information landscape. This project, designed with the co-applicant

SOLS 24/7, responds to this logic as well.

EVALUATION PURPOSE AND OBJECTIVES

The purpose of this final evaluation is to assess the overall, achievements and lessons learned of the project that will help to determine the project success and to inform future programming design and implementation in Malaysia. It should also look at signs of potential impact of project activities on beneficiaries and sustainability of results. The consultant will be responsible for producing several deliverables using a methodology and following a timeline that is mutually agreed upon with the project staff. This evaluation has the following principal tasks which must be assessed in the context of the political and environmental challenges that the project faced during implementation:

• • Assess the project's achievement towards goals and objectives and look at signs of potential impact of project activities qualitatively and quantitively, as well as sustainability of results; This includes how the project contributed to strengthening the capacity of journalists to produce high-quality, investigative content on reform process and increasing dialogues among citizens (especially women and marginalised groups) and local authorities and the assessment of the audience engagement and interaction with the content produced by the project using proxy measures.
  • Analyse the underlying factors beyond Internews’ control that affected the achievement of the project results and required adaptation of the project activities including the impact of changes in the operational environment over the project lifetime and its impact on project activities;
  • Assess the performance of the project in terms of relevance, coherence, effectiveness, efficiency, impact and sustainability;
  • Which effects, both intended and un-intended, did the program have on beneficiaries.
  • Extract learnings and good practices and provide key recommendations which could be used to enhance future interventions.

EVALUATION SCOPE AND METHODOLOGY

The evaluation covers the project implementation period from 1st January 2020 to 31st December 2022 and should encompass the entire scope of the project. Ideally the work will be carried out in country by a local national evaluator with language skills. If no suitable local candidate can be found, then this evaluation may be implemented completely remotely by the internal evaluator.

The evaluation should use the Development Assistance Committee of the Economic Cooperation and Development (OECD/DAC) criteria (relevance, effectiveness, efficiency, impact, and sustainability), which is one of the most widely used criteria to review aid effectiveness using gender and inclusion lens. The following questions might guide the project's evaluation but are subject to discussion and agreement during the period of evaluation design.

• • Did the theory of change hold true?
  • Has the project responded to the needs of the target groups and continued to do so when circumstances have changed? (Relevance)
  • To what extent were lessons learned from what worked well and less well applied to improve and adjust project implementation? (Relevance)
  • To what extent have other interventions (particularly policies) supported or undermined the project intervention, and vice versa? (Coherence)
  • How compatible is the project with other interventions in the region or organization where it was being implemented? (Coherence)

• • Have the right things been done to achieve the project's aims? Have the planned project objectives and outcomes been achieved (or are they likely to be achieved)? If not, what is needed to close the gap and improve potential for impact? (Effectiveness)
  • Were the resources and inputs converted to outputs in a timely and cost-effective manner? Did the project achieve the planned outputs? Could the outputs have been achieved with fewer resources? (Efficiency)
  • What results have been achieved in the implementation of the project? What were the contributing factors to the achievement of the project? What were the main challenges/barriers to achieving the implementation of the project? How well did the project adapt to the significant political and security challenges during implementation? (Achievement)
  • What are the tangible outcomes (positive and negative) as a result of this project? Has there been a positive or negative impact on beneficiaries as a whole? How has this impact been ascertained? What specific, real difference has the activity made to the beneficiaries? Have certain groups benefited more or been negatively impacted more? (Impact)
  • Are the results sustainable? If so, how? (Sustainability)
  • Analysis of horizontal issues such as gender (empowering women and girls), youth engagements, and HR.
  • An evaluation of the EU’s visibility, impact and results.

DATA COLLECTION TOOLS

The evaluation will primarily focus on donor key indicators that will help to assess performance and success of the project using relevant survey tools. The evaluator should find ways to ease logistical and time challenges and adapt to the respondents' schedules and needs, to the extent feasible. The evaluation and related research should abide by Internews' policies and protocols related to data protection and security and be conducted in line with ethical protocols including participant confidentiality and privacy. The secure transmission of data is of great importance and must be ensured at all stages of the evaluation. A non-disclosure agreement will be included in the consultancy contract.

The methodological approach and design of the evaluation will be developed and agreed between Internews and the evaluator. The evaluator will then be responsible for its objective implementation. The methodological approach should be participatory, inclusive, gender-sensitive and culturally appropriate. The exact evaluation methodology (including the proposed sampling of stakeholders to be engaged) should be presented by the evaluator and defined, discussed, and agreed upon with Internews during the first week of the evaluation period. However, the evaluation should include, but not necessarily be limited to, the following tools:

• • Desk review of relevant project documentation, progress reports, previous pre and post workshop surveys and assessments, etc.
  • Individual and/or group interviews with project stakeholders online – (where possible);
  • Online survey with stakeholders, if relevant
  • Focus group discussions (FGDs) and key informant interviews (KIIs) with beneficiaries and stakeholders including Internews, donor, partners and NGOs working in the same space

EXPECTED DELIVERABLES

The evaluator is expected to submit an inception report describing the detailed evaluation methodology and tools and an evaluation report that documents the evaluation activities and results, following a clear structure. The evaluation report will contain the main findings, conclusions, and recommendations as well as a recording of the lessons learned. The evaluator will initially prepare a draft report, which will be submitted to Internews for comments. While considering the comments provided on the draft, the evaluator shall use their independent and impartial judgement in preparing the final report.

The final report should be no longer than 20 pages, excluding the annexes and executive summary. The report must include:

• • An executive summary highlighting the main findings, conclusions, and recommendations;