ThaiNGO

CALL FOR EXPRESSION OF INTEREST (EOI)

Evaluation of the ASEAN-Australia Political Security Partnership (APSP) 

The Asia Foundation (TAF) is a non-profit international development organization committed to promoting lives across a dynamic and developing Asia. Informed by six decades of experience and deep local expertise, our work across the region addresses five overarching goals – strengthen governance, empower women, expand economic opportunity, increase environmental resilience and promote regional cooperation.

TAF Thailand Office is seeking a qualified evaluation team to conduct the evaluation of ASEAN-Australia Political Security Partnership (APSP) program, an investment of the Australian Government. The evaluation will be managed and overseen jointly by Australian Department of Foreign Affairs and Trade (DFAT) and TAF.

Interested team must read the Terms of Reference (TOR) for the evaluation of APSP carefully to prepare the necessary information and documents for the EOI.  

Prior to the submission of EOI due date, the interested team may submit questions concerning the evaluation to Pitchanuch.supavanich@asiafoundation.org. In the email subject title, please indicate: “Inquiry on APSP Evaluation.”

Qualified team must submit the EOI in electronic format by 21 June 2024, 11.59 pm Bangkok time to Pitchanuch.supavanich@asiafoundation.org. In the email subject title, please indicate: “Submission of EOI for APSP Evaluation.”                                                                                                       

Following information and documents are required for the submission of EOI:

• One page statement which details how the team member’s qualifications, experiences and skills are relevant to the evaluation.  
• CVs of the team members.
• Company/organization profile (if applicable)
• One page description of the methodology that the team will propose for the evaluation based on the initial information provided in the TOR.
• A statement confirming the team members’ availability to conduct the evaluation and that the team members have no previous involvement in the implementation of APSP.
• An indicative financial proposal, including daily professional fee of the team members.

The EOI will be assessed by DFAT and TAF. Only shortlisted team(s) will be contacted and may be requested to provide additional information or document.

Terms of Reference

Evaluation of the ASEAN-Australia Political Security Partnership (APSP)  

I. Background and Context:

Since 1974, Australia has cooperated with Association of Southeast Asian Nations (ASEAN) to promote a peaceful, stable, integrated and prosperous region. The ASEAN-Australia Political Security Partnership (APSP) is an AUD 10.5 million, six-year (2019-2025) investment of the Australian Government aimed at enhancing political and security cooperation between ASEAN and Australia. The program encompasses the outcomes and activities that contribute to strengthening regional stability, addressing shared challenges, and promoting cooperation among ASEAN Member States and between ASEAN and Australia in the political security landscape.

APSP began on 1 May 2019 and is currently ongoing until 30 April 2025. It is designed to provide flexible and demand-driven support to ASEAN and to promote ASEAN leadership and centrality in addressing regional political security challenges. Towards a broader goal of an improved security environment in the Indo-Pacific region, APSP intends to support stronger relationship between ASEAN and Australia and to enhance ASEAN’s ability to lead more coordinated, cohesive and inclusive responses to transnational political security issues. The program has the following end-of-investment outcomes (EOIO):

EOIO 1:           Stronger ASEAN-Australia relationships and engagement

EOIO 2:          ASEAN is addressing targeted political security issues using increasingly

                       effective cross-sectoral and/or multi-stakeholder approaches

APSP focuses primarily on four broad policy areas of 1) maritime security; 2) cybersecurity; 3) women, peace and security (WPS); and 4) transnational crime. In addition, APSP is open to support ASEAN in other political security issues as opportunity or demand from ASEAN arises. Activities of APSP include workshops, seminars, trainings, and knowledge sharing sessions, among others that engage relevant ASEAN sectoral bodies, ASEAN Secretariat and other entities associated with ASEAN. In addition, APSP also administers policy papers and researches as well as support institutional strengthening of ASEAN.

APSP recognizes that gender equality, disability and social inclusion (GEDSI) is critical for ASEAN to realize a coordinated, cohesive and inclusive responses to transnational political security issues. As such, APSP integrates GEDSI throughout the program cycle, from activity design to implementation, and monitoring and evaluation and learning.

Under overall management by Australia’s Department of Foreign Affairs and Trade (DFAT), APSP is implemented by The Asia Foundation (TAF). DFAT also allocates fund for other Australian government departments (OGD) to implement the activity with ASEAN.

APSP’s key partners and target beneficiaries are ASEAN sectoral bodies, particularly those under ASEAN Political Security Community (APSC), ASEAN Secretariat, relevant ASEAN centres and entities, and ASEAN Member States governments. Other stakeholders of APSP include civil society organizations, think tanks, academia, and private sector entities that contribute to the program's objectives and activities.

II. Purpose of the Evaluation

The evaluation is an integral part of the APSP Program's Monitoring, Evaluation, and Learning (MEL) Plan as well as DFAT’s investment. The evaluation will cover the period from 1 May 2019 to 30 June 2024 or the first five years of APSP.

Overall, the evaluation will contribute to evidence-based policy development, program improvement, and strategic planning in the field of political and security cooperation between ASEAN and Australia. In particular, it will provide an in-depth analysis and assessment of the APSP achievements and challenges against related DFAT’s quality criteria. It will provide management responses to relevant program managers and policymakers in making informed decisions, adapting strategies, and allocating resources effectively for the remaining period of APSP and beyond.

In keeping with the APSP MEL approach, the evaluation will be conducted for the following purposes:

• Accountability and Learning: The evaluation serves as a mechanism for accountability, ensuring that APSP is being implemented as intended and in line with its stated objectives. It provides an opportunity to assess the program's performance, identify strengths and weaknesses, and learn from past experiences. The evaluation will contribute to evidence-based decision-making and enable program managers to make informed adjustments and improvements for the remaining period of APSP and possibly a successor program. .
• Evidence-Based Program Development: The evaluation will assess the program's objectives and activities in addressing transnational political security challenges, while also identifying opportunities for refinement or expansion. It will identify areas for improvement and provide recommendations for enhancing its impact, taking into consideration the decisions for a possible successor program. The findings of the evaluation will guide future planning, resource allocation, and strategic decision-making within the program.  Additionally, these evaluation results will inform decision-makers and stakeholders about the outcomes and impacts of APSP, supporting evidence-based policy development, strategic planning, and decision-making in the field of political and security cooperation between ASEAN and Australia. The ultimate goal is to contribute to the formulation of evidence-based policies and interventions that effectively address regional security challenges and promote peace and stability.
• Stakeholder Engagement and Communication: The evaluation process itself provides a platform for engaging stakeholders and fostering dialogue among key actors involved in APSP. It allows for the exchange of perspectives, experiences, and lessons learned, facilitating a shared understanding of the program's achievements, challenges, and potential areas for collaboration. The evaluation results will be communicated to stakeholders through reports, presentations, and dissemination activities, ensuring transparency and promoting dialogue on program effectiveness and impact. The evaluation report will also be made available on DFAT’s website for public access.

The primary users of the evaluation results are program managers, senior managers and senior executives of DFAT, particularly at Australian Mission to ASEAN; and program manager, program team and senior executives of TAF. The evaluation report will be made available on the website of DFAT for public access.

III. Evaluation Scope and Questions

The evaluation of APSP will address the following DFAT’s quality criteria:  

• Relevance: Assess the relevance of APSP in addressing regional transnational political security challenges within the ASEAN context. Determine the extent to which the program's objectives, activities, and approaches align with the needs and priorities of ASEAN and Australia.
• Efficiency: Evaluate the efficiency of the program's implementation, including the utilization of resources, management practices, and coordination mechanisms. Assess the cost-effectiveness of the program in achieving its intended outcomes and outputs.
• Effectiveness: Assess the effectiveness of APSP in achieving its objectives and intended outcomes. Evaluate the extent to which the program has contributed to strengthening ASEAN-Australia relationship and engagement; and ASEAN’s ability to lead more coordinated, cohesive and inclusive to transnational political security challenges.
• Sustainability: Examine the sustainability of the program's impacts and outcomes. Consider the extent to which the program has fostered long-term partnerships within ASEAN and between ASEAN and Australia, built institutional capacity, and established mechanisms for ongoing cooperation and dialogue.
• Impact: Assess the overall impact of APSP on regional political security environment and cooperation. Examine the program's contribution to addressing transnational political security challenges, promoting trust and understanding, and strengthening regional institutions.
• Mainstreaming Gender Equality, Disability and Social Inclusion (GEDSI): Evaluate how APSP has sought to strengthen GEDSI in its development efforts. Assess the program's integration of GEDSI considerations in its activities, outcomes, and impacts.

The evaluation will seek to answer the following evaluation questions to generate relevant information and insights. These questions are indicative and may be refined or further elaborated during the evaluation planning process.

The evaluation will cover all activities under APSP from the start on 1 May 2019 to 30 June 2024. The evaluation will take into account the operating context, both internal and external, of APSP and how APSP has evolved and adapted throughout the evaluation timeframe, which covers the Inception Phase (May – October 2019), First Implementation Phase (November 2019 – June 2020), Transition Phase (July 2020 – January 2021) and Second Implementation Phase (February 2021 onward).

The evaluation will also pay due consideration to interests, concerns, and relevance of various actors, partners and beneficiaries involved in the implementation of APSP.

IV. Evaluation Methodology and Approach

The evaluation will be conducted by an evaluation team under guidance and management of DFAT and TAF.

The evaluation will employ a mixed-methods approach, combining qualitative and quantitative data collection methods. It will build upon existing monitoring and evaluation data of APSP in order to provide a comprehensive understanding of the program.

The specific and final methodology will be determined through consultations among the evaluation team, DFAT and TAF, and tailored to meet the evaluation's purpose, scope and available resources.

In addition, the conduct of the evaluation and all deliverables should adhere to relevant DFAT’s evaluation standards (see: https://www.dfat.gov.au/about-us/publications/dfat-design-monitoring-evaluation-learning-standards)

The following provides a general outline of the potential methodology and approach:

• Desk Review: A comprehensive desk review will be conducted to gather and analyse relevant program documents, reports, policy papers, and other available literature. This review will provide a solid understanding of the program's design, implementation, and outcomes, serving as a foundation for the evaluation.
• Data Collection and Analysis: The evaluation will employ a mixed-methods approach to gather data from various sources. Quantitative data will be collected through surveys, questionnaires, and statistical analysis to measure program outcomes and assess indicators. Qualitative data will be collected through stakeholder consultations and case studies to explore stakeholders' experiences, perceptions, and narratives related to the program.

The stakeholder consultations will involve DFAT, TAF and relevant OGDs; key ASEAN partners and beneficiaries; and other relevant stakeholders of APSP. Consultation methods could be in forms of interview (virtual or in-person), focus group discussion (virtual or in-person), and structured survey to gather diverse perspectives and insights on APSP. The identification of key partners and beneficiaries and other relevant stakeholders will be done in consultation with DFAT and TAF.

• Data Validation: The evaluation will employ data validation techniques to ensure the reliability and validity of the collected data. This may include triangulation of data from multiple sources.
• Reporting and Dissemination:   The report will present key findings and highlight actionable recommendations for program improvement, policy development, and strategic decision-making in line with DFAT’s standards. The report will be submitted to DFAT and TAF who may request the presentation of findings. The final report will also be published on DFAT’s website.  

V. Evaluation Timeline and Outputs

The evaluation team will be accountable for producing the following deliverables:

• Evaluation Plan (no more than 15 pages, 5 days): Based upon desk review and initial discussion with DFAT and TAF, the evaluation team will develop the evaluation plan. The evaluation plan, at minimum, will include evaluation questions, proposed methodology, sources of data, and data collection procedures for addressing each evaluation question. The evaluation plan will also cover a schedule of tasks, activities, and deliverables (including report outline). The evaluation plan serves as a basis for confirming a shared understanding of the evaluation between the evaluation team, DFAT and TAF and allows for clarifications and adjustments as necessary.
• Data collection, analysis, and validation and presentation of preliminary findings (20 days): Following the approval of the evaluation plan by DFAT and TAF (unless otherwise agreed), the evaluation team will commence comprehensive data collection (including stakeholder consultations), analysis and validation. Subject to the advice from DFAT and TAF, the stakeholder consultants may be conducted virtually or in-person. At the end of stakeholder consultations, the evaluation team will prepare an Aide Memoire and present preliminary findings against each evaluation criteria to DFAT and TAF.
• Draft Evaluation Report (no more than 30 pages, excluding annexes) (8 days): The evaluation team will develop a draft evaluation report based on a mutually agreed outline, that presents the findings, analysis, and conclusions of the evaluation. This report will address the evaluation questions and provide an assessment of APSP against the criteria. The draft report will also include actionable recommendations for program improvement, policy development, and decision-making. The draft evaluation report will be submitted to DFAT and TAF who will have an opportunity to review and provide feedback to ensure that it meets the intended purpose and standards.
• Final Evaluation Report (5 days): Based on the feedback received during the review of the draft report, the evaluation team will finalize the evaluation report. The final report will incorporate any necessary revisions, improvements, and additional analysis. It will provide a comprehensive and evidence-based account of the evaluation findings, conclusions, and recommendations. The report will be structured in a clear and accessible manner, ensuring that it effectively communicates the evaluation results to the intended users.
• Presentation of the final evaluation findings (upon request): The evaluation team may be requested to provide presentation or verbal briefing of the findings to DFAT and TAF. In addition, the evaluation team may be requested to prepare evaluation brief that summarizes the key findings, conclusions, and recommendations of the evaluation in a concise and accessible format. The evaluation team may also be asked to participate in a knowledge sharing activity to present the findings.  

All deliverables will be in English and be written in evidence-based manner.

VI. Qualifications of the Evaluation Team

Given the scope of the evaluation, a team of up to 2 members, including one Team Leader, will be recruited and selected by DFAT and TAF based on the following qualifications.

The Team Leader should have:

• Post graduate degree with a minimum of 10 years of relevant professional experience including conducting of independent evaluations of regional development projects or programs. Experience in completing the evaluation of a project or program involving ASEAN is an advantage.
• Sound knowledge of ASEAN policies, architecture and systems. Knowledge of DFAT aid development and systems is also an advantage.
• Extensive knowledge of evaluation methodologies, including qualitative and quantitative data collection, desk review, stakeholder consultations and ability to apply suitable methodologies to answer the evaluation questions.
• Sound technical knowledge in evaluating regional or multi-country programs focusing on political security issues.  
• Excellent interview and analytical skills. Proven experience in conducting data collection with government officials, including through interview or focus group discussion.
• Excellent verbal and written communication skills in English. Ability to write constructive reports and convey complex issues and ideas in easy-to-understand manner.
• Ability to lead the team toward high quality deliverables and within the agreed timeline.

Overall, the team should meet the following criteria:

• Demonstrate M&E skills including practical experience in monitoring and evaluation of regional or multi-country development programs.
• At least one member should have expertise in evaluating GEDSI aspect of the program. It is expected that the composition of the review team will reflect DFAT’s GEDSI values.
• Have a practical and realistic approach to program recommendations.
• Previous experience in the evaluation of the development programs of DFAT.
• No prior involvement in the implementation of APSP. Any potential conflict of interest must be disclosed at the time of application.
• Ability to ensure the team’s independence throughout the evaluation process to uphold the credibility and impartiality of the evaluation findings.

VII. Implementation Arrangements and Budget

The implementation of the evaluation of APSP will be jointly managed by DFAT (led by Australian Mission to ASEAN) and TAF (led by TAF Thailand). DFAT and TAF will jointly select the evaluation team and provide guidance to the evaluation team in the conduct of the evaluation. DFAT and TAF will jointly oversee and provide feedback to the outputs of the evaluation team.  

The external evaluation team will be responsible for delivering the evaluation outputs.

The evaluation will be funded by APSP.