Bangladesh: Final External Project Evaluation Consultancy - Growing Together! Project

1.Background

1.1 About Humanity & Inclusion

Handicap International which runs program under its operating name Humanity & Inclusion(hereafter referred to as HI) is an independent and impartial international aid organisation working in situations of poverty and exclusion, conflict and disaster. Working alongside people with disabilities and other vulnerable groups, our action and testimony are focused on responding to their essential needs, improving their living conditions and promoting respect for their dignity and their fundamental rights. HI is currently implementing projects in almost 50 countries worldwide.

1.2 Context in which the project takes place

Growing Together! (GT) project is implemented in refugee camps and internal displacement settings in Bangladesh, Thailand and Pakistan. The project is directly implemented by country teams in each location and in addition counts with a regional coordination unit based in Thailand.

Bangladesh is one of the poorest countries in South Asia with one of the highest densities of population in the world – 160 million. Since 1991-92, close to one million refugees from Myanmar's northern Rakhine State of the Muslim ethnic minority, commonly referred to as the Rohingya, have sought protection in south-east Bangladesh (Cox’s Bazar district). The last influx, with more than 700,000 arrivals from August 2017 has created more pressure on resources and increasing humanitarian needs. Refugees remain in a chronic emergency situation due to long-term challenges to find a durable solution to their status in Myanmar and Bangladesh and therefore require continuous humanitarian assistance. The host Bangladeshi communities are also affected by the presence of a large refugee population living in the region. In addition, this coastal region is continuously dealing with the impacts of natural disasters, mostly cyclones and flooding, during which meagre assets of the local population are damaged or destroyed.

HI has been working in Bangladesh since 1997, with a focus on creating sustainable changes in terms of ensuring rights and inclusion of people with disabilities in society. In Cox Bazar’s district, HI has a strong knowledge and experience in chronic emergency response. Thanks to three projects implemented before the crisis, HI is providing services in rehabilitation, protection and PSS, with an inclusive mainstreaming approach to refugees and host population.

Pakistan has important strategic endowments and development potential. However, Pakistan faces significant economic, governance and security challenges to achieve durable development outcomes. The persistence of conflict in the border areas and security challenges throughout the country affects all aspects of life in Pakistan and impedes development. There is a new government in place since mid-2018, however the overall security situation still needs improvement. There are restrictions to media reporting in country. Another important development in 2018 was the government decision to merge the Federally Administered Tribal Area (FATA) with KP, raising hopes that such merger would aid the multi-sectorial development of this conflict affected border region. The relationship with neighboring countries, in particular with Afghanistan and India, is complex due to unresolved border issues such as dispute over Kashmir territory or militant groups crossing to/from Pakistan. Afghan/Pakistan border is also subject to thousands of migrant & refugee movements. There still remains over 1 million registered Afghan refugees in Pakistan.

Pakistan remains one of the lowest performers in the south Asia region on human development indicators, especially in education and stunting. Infant and under five mortality rates present a similar dynamic. Gender disparities persist in education, health and all economic sectors. Pakistan has one of the lowest female labor force participation rates in the region.

Thailand has a long-standing history in receiving refugees from neighbouring countries even though it is not a signatory to the 1951 Refugee Convention or its 1967 Protocol and does not have a formal national asylum framework. Myanmar refugees have been hosted on the Thai-Myanmar border for over three decades; currently, about 90,000 refugees live in nine temporary shelters. Camp management is controlled by the Thai authorities, in collaboration with locally appointed refugee committees. 13 NGOs that are gathered under the Committee for the Coordination of Services to Displaced Persons in Thailand (CCSDPT) are providing services within the camps. In Thailand, HI started working in 1984 and first focused its actions on the delivery of mobility and assistive devices. Thailand. In light of the political movements in Myanmar since early 2012 through the signing of the Nationwide Ceasefire Agreement and the general elections in late 2015, a platform for further dialogue on voluntary repatriation between key stakeholders led to the start of the facilitation of voluntary return. Although this process is still very slow, CCSDPT members are coordinating closely to timely react on all known scenarios. Aligning to these contextual changes, in 2016 HI Thailand became part of a regional program, by merging with the existing mission in Myanmar.

1.4 The project to be evaluated

Project name

Growing Together!

Project Goal

Enable children with disabilities and other vulnerable children (age 0-12) to develop their full potential on an equal basis with other children and acquire fundamental educational and social skills through inclusive play, arts, games, sport, culture, and early years’ education in displacement settings in Bangladesh, Pakistan and Thailand.

Location

Bangladesh: Kutupalong and Nayapara registered and makeshift camps and 17 villages in the host community of Ukhyia and 15 in Teknaf.

Pakistan: 7 villages in the host community of the Union Council of Lala Kalay in Peshawar District and 18 in the Union Council of Jalozai in Nowshera district.

Thailand: 7 out of the 9 temporary shelters along the Thai-Myanmar border. 2 of the camps in Mae Hong Son province started implementation in October 2018.

Target groups (Beneficiaries)

· 3,968 children with disabilities and other vulnerable children in early childhood (0 to 5) (50% girls and at least 25% with disabilities).

· Parents of children under 5 years old.

· 4,968 children with disabilities and other vulnerable children of primary school age (6-12) (50% boys/girls, at least 20% with disabilities).

· 91 local child-development service providers.

Partner

· Pakistan: CHIP

· Bangladesh: MUKTI

· Thailand: Karen Women Organization (KWO), Karen Refugee Committee Education Entity (KRCEE).

Funding

IKEA Foundation.

Start date

June 2016

Length of the project

54 months

1.5 Justification for calling upon a Consultant/External Evaluator

The Growing Together! (GT) project ends on the 30^th of November 2020. The project has been granted with a no-cost extension, where the implementation of activities will take place until May 31^st 2020 and the phase-out stage will take place from June to November 2020, the last 3 months being dedicated entirely to the administrative closure.

Approaching the end of the project’s implementation period, the aim is to conduct a final external evaluation to examine the project’s overall performance and effectiveness, identify intended and unintended impacts generated, consider the sustainability of the benefits produced to the population, and to identify best practices and lessons learned for future initiatives. As well, the project is looking to have recommendations based on the findings, aiming to support possible adjustments of the project’s approaches and interventions for replicability in other displacement contexts.

2. SCOPE AND OBJECTIVES OF THE EVALUATION

The objective of the final evaluation of GT! project is to measure the project’s impact in the life of beneficiaries, the achievement and sustainability of outcomes and objectives against the project’s performance indicators, to assess program effectiveness by comparing end-line results to qualitative and quantitative data collected at the program’s baseline and mid-term evaluation, as well as to analyze aspects of the project according to HI’s project quality framework, with a practical emphasis on a quality analysis of the results achieved and the expected and unexpected changes produced. The evaluation is intended to provide guidance and learning to IKEA Foundation, HI and its project partners that can inform the design of future programs. It should also look at the identification of good practices and lessons learned for 1) a possible project replication in other displacement contexts; 2) a phase 2 within the current project locations.

The expected outcomes are:

· A participatory external final evaluation is conducted on the effectiveness, efficiency, change, sustainability, capacity and participation of the action, providing a comprehensive understanding of project outcomes and impact, taking into account recommendations issued in the mid-term evaluation;

· Best practices and success stories, lessons learned and recommendations of the project are identified. This should contribute to feed 2 different scenarios: 1) adjustments of the project’s approaches and interventions for replicability in other displacement contexts paying particular attention to the use of similar approaches within both development and humanitarian contexts; 2) for a phase 2 of the project within the current locations;

· Evaluate the strengths and weakness of the GT project in the countries of intervention in terms of efficiency and effectiveness;

· Examine cross-cutting issues such as gender equity, resilience and extrapolate lessons learnt.

2.1 Evaluation grid

The consultant(s) will articulate their analysis around a set of evaluation questions and indicators as presented in the evaluation grid below in line with HI’s project quality framework (these were also used for the mid-term evaluation). These questions are not exhaustive and will be reviewed by the consultant during the desk phase. Any substantial change needs to be agreed with HI and reflected in the Inception Report. The following criteria should be looked into, though other criteria can be suggested by the evaluator:

Criteria, sub-criteria: evaluation related questions and indicators suggested:

CHANGES

1. Effects: The project brings about positive short- medium and long-term changes in the lives of the beneficiaries and targeted communities (intended and unintended)

· Nature, type and degree of change of the project key target groups/ beneficiaries.

· Feedback from beneficiaries on the various components of the project (e.g. parent clubs/ children clubs/ access to service providers, etc) have led to any tangible change with regards to inclusion of children with disabilities in the community.

· Feedback from service providers (schools/ rehab centres/ libraries/ pre-schools/ early year providers etc) on the level of “inclusion support’ provided and the alignment with their priorities/strategies.

1. Impact: To what extent can it be said that the effects/ impacts are attributable to the project intervention?

· Nature, type and degree of change of the project key target groups/ beneficiaries.

· Unintended (positive and negative) effects of the project.

· External factors which have played a role in the effects/ impact during the project implementation.

3. Mitigation: Any negative changes that might be brought about by the project are avoided, minimized or offset

· Absence of negative impacts and presence of a strategy to minimise/offset foreseen negative impacts.

CAPACITY

1. Competencies: The project helped to build the capacities of HI teams, community volunteers, partners and service providers on inclusive child development and child participation and continue to involve persons with disabilities in their activities.

· Extent to which project stakeholders gain general and specific technical skills thanks to their participation in the project, and if possibly continue their applicability.

SUSTAINABILITY

1. Anticipation: The post-project phase is anticipated and planned from the outset

· Existence and relevance of a quality exit strategy per country and for the project.

· Existence and relevance of an exit strategy discussed by HI, its partners and other relevant stakeholders.

2. Continuity: The intervention has met the identified needs of the populations and/or the project's activities have been transferred to other stakeholders in a position to continue it

· Existence of methodology and approaches used to ensure the transfer of capacities and their sustainability (including development of resources and tools).

· Degree to which the project results are likely to be sustainable beyond the project’s lifetime and provide specific recommendations for strengthening sustainability in the long run.

· Extent to which the capacity building of community structures is preparing them as a skilled workforce for the direct provision of services.

3. Resilience: To what extend the project has contributed to reduce the vulnerability of the beneficiaries

· Extent to which the project has contributed to the reduction of vulnerabilities faced by beneficiaries.

· Analyse the degree to which the existing project structures and technical approaches contribute to the reduction of vulnerabilities.

EFFECTIVENESS

1. Feasibility: The project has the necessary resources (human, financial, logistical, technical…) to achieve its objectives

· Gaps between the planned and needed resources and existence of proposed reorientations to adapt the project to the constraints faced.

· Gaps in the implementation of the project, including those gaps affecting its quality, due to insufficient resources.

2.Feasibility: The degree to which the project objectives and results have been achieved according to the plan.

· Difference between the planned expected activities, results and objectives and achieved activities, results and objectives.

· Factors explaining delays or failure to achieve objectives.

1. Consistency: The project implementation and activities contributed to the stated project results, goals and objectivities

· Level of effectiveness of the technical approaches (methodologies and tools) towards the project’s objectives.

· Was the project inclusive and adapted to the needs of all the beneficiaries whatever their gender, age or disability?

· Did partners contribute sufficiently and optimally to the outcomes of the project?

EFFICIENCY

1. Strategy:The implementation of the intervention made effective use of time and resources (financial, human) to achieve results

· Appropriateness of the budget allocation for achievement of the project results and objectives.

· Appropriateness of human resources for the project objectives (number and organisation of HR, technical expertise of teams, etc.).

· Clarity in the roles and responsibilities of project partners at all levels.

PARTICIPATION

1. Consultation: Beneficiaries, partners and other relevant stakeholders have been involved/consulted at every stage in the project cycle

· Existence of mechanisms for involvement of beneficiaries, partners and other relevant stakeholders in the project.

· Feedback from beneficiaries and other relevant stakeholders on their involvement in the project.

2. Expression: Beneficiaries and other relevant stakeholders have the means to express their opinions (complaint mechanisms, satisfaction surveys...)

· Existence of indications that the beneficiaries have been encouraged and empowered to express their level of satisfaction during the implementation phase.

· A gender-age-disability sensitive feedback mechanism is in place.

The evaluator should also identify any examples of good practices, which HI, as a key global player on disability and social inclusion in developing countries can disseminate. These may include tools, approaches, training materials, management practices etc. The evaluator should explain why this is considered a good practice and make suggestions on their wider applicability.

3. METHODOLOGY

The exact methodology should be proposed by the consultants in their applications.

The evaluation should consider the opinions of the different targeted actors and compare their views and perceptions on the project’s impact and supported changes if any. The methodology is required to have child-friendly approaches and a strong participatory focus where people with and without disabilities are consulted.

The methodology should include but not be limited to quantitative data. The sampling techniques and target groups will be developed by the consultant, as well as any protocol. The consultant will adopt a mixed approach where s/he will apply qualitative and quantitative methods. In order to calculate sample for acquiring quantitative information, the consultant should ensure 95 % confidence level and 5% of margin of error. Sampling should exhibit appropriate representation of target population (women, men, girls and boys as well as inclusion of impairments).

The technical feedback on the tools and the inception and final report will be delivered by the Regional Program Coordinator and the Regional Technical Coordinator.

A wide range of project documents and existing studies will be made available to the evaluators for their review. This includes baseline and endline reports, mid-term evaluation reports, products of the participatory M&E activities performed within the project, ScopeoKids baseline and endline reports, and GT technical framework amongst others.

The mission will consist in:

1. Desk phase, during which the consultant will:

· Review existing project documents and all other relevant documents;

· On this basis, develop the inception report (including evaluation protocol based on the evaluation grid presented above, including number of interviews and meetings; data collection tools: questionnaires, focus group guide and semi structured interviews guide and any other participatory methodology, work plan including the list of stakeholders to meet during the field phase);

· Initial Skype discussions will take place with the Regional Coordination, HQ and country teams (Program Directors, Operational and/or Technical Coordinator and Project Managers).

· Adjust the evaluation grid if needed;

Prepare the surveyors training and materials if applicable.

· Coordinate the translation of the tools from English to Karen, Burmese, Urdu and Bangla;

These elements will be combined in an inception report. HI team will validate the inception report.

Note: The endline report and endline ScopeoKids (Quality of Life) reports will be provided after the field phase by HI, so the evaluators can have a direct and neutral first impression of the project.

2.Field phase (steps/methodology to be detailed by the consultant)

· This includes the collection of primary data through direct consultations with key stakeholders and beneficiaries at field level.

Note: The consultant should have a Plan B for gathering this information in Pakistan, in case due to security/visa restrictions the consultant cannot travel and directly gather the data. Consistency and harmony in terms of approaches and quality of the findings is expected across locations.

3.A reporting phase during which the consultant will:

· Organise a debriefing workshop with HI GT Regional team, in Thailand or remotely at the end of the field phase to present the findings (Power Point presentation) with the aim of exchanging and sharing feedback;

· Submit the preliminary report to get comments and feedback from HI team (regional coordination, country teams and partners, and HQ) within 10 working days;

· Submission of a final survey report to HI of 50-pages maximum including Annexes.

4. DELIVERABLES

· Produce an inception report in English, including all proposed tools (protocol: sample size, data processing and rating systems, detailed schedule of FGDs, KII, etc), to be introduced at the end of the desk phase. The inception report and tools will have to be validated prior launching the field phase.

· Organize a restitution workshop to GT Regional. A PowerPoint presentation will be produced by the consultant, including:

I. A detailed explanation of the methodology and tools used and timeframe;

II. A preliminary analysis of findings/results of the survey;

III. First results and recommendations addressing each of the project’s components;

IV. Mains lessons learnt and suggestions for similar projects in the future or a continuation of the project in the locations.

· A final report (50 pages maximum incl. Annexes) in English, including the feedback from the debriefing workshop. HI will provide comments within 10 working days for the consultant to finalise/adjust the report, in the following 5 days. The final report should include the following sections:

• Executive summary (2-3 pages);

• Introduction to the context (1-2 pages);

• Evaluation methodology, including selection and sampling methods, and mention any constraints and challenges encountered, and strategies used to overcome them (3 pages);

• Detailed key findings and conclusions presented per criteria and including case studies and lessons learned (15 pages)

• Recommendations (3 pages) (including those to feed 2 different scenarios: 1) adjustments of the project’s approaches and interventions for replicability in other displacement contexts paying particular attention to the use of similar approaches within both development and humanitarian contexts; 2) for a phase 2 of the project within the current locations).

• Annexes – all data collection tools; success stories and best practices

• Database;

• List of persons met during the evaluation process and salient points of the meetings.

All reports will be delivered in English and the report will be introduced in soft copy.

Within the report confidentiality will be respected when representing personal information. A consent form needs to be used prior taking any photo, any inclusion of pictures of children will have the statement within the document: “All names & information about the location of children and family privacy in conformity with HI Child Protection Policy”

NB: For reasons of confidentiality, the evaluation report remains the intellectual property of HI exclusively.

6. TIMELINE

The evaluation consultancy is expected to start in March 2020, lasting approximately 52 days (without counting approximately 15 days of in between time for HI validation of the Inception and Final Report for final completion by the consultancy). The field phase in Thailand and Bangladesh should take place in May which is the last month of activities’ implementation (BGD: ideally 17^th to 21^st) and beginning of June in Pakistan’s case (1^st to 5^th June ideally) due to Ramadan in May. The deadline for the submission of the final evaluation report for HI comments is the 19^th June 2020. The final report, including HI validation should be submitted by the 10^thJuly 2020. The evaluation mission will be planned in accordance with the project team (HI and partners) and dependent on activities planned for the proposed timeframe. It is to note that HI expects to select the consultant by end of January, enabling enough time for visas applications.

7. PROFILE OF THE CONSULTANT

The evaluation can be carried out by an expert or a team of experts/ support staff.

If a team of experts is selected, the evaluation will be put under the responsibility of one team leader chosen among the team of experts. This person will ensure all communication with GT Regional team and will be the sole responsible for managing the organization of the evaluation.

The expert or team of experts should combine the following skills, experience and knowledge:

· Required:

o Written and spoken English;

o Proven experience in external project evaluations and in program development/ design, program management & implementation, monitoring & evaluation;

o Experience in project impact evaluation, including experience in evaluation of regional/ multi-country projects delivering a complex intervention;

o Knowledge about data collection and data analysis tools/methods used for five DAC-OECD evaluation criteria.

o Background in disability -inclusion, preferably with a working knowledge on child development.

o Good knowledge of MS office and specially MS Excel;

o Proven experience in data analysis and reporting;

o Experience in conducting participatory (qualitative and quantitative) evaluation techniques, including child-friendly techniques;

o Experience in refugee and IDPs contexts.

o Experience working in conservative cultural contexts.

· Preferred:

o Experience on child protection, child participation, inclusive education, early child development and early year education for children is an asset;

o Background in vulnerable/ marginalized groups preferably with a working knowledge on child development.

o Cross cultural & field-based experience in developing contexts

o Experience working with Non-Governmental Organizations; experience working with HI is an asset;

o Experience working in countries of intervention (Thailand, Bangladesh, Pakistan) is an asset;

o Knowledge of local language(s) is an asset.

8. FORMALITIES

Proposals from interested consultant(s) should include:

1. Letter of expression of interests, including how the skills and competencies described in the Terms of Reference are met (compulsory);

2. Curriculum vitae (compulsory) detailing the consultant’s experience and qualifications on impact evaluations and disability work; reference of previous assignments done or sample of work accomplished (if it is a team of consultants, all CVs should be included);

3. Technical proposal (compulsory) including the evaluation design and methodology, data collection and analysis, activities proposed to accomplish the objectives of the assignment. It should include a proposed timeline considering contextual limitations (see details below);

4. Financial proposal in USD (compulsory). All costs related to the consultancy without exceptions should be figured in the financial plan of the consultant, including consultancy fees differentiating i) field days; ii) desk days; and iii) report development), domestic and international travel if needed, visa, accommodation, interpreters, data entry, logistics, stationary, etc. (transportation by land to the refugee shelters will be provided by HI);

Note: No per diem will be paid to the consultant(s). Also, international travel days will not be considered as working days and will not be paid.

The consultant will be responsible for its own security in all three countries, HI will not cover any insurance fee during the consultancy period.

1. 3 references of which 2 should be from a previous similar experience;

2. List of relevant documents requested for the contractual process in case of selection (Passport, insurance …).

NOTE - Camp/ field visit limitations:

· The consultant will need to comply with HI security regulations of each country. A security briefing will be provided by the security focal point of each country before or upon arrival to the country.

· It is suggested that the consultant take a suitable amount of cash before living to the areas of activities implementation.

Thailand:

· No entry to the camps without camp-pass permission, which should be processed one month in advance the field activities.

· No camp visit during Bank Holidays.

· No work/interviews on the weekends at the temporary shelter level.

· Mae Sot is the main office for HI in Thailand. There are 4 direct one hour-daily flights from Bangkok to Mae Sot by NokAir.

· Tak camps: Mae La is 45 minutes by car from Mae Sot; Umpiem Mai and Nu Po camps are about 2 and 6 hours each from Mae Sot. There is accommodation at Umpang (mid-way from both camps).

· Mae Sariang camps: Mae La Oon and Mae Ra Ma Luang camps are 6 hours from Mae Sot by land. HI has a guesthouse in Mae Taw La village mid-way from both camps (no access to phone connection, limited access to internet.

· Mae Hong Son camps: Ban Mae Suring and Ban Mae Na Soi are 7 hours from Mae Sot by land. The consultant will stay in Ma Hong Son in a hotel. There are direct flights from Bangkok to Mae Hong Son.

Pakistan:

· No entry to the target area (Peshawar and Nowshera) without No Objection Certificate (NOC), permission from Home Department which is granted once the visa is issued; the documents should be submitted to the Home Department at least 2 weeks before the visit for the NOC to be issued.

· Activities can be only implemented from 8:30 am until 3:30 pm.

· Please note that Ramadan will likely start on April 24^th to May 24^th. In this case shorter days will need to be considered for activities with beneficiaries as they will be fasting.

· No work / interview on the weekends -Saturdays and Sundays.

· Respect to local cultural norms and standards and follow the local dress code in the field.

· In-case if the consultant is a male, he should always be accompanied by HI women staff during data collection.

· Separate group meetings with women, men, boy and girls as per local culture.

· Peshawar is 2.5 hours from Islamabad, Humanity & Inclusion (HI) office;

· Peshawar target area (Lala Kalay) is 1 hour from Peshawar, while Nowshera Jalozai target area is 1.5 hour from Peshawar.

· Mobility in Peshawar is limited due to security reasons.

Bangladesh

· Activities can only be implemented from 8:30am until 4:00 pm;

· “No entry to the camps without camp-pass permission which needs to be processed in situ with the applicant, submitting the visa and allows tourist/business visa holders to access the camps for 3 days only;

· No work / interview on the weekends, which are Fridays and Saturdays;

· Travel time from Dhaka towards Cox’s Bazar by air is 50 minutes;

· Travel from Cox’s Bazar to Ukhiya by car is about 45 minutes;

· Travel from Cox’s Bazar to Kutupalong and Teknaf host community is about 1.5 hour by car minutes.

Note:

Evaluation of the applications will be made through a selection committee in 2 phases:

· Administrative selection: checking for completeness of application (all compulsory items listed above in Formalities). Incomplete applications will not be taken into consideration for technical selection.

· Technical selection: criteria to select the best application will be based on the quality of the technical proposal, competitive financial proposal, human resources skills and previous experiences, demonstrated expertise of the applicant.

The deadline for submission of applications is 6^th of January 2020 at midnight Thailand time.

Proposals should be submitted to the following email: gt.recruitment@thailand.hi.org, including in the email subject: “GT Final Evaluation Consultancy”.

Only candidates who pass the administrative selection will be taken into consideration for the technical assessment and they will be afterwards notified of the final decision. Selected applicants may be invited for a (phone/skype) interview. Interviews will be conducted on the 13^th and 14^th of January.

HI reserves the right to contact the applicants for further information before the final selection of the selection committee.

“Humanity & Inclusionis committed to protect the rights of the children and opposes to all forms of child exploitation and child abuse. HI contractors must commit to protect children against exploitation and abuse”.