Credit to Rasande Tyskar, Flickr
Much like development assistance in general, humanitarian aid to refugees is under increasing scrutiny to demonstrate its own effectiveness. This is particularly true in the context of massive refugee camps in the Middle East and Eastern Africa where donor money fuels the work of large international NGOs. Organisations working to fulfil basic needs like shelter, protection and livelihoods support are putting in place ever more elaborate systems for the monitoring and evaluation (“M&E”) of their service delivery. In rich, industrialised countries, on the other hand, those organisations that care for newly arrived displaced people operate differently. In Germany and Berlin in particular (where I live and work), there is no systematic M&E for improved impact in refugee support. I contend that this is due to the diversity of organisations and services they provide, as well as a reluctance to incorporate M&E in standard procedure. In this post, I take stock of the most important services and highlight aspects which prevent service providers from embracing impact orientation.
I base this piece on a broad definition of impact orientation: A service provider defines the goals of their work in such a way that the achievement of goals can be empirically verified. At the same time, they describe exactly how their activities will lead to the achievement of goals (i.e. they formulate a theory of change). This way, the potential impact generated by a service in the lives of the targeted population is attributable and measurable, and the service can be optimized to yield the desired impact in the most effective way.
When it comes to professionalism and effectiveness, the delivery mechanisms of basic services to asylum seekers and refugees in Germany are very diverse. There are large and well-established social welfare organisations linked to the two main Christian churches – Diakonie (Protestant) and Caritas (Catholic) – as well as equally established secular ones such as AWO (workers’ welfare) and the German Red Cross. These organisations run shelters for refugees in much the same way they operate hospitals, old-age care homes and kindergartens: with their own set of historically-grown standards and set practical priorities according to their philosophical ethos. In 2015, the Federal Consortium of Welfare Associations (which includes the four organisations above) published a position paper on impact orientation across all their sectors of care and assistance. The position paper supports the idea of impact orientation, arguing that by defining goals and using measurable indicators, providers of social services can both improve the quality of the services provided and make a concrete case for the necessity of their work by demonstrating the benefits created for the people they work with. At the same time, the consortium underlines the practical difficulties and high costs of accurately measuring impact in social work and therefore refrains from practically committing to M&E for impact.
Other shelter operators show less affinity to impact orientation, which has to do with the public administration that regulates them. In the last two years, a host of smaller and younger charities, as well as private companies joined the large welfare associations on the “market” of refugee shelter management in Germany. When it comes to commercial service providers, it is not up to the contractor to set standards, but rather the contracting institution sets the standards. In the Federal State of Berlin, the relevant authority – Landesamt für Flüchtlingsangelegenheiten (LAF) – defines only very minimal quality standards such as the “availability of sufficient and qualified personnel” and “implementation of social-integrative measures”. Upon an emailed enquiry to LAF about more detailed standards or goals (which might serve to measure the quality of service delivery), I was told that quality standards are currently being developed by the administration. No details about the nature of this process or the type of standards were given. For the time being, the authorities do not seem focused on the impact of the services they fund.
Another factor rendering M&E difficult is the role played by volunteers. Most if not all shelter operators rely on the work of local volunteers who assist in child care, organise cultural events, and teach German language. Since the “summer of migration” in 2015, the same volunteer force fuels a multitude of more specialised online and offline services, for example aiming to aid newcomers in finding their own place to stay and giving them access to various kinds of information. Volunteers are by nature very independent in the way they work which means that there is often no formal commitment to quality standards. In addition, M&E requires resources that volunteer initiatives often do not have.
The question of impact orientation is a complex one due to the diverse set of service providers – from the Red Cross down to local volunteer networks – working in a multitude of areas such as legal advice, language learning and psychosocial counselling. A 2013 study published by a consulting firm found that while the majority of 83 large and medium-size charities in Germany are aware of potential positive effects of impact orientation, only 13% say they are able to monitor their own impact without challenges. Many social workers also share a more categorical resistance to the idea of impact orientation as such which they see as an economization of social work. This critique fears that quantifying resources for maximum impact means that beneficiaries will be treated not as individual humans but make them into “output numbers”.
While service providers in refugee assistance do not currently monitor or evaluate their work for impact – either because they do not want to or are unable to do so – economists have started to generate insight about the effectiveness of support services. Following the recent period of unrestricted immigration by asylum-seekers in Germany (from the summer of 2015 to early 2016), business leaders became interested in the economic potential of newcomers. As a result, various studies trace the obstacles faced by refugees when entering the German labour market. In comparison to other groups of immigrants, refugees find qualified work at a slower pace. This is attributed to a lower level of preexisting relevant qualifications and fewer language skills, fewer interpersonal networks, and a delayed start to the job search due to lengthy asylum procedures. This body of research infers lessons for impact-oriented design of refugee support: if language skills, qualifications and networks are lacking then service providers need to focus on providing these in the most effective way possible. To some extent, these lessons are starting to shape services and policy. For example, many organisations and individual volunteers help refugees write out their CVs and use existing networks to circulate them. Economists at ifo-Institut in Munich are currently studying the impact of one such initiative (results to be published later this year). Conceivably, it is because the economic integration of refugees is a very concrete and measurable goal that impact orientation in this sector is as advanced as it is.
The mixed status quo of impact orientation in refugee support in Germany is due to the complex range of services provided as well as opposition to the idea itself. From a political perspective, caution is justified: in the last two years, debates around the “refugee question” are increasingly adversarial. Often, politicians do not discuss ways to help newcomers find a place in society, and instead talk about how to keep their numbers low. In this reductionist view, impact-oriented services risk becoming a mere tool to “manage” those in need of support at the lowest possible cost to the tax-payer. Impact orientation can be a powerful way of doing things, but it works to the advantage of refugees only if the defined goal is foster their full participation and inclusion in society.
