An impact assessment framework to evaluate the effectiveness of Information Communications Technology (ICT)-led development project at micro (community) level in developing country; Bangladesh

By: Md. Mahfuz Ashraf (Bangladesh)



Learn about the Project

Research in the multi-disciplinary domain of ICT and Development indicates there is potential for information and communication technologies (ICT) to contribute to a nation’s socio-economic, socio-technical and socio-cultural development. With this in mind, developing countries have been rushing to implement ambitious ICT for Development (ICT4D) projects in rural areas through the direct/indirect supervision of institutions such as the World Bank, the United Nations (UN) and other donor/local agencies. These interventions aim to provide positive developmental impacts on people’s lives at an individual, group or community level. Interestingly, the main focus of the interventions has been on the implementation of ICT4D projects themselves, rather than on understanding their impacts at the recipient or community level; and such lack of understanding has led to many failures of ICT4D projects as reported. This proposed research (two years in duration) will develop and test a conceptual framework to investigate ICT-led developmental impact in rural areas in the developing country, Bangladesh. The framework highlights some important perspectives on research into ICT and Development while understanding the intentions behind the adoption and implementation of ICT interventions in developing countries. Our framework will encourage further investigation into ICT-led development projects which explicitly acknowledges the perspectives of the to-be-affected community/ies. The nature of this research is explorative because little research has been undertaken within the socio-economic context of rural Bangladesh. While conducting the empirical study, we will focus on complex social actions in order to understand ICT-led development from the actual beneficiaries’ perspectives. Further, understanding ICT-led development is subject to social interpretation by different groups in different ways. Hence, this research employs an interpretive approach with the qualitative tradition for data collection and analysis.

Research question
How and to what extent do ICT-led development/ ICT4D projects influence the sustainable socio-economic progress at community level?

Problem background
There is a general consensus that ICT has potential impact on socio-economic progress. By realizing so, developing countries are rushing to implement multi-billion dollar ambitious ICT projects, which aim to use of ICT in education, poverty reduction, gender, health in urban-rural telecentres, Information Kiosks through the direct-indirect interventions of The United Nations (UN), the World Bank, the AusAID and the International Telecommunication Union (ITU) and other public-private bodies. Unfortunately, focus has been given to the supply side (infrastructure) of the interventions than understanding the demand (impact) towards socio-economic progress or development of the end users or beneficiaries (Heeks, 2002). Further, how far ICT projects address socio-economic progress at community level remains questionable (Adam & Wood, 1999). Without appropriate research on the ‘impact’ of ICT interventions, it is difficult to determine the extent to which, or even how, ICT influences or effects socio-economic progress. Though there exists some research, little attention has been paid to the participant, recipient, end users or actual beneficiary perspective at community level (Huerta & Sandoval-Almazan, 2007), which Keniston (2002) considers an ‘empirical vacuum’ of ICT impact research. This lack of understanding has led to many failures of ICT interventions in the literature (Heeks, 2002). Hence, international donor agencies and governments are calling for greater accountability in the planning and implementation of these projects. In order to measure the effectiveness of the investment or development, there is an increased demand for the development of reliable impact assessment frameworks to evaluate or assess the impact of the projects.

Building on existing work (Ashraf, Hanisch & Swatman, 2007) using data collected from three villages in a developing country; Bangladesh at the community level, a preliminary conceptual framework has been developed. This model indicates the need for a holistic impact assessment framework. Further work is needed to develop the assessment framework as a proof of concept for ICT4D projects taking into account of complex array of factors encompassing physical, digital, human, and social resources and relationships that exist within community level.

Justification
Too often ICT/4D impact assessment frameworks focus on the efficiency and availability of computer hardware and software, ‘paying insufficient attention to the human and social systems that must also change for technology to make a difference’ (Warschauer, 2002, p. 4). There are some impact assessment frameworks built in socio-technological perspective considering the challenges of integrating complex human and social systems, but those are limited at strategic and organisation levels, and community level is conspicuously absent. There is an urgent and clear need for an impact assessment framework that evaluates ICT initiatives from end-to-end, because the failure rate of ICT projects remains high, and despite considerable investment by donor agencies, it appears many projects are not providing long-term or sustainable improvements. As the current assessment of the impacts is disparate and superficial, it is difficult to determine the contextual factors associated with the projects. This may lead to overlooking varying degrees of bi-directional impacts of ICT and society as well as the failure to determine whether an ICT project established one country will be applicable in others. Table 1 presents a summary of the key challenges we have identified, and some conceptual frameworks, with their respective authors:

Whilst there is a variety of frameworks for assessing ICT projects, all of those reviewed rely heavily on short-term measures of success, including (for example) questionnaires set by donor agencies. None of the frameworks assesses the actual demand of the recipients at the micro level; and in addition, they all accept indicators for measuring ICT impact on development which are designed in one context and then applied in another. It might appear attractive for simplicity’s sake to derive indicators from one context and apply them to another, but there remains the problem caused by drawing theories or frameworks from other disciplines without taking into full account the local contexts and issues in developing countries. Heeks (2005a) argues that improved ICT projects need to be associated with local data content and local ICT skills: thus, indicators for measuring ICT impact on development should either be generated or adapted within their own, surrounding, local contexts.

Therefore, the ultimate objective of this research is to develop an impact assessment framework for understanding the processes of ICT-led socio-economic progress from the end-users’ perspective at local context and, finally, apply the framework to a range of case studies for further generating concepts and evaluating the impact of ICT interventions. The framework includes identifying the dynamic nature of the relationships between actors within community level.

The empirical study will take place in Bangladesh for an improved understanding the impact of an ICT4D project in Bangladesh, a developing country in Southeast Asia. It has the potential to benefit Bangladesh, as no such attempt has been taken so far; and it provides local knowledge, application and the experience of working in developed countries. In terms of economic self-interest, developing countries are becoming an attractive target and fertile ground in which to invest for developed countries (James 2007) . The notion is that ‘…technology can be profitable designed for the poor on a large scale with low profit margins’ (James 2007, p 291) — for instance, developed countries are providing low-cost computers to rural Telecenters in developing countries — so an improved understanding of the impact of ICT-led development in developing countries is useful for the developed countries’ diffusion and innovation of ICT.

The Government of Bangladesh has declared ICT ‘a thrust sector’ for national development, and initiated several ICT projects in the region (Gazette 2002). However, there is currently no reliable way to understand the effectiveness of such projects in that country; so [2] this research will be of significance for policy-makers. Our results are expected to improve understanding in planning future ICT4D projects, providing factors that influence their success or failure. This research contributes to this body of knowledge and its contributions will help guide policy-makers, international donor agencies and NGOs to a better understanding of ICT4D projects at the community level, from the participants’ point of view.

Findings

Adopting Sen’s (2001) notion of ‘development as freedom’ to understand our field results, we considered five basic freedoms and mapped the field results to these, as shown in following figure. Participants gained access to Gonokendra and GPCIC, where they participated in multi-media based information dissemination programs and become aware of different social-economical-cultural-political issues. They applied their knowledge in their daily lives, and disseminated information to others. We
found Sen’s (2001) approach to socio-economic development to be very useful as a foundation for analyzing ICT related development activities.

This research is an attempt to investigate the impact of ICT-led interventions in Bangladesh implemented by two organizations in two different villages, using a framework inspired by Sen’s (2001) five freedoms. The empirical work here is of a qualitative, interpretive, and exploratory nature, aimed at analysing the field data from the participants’ perspective, rather than from the programme objectives. We found positive reactions or outcomes to the activities of Gonokendra and GPCIC. There are some limitations in the conduct of this research, the most important of which relate to data collection and attribution of impacts, as other development activities were taking place in the villages simultaneously. Much effort was devoted to motivate the respondents to provide relevant and concrete data. The improved ICT4D impacts understanding through Sen’s five freedoms would help international aid agencies to better plan, monitor and evaluate future project development and implementation. This translates to national benefit by improving the effective delivery of aid to developing nations while improving the social benefit for developing countries. This framework will be applicable to rural-remote areas where community level impacts ICT are poorly understood. From our research in general and application of Sen’s five freedoms’ in particular, best practice can be developed with the potential to save time and costs through a framework with general applicability, thereby optimizing funding of future ICT projects. By focusing on local values and capabilities, participants are more likely to consider the longer-term environmental sustainability of their development aspirations. Further research would be required to refine the research approach to obtain more relevant data and to explore whether it is possible to better understand the role of ICT4D initiatives such as Gonokendra and GPCIC in the context of other developmental activities in a particular area.

PI’s Current Progress (To be made available soon)