Open Quality Standards Initiative's project due diligence design procedures - are they practical?
The Development Intelligence Organization's annual "Leading Issues" workshop took place in Alexandria, Virginia, 9-11 August, 2019. The Keynote address by Hector McNeill of the George Boole Foundation (GBF) was extensively discussed. This outlined a set of new evaluation criteria for projects, programmes and policies, in the context of Agenda 2030 Sustainable Development Goals. In particular, these have been designed to accommodate and adjust project design and implementation management to support Reduced Inequality (SDG10), Responsible Consumption and Production (SDG12) and Climate Action (SDG13). Hector McNeill, the Director of GBF, is a leading development economist who has worked in this field now for over 50 years. He is the world's leading developer of the Real Incomes Approach to economic development having initiated this work in 1975. This specific topic was covered in a separate interview that took place in 2015.
This interview, is divided into three parts. The first part deals with the problems the OQSI work has been designed to solve. These relate to gaps in current practice that exacerbate the state of affairs of SDGs 10, 12 and 13 as well as many others. Once this background is clarified it is fairly easy to understand the purpose and likely impact of the OQSI's new project due diligence design procedures (3DP). Below is the content of Part 2 covering some of the shortcomings in common administrative arrangements and procedures applied to project cycle management and Part 3 will cover the OQSI proposed solutions where I assess their practicality.
(Inserted note: The relevant section of the report referred to is contained in the box on the right - N. Turk)
Project memory is a documented record of all of the cumulative experience, analyses, narratives and data generated by all activities contributing to the projects identification, design and implementation, including implementation decisions and their results. A review of thousands of projects by the World Bank Evaluation Group (2010) reported that the completion of tasks by different departments and institutions led to a fragmentation of information on a project, there was a tendency for documents covering previous phases to be missing at evaluation assignment stages. In some cases the results of economic rate of return calculations were missing. A particular issue linked to this poor documentation is the damage done to project memory when a key member of a project team leaves at any stage in the cycle. Those taking their place have no good documentary sources to come up to speed. Naturally this situation also results in evaluation assignment personnel not having access to complete sets of documentation.
In short, the current administrative arrangements are inadequate both in terms of oversight of use of funds, the quality of design and of management of project, programme and policy implementations.
Nevit Turk: But what are the alternatives? I realize this is a bit bureaucratic but how does this relate to Agenda 2030?
Hector McNeill: In addition to administrative structures working against a useful and coherent project memory, the live communications are also inadequate. In spite of W3 (world wide web) capabilities we still observe very slow reporting cycles to fit in with internal administrative procedures within government, development banks and other donor organizations which is exacerbated by the distance of these organizations from the project level activities. For example, if a portfolio manager is asked about the progress in any particular project, reference is made to reports that often have passed through a highly tiered reporting structure. This not only takes time but also can contain out of date and even distorted information.
Nevit Turk: Why is this information distorted?
Hector McNeill: In highly-tiered reporting structures each administrative level adds information to project reports which reflects the interests of those vetting reports at each level. Because performance at any level is a reflection of the effectiveness of management of that level there can be uses of words more designed to divert attention from the possible identification of the facts surrounding performance deficiencies. Also, sometimes a project can be delayed because of internal administrative procedures. For example, Ministries of Finance often operate on the basis of a standard accounting year, and don't coordinate payment transfers to agricultural projects to fit in with the agricultural year. As a result funds can be made available at inappropriate times. I have seen cases where projects have been delayed because of this type of inefficiency causing projects to face ver a year in delays in implementation. There are also examples of international agencies who supply the funding, following established time lines for budget line activity and simply closing the budget lines of perfectly good projects, with competent teams, so as to end up with a failed projects. Such failures can be the result of failures in communications caused by inappropriate administrative procedures. The closing of budget lines can be related to tidying up officials' portfolios to show efficiency in completing budget allocations and completion of internal admin cycles, with no insights or concern with the project level circumstances. In other words emphasis on internal adminitrator's performance can take precedence over the substanative issue, project performance.
Just as an example of another administrative constraint. Procurement procedures are quite often not handled by a project team but by specialized sections of Ministries of Finance or Agriculture. Project teams seldom have any direct experience with the legal frameworks, regulations or procedures involved in procurement. They are just expected to comply with the regulations and procedures. The details are managed by separate agencies. A common oversight in project designs and proposals is to not allow sufficient time for the normal procurement cycles to be completed. These should include, for example, calls for tenders, assessment of tenders, contracting and then time to product and service delivery to the project, commissioning at setup ready for use. Many research projects become delayed because of this simple oversight and production seasons can be lost in the case of agricultural production projects with essential equipment turning up afer long delays.
Nevit Turk: But isn't that sort of thing picked up by evaluation teams in lessons learned?
Hector McNeill: Well, you are highlighting what is part of the problem, rather than the solution. What is the point of waiting for this to happen and then organizing an evaluation assignment only to be informed this happened and classify it as an issue to be avoided in the future? The system needs to be organized so that this could never happen. It should also be organized so that any problems and lessons learned should be applied in real time so as to benefit the project in question. Being informed about such matters after the event and sometimes after a project has been completed is hardly an effective use of resources. For example by including financial and procurement administrative procedures as part of the identification phase constraints analysis, such risks can be identified and be accounted for in designs and project plans.
Nevit Turk: How can this be achieved?
Hector McNeill There is a need to short-circuit the process between on the ground analysis of gaps and needs, identifying priorities, designing solutions and then assigning funds to those that have been identified as priorities. As OQSI work has identified, in the multi-organization system, there is a problem with communications and lack of coherence in records but also nationally, this is exacerbated by the vertical distribution of qualified human resources with too few working at the design and implementation levels. As a result, with the growth in complexity associated with SDG solutions design and management, the issue of the number of people available to undertake the necessary work of an acceptable quality is a, so-far, hardly mentioned problem.
We know the rate of project failure under the current system is costing some $75 billion each year, so there is a need to solve this issue while also improving the relevance of all projects to solving the climate crisis. The evolving crisis in climate action requires more competent practitioners to work directly on projects so as to ensure sustainabilty and climatic positive projects through sound ERRs (Economic Rates of Return) combined with sound RREs (Rate of Return to the Environment).
An essential factor in successful projects in the involvement of stakeholders in the whole project cycle. Stakeholders need to include both potential beneficiaries as well as those who might be prejudiced by the growth of some activity. An early involvement of all is an important aspect of gaps and needs analysis so as to fine tune the identification of project targets and thereby clarify the existing constraints facing those targets. Too often, stakeholder involvement is almost cosmetic but, from my own experience, such involvements require more effort because this helps to avoid over and under-ambition and help reduce likely risks. OQSI procedures do make provisions for the involvement of stakeholders including oversight of project performance.
Nevit Turk: So we not only face an administrative constraint but also one related to the numbers and distribution of qualified manpower. But how can these issues be addressed within the sort of periods set out as time lines for projects, after all, Agenda 2030 has set the year 2030 as a milestone in progress whereas 5 years into the programme we are going backwards and there are only 15 years left. How can the necessary changes be accomplished in time to have any positive impact?
Hector McNeill Well, you have summed up the nature of the crisis facing us in your question. This is the reason OQSI altered its focus after 2015 to ensure that the issues of technology and know how transfer to more people can be accomplished within a short period. Fortunately modern technologies and the W3 (world wide web) offer an effective solution. The focus has been on the development of good practice meaning introducing procedural methods that facilitate the identification of priorities by providing support for project designers. The foundation of this development is a set of due diligence design procedures (3DP) that cover all relevant factors in order for these to be given due consideration. Rather than leave required analysis as narrative in a guideline the OQSI system is accompanied by analytical tools to complete necessary calculation, simulations and projections. This lays down the foundation upon which to build project memories to which to add all design information, proposal appraisal information, implementation events, decisions and results and evaluations. An important requirement is that project memory remains as an accessible resource to all with an interest in a project.
Nevit Turk: Thank you for explaining these administrative constraints that need to be overcome and the direction OQSI is going. It isn't clear to me yet how this fundamental issue with macroeconomic policies, being an important part of the problem facing SDGs, will be addressed. There would appear to be issues associated with the quality of project design and a major issue related to the need to upgrade the capabilities of the people involved in managing project cycles. Potentially this must involves an enormous number of people but who currently are not in place or do not exist. In the last part of our interview I look forward to hearing how the OQSI recommendations address these issues in a practical manner.