SDGToolkit is the result of the evolution in developments of the activities of the George Boole Foundation's Decision Analysis Inititiave 2010-2020 (DAI). This involved specialised collaboration between SEEL-Systems Engineering Economics Lab, building and evaluating cloud-based module prototypes to the specification of OQSI (Open Quality Standards Initiative) recommendations.

This project was initiated as a result of dissatisfaction with project cycle management recommendations that have existed since the 1960s. This dissatisfaction arose from the high rate of failure of agricultural projects and direct experience from the results of project design, management and evaluation methods applied by the leading development agencies including the EU, EuropeAid, Sida, World Bank, FAO, DfID, USAID as well as private corporations. This work has included reviews of recommended project cycle management manuals and evaluation techniques such as OECD DAC.

The high rates and geographic extent of project failures was originally identified by an internal report at the World Bank in 1992 and these findings were reconfirmed in 2010. Because of the extent of the losses and associated impacts on local communities summing worldwide to something like $75 billion each year, the George Boole Foundation initiated the DAI in 2010, to identify solutions to this problem. The George Boole Foundation is the only private organization to have dedicated a decade long research and development effort to this issue.

The first stakeholder involvement in this effort involved researchers and extension agents in Mozambique in a prototyping workshop in 2011. This was funded by the European Union and was hosted by the Mozambique Institute for Agragrian Investigations (IIAM). Participants were asked to come up with any project proposal in areas that the partcipants had a good working experience. After these had been presented, the workshop cooordinator, Hector McNeill, then asked the five project teams to make the projects more resilient, lower cost, to be completed within a shorter time frame and to be more productive and sustainable. The national managers were concerned that because the workshop participants were all experienced agricultural extensionists and researchers, there might have been a negative reaction to this request. Indeed, at first the particpants seemed puzzled by this unexpected request but the reaction was positive and this new task was executed with considerable enthusiasm. This required an intensive sharing knowledge to come up with solutions. This, in reality, was McNeill's objective. All participants found this workshop to be instructive because just within the confines of the wokshop, and within a short period, the teams came up with major improvments in their project designs. Recalling this initial workshop, McNeill said,

"The purpose was to demonstrate to participants that they all possess sufficient knowledge that in combination could create impressive results or raise crucial questions, through intensive collaboration. It was also a demonstration that by moving outside standard "protocols" and "procedures", and focusing on different core factors including effectiveness, efficiency, sustainability in terms of feasibility, resilience and risk reduction, better proposals could be identified. Many participants confirmed to me, following the workshop, that they were happy to have participated because it encouraged them to ask and provide answers to more relevant questions which did not feature in normal research or project design protocols. But this is a learning process." "One workshop cannot do the trick. What was also evident was that several excellent questions could not be answered because there were no convenient analytical tools (ATs) available at that time. Appropriate ATs are essential to enable teams to gain rapid answers to more questions so as to end up with improved quantified evidence and with each step, refining design quality. Tools need to be simple, transparent and able to come up with useful answers in a short time. This is why ATs are a major feature in the SDGoolkit and became our core development activity."

DAI workshops have involved an international array of stakeholders including from Africa, South America as well as Central and Southern Europe whose involvement has been with projects ranging from experimental design to agricultural production investments through to policy decision making. Initiated in 2010, this work has been contributing to the formulation of improved project design methods of direct relevance to Agenda 2030 Sustainable Development Goals before Agenda 2030 was launched.

In May 2019, a software licensing issue resulted in an inability of providers to meet SDGToolkit's stringent target minimum operational costs for users. As a result, the whole system, which was to be launched in mid-2019, was re-coded in another language and linked to a new database system. As a result of this experience, the Foundation no longer makes use of any third party and proprietary IT software and all developments are based on inhouse resources. The new system, including personnel training, evaluations and testing, have resulted in functional upgrades, took more than 21 months to complete. The new stable version of SDGToolkit Software-as-a-Service will be launched next month.

An emphasis on design

Following a detailed evaluation of project cycle information management systems, the OQSI noted that very few systems support the project design process but rather provide a container, linked to a database, into which existing designs can be input. Quite often what is input is a schedule of activities typically a Log Frame associated with a Gantt Chart. In a world of advances in technologies and knowledge on the world's resources, most project plans lack explanations as to why specific configurations of activities have been selected, and this lack of evidence undermines confidence that any given project design represents the best feasible solution (BFS).

Most conventional project cycle management guidelines, issued by leading international development agencies, have not yet been adapted to the SDG requirements. The most significant gap in project identification and design activities is that most project teams still do not have access to the appropropriate ATs for optimising project designs.

As a result, it was to be expected that the Agenda 2030 international project portfolio performance has been shown to be failing to achieve critical SDG objectives. Economic growth has been directly correlated to rising income disparity, falling sustainability and rising temperatures. These facts were reported in the United Nation's 2019 Sustainable Development Report. Because of this misalignment of economic policies and SDG results, the policy level issue needs to be placed into the context of national priorities.

Covid-secure signifies that nothing in the operational structure of this service presents a risk of infection as long as users observe normal precautions within the service reception environment

Covid-19 secure

The basic approach to the SDGToolkit was to develop a cloud-based SaaS with an operational functional capability of surpassing the conventional visits-based international technical assistance approach. This is to be achieved by supplying a range of ATs to cover national constraints analysis, project gaps and needs analysis and ATs for responsive design and a range of monitoring and evaluation tools to support project implementation. In addition the SDGToolkit can provide support for any number of projects as a portfolio. Besides enhancing the quality of the technical support the parallel objective was to minimize the delivery cost of this advanced service to institutions and practitioners operating in low income countries. With this in mind, SDGToolkit does not require the purchase of any specialised software and all that is needed is a browser.

Angus Raeburn, Coordinator of the launch of SDGToolkit observed that,

"All operations are remote and therefore fortuitously "Covid-secure", although the pandemic issue was not part of the original specification. By Covid-secure we mean that nothing in the operational structure of this service presents any risk of infection as long as users observe normal precautions within the service reception environment. Running through a browser users do not have to share Apps or programs but can work separately on the basis of the browers in their laptops or tablets or hand held mobile smart phones. This can be done in an office, at home or in the field if there is adequate reception". "SEEL has estimated that with the introduction of suitable vaccinations the majority of the world's population will only receive adequate coverage by around 2025 or 2026 linked to logistics issues in rural regions in low income countries. Therefore the Covid-secure status of the toolkit has become an important feature."

Hector McNeill of the George Boole Foundation and lead designer of the system added,

"During the last decade in particular there has been a rise in income disparity and falling real incomes and it was apparent in 2010 that aid budgets were likely to decline in real terms because of the rapid rise in financialization linked to quantiative easing. Covid-19 has exacerbated this state of affairs lowering the funding ability of private and government donors and private investors resulting in significant reductions in budgets in aid and investment. Recently, the United Kingdom government reduced its foreign aid allocation by almost 30%, from 0.7% of GNP to 0.5% GNP resulting in a reduction of around £10 billion." "SDGToolkit.com addresses these issues to some extent by raising the productivity of funding and institutional budgets so that the negative impact of reduced budgets is significantly reduced."

What comes in the service package?

The toolkit is divided into a sequence of service areas each with a specific function and supported by a range of ATs. The names of the sections are as follows:

Source: White Paper "A summary of the SDGToolkit project design process", GBF,January, 2021.

GCA - Global Constraints Analysis 3DP - Due Diligence Design Procedures Designer - Selection of best feasible solution RTME - Real Time Monitoring & Evaluation

GCA-Global constraints analysis

The power of the keyboard After an intensive decade of reviews and analysis of existing project cycle management systems, published guidelines and the advent of Agenda 2030, OQSI has reconfigured how projects should be designed and managed. Before committing substantial public or private finance to projects the SDGToolkit ensures that the best options from the standpoint of feasibility, costs, potential benefits and associated risks are determined through simulations, based on decision analysis models. Emphasis is placed on resilience or adaptability to inevitable changing conditions. By investing additional time in decision analysis it is possible to avoid significant losses, sometimes involving £millions resulting from inability to adapt to change. Such costly mistakes arise from a misunderstanding of optimization as a single best solution when, in reality, optimization relates to the ability of a project to enable smooth adjustment to the most likely changes and still deliver close-to-original objectives. Today, there is an imperative to design and implement projects that reduce income disparity, are environmentally sustainable and contribute to reducing the rate in the rise of global temperatures in an increasingly unstable environment; resilience and adaptability are essential properties of good projects.

The GCA is an in depth analysis of national constraints that will affect any project and this includes projections of population, real incomes per capita and the identification of critical income thresholds and declining real incomes for the lowest income segments. The associated ATs include commodity balance sheet generation, real income projections, crop gross margin analysis to identify feasible production systems to address feasible unit prices, areas of additional agricultural land required to address food needs and the identification of critical limits of national carrying capacity.

Currently the GCA is supported by fifteen ATs and a further ten are under development in response to stakeholder suggestions.

These tools generate projections and a report in narrative, tabulation and graphic formats.

Avoiding misinterpretation of results

The narrative reports output by the system are human readable text that is generated automatically. This is to ensure that, in the case of more complex analyses, the results are fully understood and to avoid misinterpetation. Angus Raeburn commented that, in some analyses, these systems-generated narratives are an essential support to avoid decisions leading to the assignment of resources to inappropriate priorities.

3DP-Due diligence design procedure

The 3DP is a stepwise series of procedures that basically check everything that might be relevant to the good design and operation of a project. There are currently forty seven procedures divided into 10 subgroups and 50 ATs:

• Registration - 1 tool
• Population, cultural education and health status - 7 tools
• Economy, livelihoods, markets and logistics - 9 tools
• Environment, ecosystems, carrying capacity & sustainability - 16 tools
• Recorded selection criteria - 5 tools
• Constitutional & administrative constraints - 6 tools
• Cross-relationships - 1 tool
• Additional information - 2 tools
• Critical Economic Rates of Return - 2 tools
• Critical Rates of Return to Environment - 2 tools

Some procedures, depending upon the nature of a project's objectives, may not be relevant and can be skipped but with a recorded justification. Angus Raeburn commented on the large range of tools by stating,

"These represent a core requirement to design any project. However, it is very apparent from feedback from evaluations and stakeholders that there is a lot of room for future refinements as well as additional tools. Any future customers would receive these as they are released as part of the service, free of charge. Indeed, the more proactively customers can provide feedback on their specfic requirments the more rapidly we can improve and/or develop new tools to add to the analytical tools library. There is no need for an App store or need to download and install anything, everything is just a click away and already installed in the system. All that customers need is a browser and their assigned security details."

The OQSI and SEEL have addressed the main performance gaps in the Agenda 2030 portfolio identified in the 2019 Sustainable Development Report by developing a series of specific analytical tools under the name of Options Benefit Analysis (OBA). These enable project designers address the issues of:

• Economic rate of return (ERR) determined on the basis of:


• costs and benefits as financial return
• costs and benefits as real incomes


• Rate of return to the environment (RRE) determined on the basis of:


• costs and benefits as carbon footprint reduction
• costs abnd benefits as carrying capacity equilibrium or rise

in a very practical results-oriented fashion. These tools are geared to simplicity, transparency and practicality and do not require elaborate cross-references to carbon footprint recipes for example which, in many countries as typical inputs, do not exist. The specific OBA system calculations have so far failed to appear in most project cycle manuals because most guidance does not emphasize any detailed design methods. OBA is unique to SDGToolkit.

The colletion can be used to calculate Sustainable Critical Path options analysis outcomes as a reiterative procedure of comparing the impacts of changes in technology, or input-output, on the OBA profiles as a basis for selecting the most suitable project design. Applying this system there is little reason why each of these critical gaps are not addressed in an optimised coordinated fashion.

Moving from TOC to a FCS In the light of the very large sums of donor and international development agency funds dedicated to projects, the OQSI was disatisfied with the lack of robustness and quantification of such evaluation or recording methods as: Theory of Change (TOC) Log Frame Analysis SWAT analysis All lack specific methods to quantify to risk, resilience and sustainability lowering the levels of confidence applied to the determination of economic rates of return and more importantly rates of return to the environment. The SDGToolkit is designed to provide the basis and evidence to issues a best Feasible Change Strategy (FCS) as a specific plan with quantified time-based trajectories for likely performance.

Angus Raeburn in explaining this particular set of ATs said,

"A design system needs to provide project teams with a high degree of certainty that their project design can reduce income disparity, raise sustainability and, in particular, not contribute to CO2 emissions and preferably to reduce CO2 emissions by a quantified amount. The OBA system is designed to achieve this in a simple, direct and practical manner."

Setting a high project design bar

The 3DP generates a priority ranking of levels of risk and needed action in relation to each of 50 factors. Unlike any other project management information systems, SDGToolkit embeds some of the more important declared requirements and standards of international development agencies by providing analytical tools to facilitate compliance with these by project design teams. For example, the World Bank Operation Procedure (OP: 10.04), the Economic Evaluation of Investment Operations².

However, SEEL research has established that the increasing instability caused by seasonal variations in meteorological conditions as opposed to slower changes in average climatic conditions e.g. temperature require a complete revision of the current financial appraisal norms. The details of the SDGToolki's handling of this issues is covered in an article by Nevit Turk entitled, "The role of SDGToolkit in improved economic appraissals of projects".

Designer

Designer is a project design input tool that makes use of the more than 50 tool outputs of the 3DP which ranks priority actions into an operational change strategy. This tool guides, the inputs of task sequences and timing, task processes, inputs and outputs, human resources inputs in support of each task and capital equipment used. All inputs and outputs (where relevant) are also associated with prices and quantities to generate budgets as well as cost-benefit analyses.

The human resources system has some unique features in containing terms of reference (ToR) for each task so as to raise the likelihood of a close match between the project's human resource profile requirements and any individual employed to undertake this work as an employee or contractor. A planned upgrade includes a way to post ToRs online from within the SDGToolkit to attract applicants and an online profile form to enable applicants to submit their CVs for consideration.

Rather than a Log Frame, Designer generates a Logical Project Option (LPO) which is characterised as the best current feasible solution but it is acknowledged that the LPO is likely to undergo alterations depending upon inevitable changes in conditions during implementation. In this context, SDGToolkit has a more practical approach to project budget tranche management. This approach avoids the common problem associated with fixed budgets which can end up causing project failures. This has been a problem associated with Log Frames that can default, for convenience sake, to becoming a single target solution upon which a fixed budget is decided and thereby losing essential flexibility. This can result in a cascade of events which undermine the effective value of monitoring and evaluation³.

Real Time Monitoring & Evaluation

The RTME system provides access to the selected project design, including: activities, time schedules, resources used in each activity and expected output input by the Designer module. This data establishes the performance expectations and which become benchmarks for subsequent monitoring and evaluation. It contains the Logical Project Option (LPO), the optimised project design presented in the form of a more detailed Log Frame but providing an ability to access more detailed information on each activity. There are also phase and whole project Gantt Charts.

The RTME provides an on-demand analysis and reporting response system for oversight personnel such as project managers and donor portfolio managers. A series of tools provide an operational internal monitoring and evaluation system where each activity can be assessed and performance recorded as well as decisions taken to correct under-performance. This internal monitoring and evaluation exercise would be carried out on the completion of each project activity and remain in the system's database. Any external monitoring and evaluation personnel have access to all of the project design steps from GCA, 3DP and project design performance simulation results. In addition to this rich project foundation they also have access to the internal monitoring and evaluation records.

Portfolio management utilities

The toolkit has a built-in real time, on-demand analysis and reporting system on any current status of any task. This provides access to standard preformatted template reports or by using queries to reports of requested formats. This allows portfolio managers to compare items of performance for single or any number of projects across a portfolio, in tabular and graphic formats and in one view. This part of the toolkit will be the subject of a major extension in order to embed the portfolio data warehouse concept developed by SEEL to support extension services and advisory activities with a source of accumulating knowledge on comparative practices.

Current tentative launch schedule

The range of activities required to launch this system is large and the tentative launch schedule is provided below which the Foundation wants wishes to satisfy.

Event Description Date Submission of application to GAG Requested evaluation by Global Action Group in the context of Agenda 2030 SDG projects. If accepted the response is expected some time in mid-April, 2021. 12 February, 2021 Website launch Site now accessible at:   SDGToolkit.com 22nd April, 2021 Website launch Boolean.org.uk to lauch new website with content covering the SDGToolkit 20th May, 2021, 2021 Release of White papers Publication of a series of WPs providing descriptions of the objectives and functionalities included in the system made available through the new Boolean Library section as from 25th April, 2021 General announcements Package provisions and offers through The Sustainable Development Facility; terms of use - timing depending on donor and sponsor negotiations. May 20th, 2021

---

¹ The OQSI project evaluation criteria recommendations are more exensive and detailed than those provided by OECD DAC. The additional criterion of "resilience" covers the risk assessments based on quantitative data generated by the GCA and 3DP and OBA analyses

² "A Review of the Valuation of Environmental Costs and Benefits in World Bank Projects", Paper No. 94, Environmental economics series, WB, 2003.

³ Projects require an ability to adjust performance benchmarks according to unavoidable change. Otherwise performance criteria fixed at a project launch are applied to what is, essentially, a different project. Performance needs to be related current attainable targets which might change according to external and internal factors. The timing and quality of decision analysis applied in response to change will determine the degree to which change causes variance of outcomes from original objectives. Therefore the "performance" of decisions are an important M&E metric measured by the difference between decisions leading to altered operations and the attainment of the original objective compared with no decision being taken. The SDGToolkit embed the required internal decision recording and monitoring capabilities.

The other problem with Log Frames is that they are often used to fix budgets which often causes subsequent problems of lack of flexibility to accommodate change leading to a need to downgrade indicators (objectives) and preventing adequate actions due to lack of assigned financial resources.

Posted: 20100417 We welcome questions and feedback:   To submit questions or comments on the contents of this article please contact the author or main reference source by email. The relevant emails are provided below:   Author:   John Penrose:     penrose@agroinfosys.org         Source:   SDGToolkit:     angus.raeburn@sdgtoolkit.com

---

APEurope.org