Decision analysis as a basis for more effective agricultural innovation
In the recent bulletin covering the joint extension service offered by Navatec and Strides, Hector McNeill1, referred to the term decision analysis. The article below is Part 2 of a series of 4 with the following titles:
In this article Hector McNeill explains that decision analysis has a dynamic contribution to the management of project implementation after the decision on which project to implement has been taken.
Part 2: Designing resilient projects
The process of agricultural innovation consists of a sequence of linked projects covering:
Today a major gap exists in the low levels of application of decision analysis to project cycle management leading to lower than desirable project performance. The potential contribution of decision analysis increases with the evolution in analytical techniques and the increasing power and falling costs of communications and data processing technologies.
The quest for operational coherence
In Part 1 of this series on decision analysis I set out an outline of how deision analysis is used in establishing options for a project design based on simulation models. However, decisions on actions in any phase of a project implementation need to remain consistent with the then current conditions and feasible desired objectives in order to maintain opertional coherence. There is a tendency for project plans to be considered to be a fixed structure Unfortunately many project managers struggle to keep projects performing in line with the original plans and this can end up with project failures. It is often the case that few additional funds are made available to facilitate reallocations of resources in response to change. As a result project managers do not have enough leeway to carry out the necessary decisions and actions to keep a project on course in the face of change. This financial constraint exacerbates the negative impacts of some changes leading to declining performance.
In decision analysis a decision is defined as an irrevocable allocation of resources to the design option that has been seleted as a plan of action. Any changes in conditions requiring a change in a decision will result in the need to allocate additional resources. This makes the initial decision on a plan particularly important but it also acknowledges that allocations might have to be altered if conditions change. Decision analysis is therefore not limited to the project "planning" phase but it also has an important role in optimise decisions in response to changes that occur during the implementation post-planning phase.
Theory and practice
Most written guidelines on project cycle management acknowledge that plans may need to be altered in the light of developments. Very often changes in allocations or project focus come as a result of monitoring & evaluation (M&E) assignments. However, in practice, in spite of a logical and good intent, the time delays between the detection of critical changes requiring a decision through the carrying out of a monitoring and evaluation exercise and issuance of recommendations, tend to be excessive. Where there are small teams that do not combine the required capabilities in relevant disciplines there is often an over-reliance on monitoring and evaluation personnel when often they are not equipped to help in shaping immediate project level decisions because of an inadequate access to essential information. With the practice of subcontracting M&E assignment personnel there has been a default position arrived at where indicators drawn up at the beginning of a project assume a too much importance and evaluation becomes a comparison of expected indicator values with those achieved.
Project managers should not have to rely on an M&E report to take decisions to sustain a project's performance. Quite often adjustments require additional funding but in many cases contingency funds or authorization procedures are too complicated or time consuming. As a result M&E timing and outcomes and funding quantity is too small and access and timing delayed resulting in decisions coming far too late to benefit an ongoing project to the extent required and within the realms of feasibility. Therefore the project loses coherence because such delays result in the focus continuing to be the original plan contained in the Log Frame.
Although projects are considered to be a single operation they are made up of several related activities each with a specific type of transformation of inputs to outputs. In the Navatec System these are referred to as transforms and these make up the building blocks of the project decision analysis model. For example, Diagram 1 shows a single schematic of a standard transform unit (STU).
A simple 4-transform process
Inputs include consumable items refered to as variable inputs and whose quantity varies with the scale of operation of the specific transform. For example seed, fertilizer, energy and information and human resource inputs. Other inputs include fixed equipment and assets such as land and work areas and occasionally inputs or parts of processes are provided by external (outsourcing) services. The "simple" example provided can be seen to be quite complex and the design process needs to identify the most appropriate transform processes, to specificy inputs and outputs for each one, to identify adequately qualified individuals in terms of training, experience and capability to manage the process, to identify the quantities of fixed capital items and to establish a budget for all of these these inputs.
Assessing the impacts of potential changes
Changes that occur during project implementation usually affect transforms differentially. There are many types of possible change. These are identified and specified in the Navatec System due diligence design procedures as project design constraints initially as shown in the box on the right. There are currently 19 items each of which is associated with a core dataset consisting of around 5-6 items so the total dataset on constraints is some 100 items. These all make up contributions to the shaping of the decision analysis model of a project. The changes that occur during implementation are essentially changes in the properties of these constraints, so the completion of the constraints analysis embeds these in the model.
For example, limiting our analysis to two types of constraint on project performance:
These can change during implemtation relaxing or tightening constraints.
The natural environment and ecosystem include seasonal weather conditions and impacts of ecosystems that can affect specific processes within a project. For example a transform handling the husbandry of a crop would be associated with assumptions on timing of cultivations and expections in terms of the final production levels per hectare. It is self-evident that if temperatures, water conditions are not as expected, timing and growth of the crop will move from the projected schedules and expected production curves. If this results in a lower output then unit costs of production will rising because input costs will be applied to a smaller output.
As can be observed, the environmental changes (natural or financial) usually impact one transform and then there is a cascade effect as the change in perfomance of that transform affects downstream transform performance in terms of timing as well as expected output in quantitative or qualitative terms.
There are therefore two specific considerations in relation to reducing risks to a project with respect to changes. The first is to build into decision analysis models a way to include potential environmental and ecosystem changes and to ensure that the transforms most directly exposed to these changes are identified. It is also necessary to trace the impacts of changes in these transforms on downstream transforms.
Therefore besides the complexity of the design process to identify a project baseline consisting of a sequence of transform processes, inputs and outputs the decision analysis needs to be extended to manage the dynamic analysis of the more significant impact-cascades that can occur during implementation. This analysis can be undertaken conveniently making use of such operations research techniques as Monte Carlo Simulation which is a robust and proven method to end up with clear analyses to rank the degrees of risk of each transform to each type of potential change and to establish a means of ranking the overall risk to a project by type.
In the box on the left, which contains a short explanation, the small green and grey graphs represent "stochastic" inputs in that the particular property of the input that affects the project transform can be represented by a range of values and a specific distribution curve that has been determined by statistical analysis of data. By using feasible ranges (usually based on past observations and analysis of records) and likely or known distributions, the process model can be used to estimate the likely impacts of change. This is how Monte Carlo Simulation works. The examples reviewed only relate to two basic types of constraints and changes in constraints but the same project model can be used to setup any number of stochastic inputs and the Monte Carlo Simulation will generate the corresponsing outputs. There are around 100 basic properties related to possible constraints and all of these can be represented by stochastic input values. In this way Monte Carlo Simulation can be used to generate comprehensive calculation of the expected outputs at individual transform level, of potential changes in conditions during implementation.
The significance of this analytical capability
The output of MCS is a series of input and output distributions and by taking single input and output values it is possible to identify a single project option. Each option is a logic feasible association of output values with input values. In the Navatec System these instances or scenarios are referred to as Logical Process Options or Logical Project Options (LPOs).
Recording design options
Simulation provides a basis for assessing the potential impacts of changes in conditions on project performance. A project team can explore a wide range of conditions that can occur during implementation without committing any funds. This process provides teams with a deeper understanding and insight of the actual vulnerabilities of any project design and gain a deeper appreciation of which design options are likely to be more resilient to the likely changes.
This information is important in providing guidance to subsequent decisions required during implementation if and when conditions change. An optimized design consists of a combination of the best operational designs for each transform. Each combination of inputs, outputs and a sub-processes is represented by a Logical Process Option (LPO). LPOs are recorded in an Accumulog1. This approach to project design does not presume that there will be a single fixed plan since this is usually unrealistic. It is assumed that any project plan selected from the design options will be subject to changes in the future as conditions change. If and when these changes occur the Accumulog contents can help guide analysis and decision making. If no equivalent circumstances to those taking place are recorded in the Accumulog, then inputting the then current data can generate the appropriate decision-support within a few seconds.
In order to coordinate oversight and implementation decision-making, Navatec System makes use of a real time audit (RTA) system3 which monitors changes specific to each project in a portfolio. This provides oversight to help ensure a prioritisation of changes in terms of their potential impacts and a practical schedule of the required decisions. This combination serves to provide a tactical support to sustain a project's momentum and to complete the cycle having maximised the likelihood of achieving the original objectives.
This proactive Accumulog-based approach is distinct from the conventional approach to project planning, monitoring and evaluation based on a Log Frame which is based on a single project option. Log Frames usually carry no information of the design detail that explains why the content is the way it is. Without a full knowledge of design constraints and why the current option was selected it is difficult to provide informed advice as a result of an M&E mission. Quite often the people carrying out an M&E assignment do not possess the technical knowledge or knowhow that was necessary to design a project and to manage it. They are therefore often reduced to limiting their attention to the comparison of the indicators drawn up at project initiation with the state of indicators recorded during implementation. This has become a regular and instittionalised practice. With this information it is possible to measure "performance" but this provides no guidance on how a state of under-performance can be reversed in an optimised manner. The single Log Frame is somewhat like an initial LPO attempting to address inevitable changes in circumstances that will occur during the 1st through 2nd, 3rd implementation year or even beyond, resulting in a declining coherence between LPO assumptions and the evolving reality. Similarly, in the absense of such expert guidance on required adjustments, Log Frames can become increasingly irrelevant.
By way of contrast the Navatec System design approach, based on LPOs, does not assume that circumstances will remain the same over the life cycle and as a result all of the necessary resources are made available to facilitate LPO updates at a low cost and within a short analytical period. This has the effect of keeping LPOs and project plans completely up to date and thereby maintaining the coherence of the LPO content to changed circumstances.
The reiterative LPO simulation approach maintains coherence under changing conditions
There is no doubt that even with such analytical resources at hand it is unlikely that LPOs assumptions, that is, the quantitative data embedded in each design, can remain fully coherent with changed circumstances. Even if this were possible it is also likely that project performance expectations and realizations would not be the same as the original plan.
The LPO approach is currently the most effective means of helping managers of project portfolios and projects securing the best outcomes in accord with the original objectives. This has some fundamental advantages:
1 Accumulog - an Accumulog is a blockchain first identified in 1985 in an ITTTF initiative as an essential cumulative record for the support of learning and knowledge recall in learning systems. Originally prototyped in the Seel-Telesis decision support system in 1990, Navatec System is the first system to integrate this essential capability.
2 Due diligence design information - the information recorded in Accumulogs, in addition to LPOs, is gaps and needs analysis, constraints analysis, the feasibility envelope and prioritised list of feasible objectives and baseline design.
3 RTA-real time audit was devleloped to take advantae of the WWW global access to point data in projects. See RTA.Systems