Home page

Decision analysis as a basis for more effective agricultural innovation

In the recent bulletin covering the joint extension service offered by Navatec and Strides, Hector McNeill1, referred to the term decision analysis. The article below is Part 2 of a series of 4 with the following titles:

  • Part 1: What is Decision Analysis and how does it work?
  • Part 2: Designing resilient projects
  • Part 3: Tacit knowledge & performance
  • Part 4: The cloud-based decision analysis tool box

In this article Hector McNeill explains that decision analysis has a dynamic contribution to the management of project implementation after the decision on which project to implement has been taken.

Part 2: Designing resilient projects
The power of the keyboard


Before committing substantial public or private finance to actions it is preferable to determine the best options from the standpoint of feasibility, costs and risks through simulations, based on decision analysis models. By investing in decision analysis, millions if not billions can be saved by avoiding losses resulting from costly mistakes, lack of optimization, lack of appreciation of the dependency of a project on specific conditions and "over-runs".


Summary

Decision analysis is a precise method of taking into account the factors that determine the outcomes of decisions to allocate resources to some action with a defined objective. It can be used to identify projects, support their design and understanding of their degrees of resilience and sustainability. It is an essential tool in the management of project risk.

The work undertaken to design a project is sometimes limited to the preparation of a feasibiity study that explores all of the options for securing a given objective. However feasibility studies need to state the types of support project management requires to ensure success.

Projects can be one-off actions with a limited life, such as surveys or some research projects. However, most projects need to be sufficiently resilient and sustainable to continue into the future beyond the period during which project operations require investor or donor funding. In all cases there is a requirement for decision support over the funded activity phases and beyond. The nature of the required decisions change with project and post-project status phases.

External investor or donor funded project decisions cover:
  • the design phase
  • the project setup phase which includes procurment
  • the implementation phase
Internally generated funding from sales of goods or service project decisions cover:
  • post-investment operational phase following the completion of the funded setup and implementation phases

The process of agricultural innovation consists of a sequence of linked projects covering:
  • research
  • proof of concept and prototype demonstrations
  • feasibility studies and commercial investment
  • production and sales
Clearly there is a need for sound decision analysis throughout this process.

Today a major gap exists in the low levels of application of decision analysis to project cycle management leading to lower than desirable project performance. The potential contribution of decision analysis increases with the evolution in analytical techniques and the increasing power and falling costs of communications and data processing technologies.



The quest for operational coherence

In Part 1 of this series on decision analysis I set out an outline of how deision analysis is used in establishing options for a project design based on simulation models. However, decisions on actions in any phase of a project implementation need to remain consistent with the then current conditions and feasible desired objectives in order to maintain opertional coherence. There is a tendency for project plans to be considered to be a fixed structure Unfortunately many project managers struggle to keep projects performing in line with the original plans and this can end up with project failures. It is often the case that few additional funds are made available to facilitate reallocations of resources in response to change. As a result project managers do not have enough leeway to carry out the necessary decisions and actions to keep a project on course in the face of change. This financial constraint exacerbates the negative impacts of some changes leading to declining performance.

In decision analysis a decision is defined as an irrevocable allocation of resources to the design option that has been seleted as a plan of action. Any changes in conditions requiring a change in a decision will result in the need to allocate additional resources. This makes the initial decision on a plan particularly important but it also acknowledges that allocations might have to be altered if conditions change. Decision analysis is therefore not limited to the project "planning" phase but it also has an important role in optimise decisions in response to changes that occur during the implementation post-planning phase.

Theory and practice

Most written guidelines on project cycle management acknowledge that plans may need to be altered in the light of developments. Very often changes in allocations or project focus come as a result of monitoring & evaluation (M&E) assignments. However, in practice, in spite of a logical and good intent, the time delays between the detection of critical changes requiring a decision through the carrying out of a monitoring and evaluation exercise and issuance of recommendations, tend to be excessive. Where there are small teams that do not combine the required capabilities in relevant disciplines there is often an over-reliance on monitoring and evaluation personnel when often they are not equipped to help in shaping immediate project level decisions because of an inadequate access to essential information. With the practice of subcontracting M&E assignment personnel there has been a default position arrived at where indicators drawn up at the beginning of a project assume a too much importance and evaluation becomes a comparison of expected indicator values with those achieved.

Project managers should not have to rely on an M&E report to take decisions to sustain a project's performance. Quite often adjustments require additional funding but in many cases contingency funds or authorization procedures are too complicated or time consuming. As a result M&E timing and outcomes and funding quantity is too small and access and timing delayed resulting in decisions coming far too late to benefit an ongoing project to the extent required and within the realms of feasibility. Therefore the project loses coherence because such delays result in the focus continuing to be the original plan contained in the Log Frame.

Project structures

Although projects are considered to be a single operation they are made up of several related activities each with a specific type of transformation of inputs to outputs. In the Navatec System these are referred to as transforms and these make up the building blocks of the project decision analysis model. For example, Diagram 1 shows a single schematic of a standard transform unit (STU).

Diagram 1


The STU consists of a simple input-output module used to define the process in each transform. Diagram 2 shows a simple project consisting of 4 transforms. As can be seen, this is a supply chain.

Diagram 2

A simple 4-transform process


For example, the project's INTERNAL process structure might be made up of 4 transforms


Inputs include consumable items refered to as variable inputs and whose quantity varies with the scale of operation of the specific transform. For example seed, fertilizer, energy and information and human resource inputs. Other inputs include fixed equipment and assets such as land and work areas and occasionally inputs or parts of processes are provided by external (outsourcing) services. The "simple" example provided can be seen to be quite complex and the design process needs to identify the most appropriate transform processes, to specificy inputs and outputs for each one, to identify adequately qualified individuals in terms of training, experience and capability to manage the process, to identify the quantities of fixed capital items and to establish a budget for all of these these inputs.
Some primary constraints

Constraints analysis is used to define a feasibility envelope within which a project can find resilience. However it is the constraints that also can change during implementation.

  • Cultural & social issues
  • Economic issues
  • Factor markets
  • Produce & service markets
  • Logistics
  • Environmental factors
  • Ecosystems
  • Key location co-ordinates
  • Key location distances
  • Other locational factors
  • Eligibility
  • Legal & regulatory
  • Administrative procedures
  • Financial criteria & budget
  • Other relevant initiatives
  • Wellbeing
  • Environmental sustainability
  • Climate change & impacts
  • Governance of x-cutting issues

Based on the OQSI:1 (2017) recommendation subject to update in June 2018.

Assessing the impacts of potential changes

Changes that occur during project implementation usually affect transforms differentially. There are many types of possible change. These are identified and specified in the Navatec System due diligence design procedures as project design constraints initially as shown in the box on the right. There are currently 19 items each of which is associated with a core dataset consisting of around 5-6 items so the total dataset on constraints is some 100 items. These all make up contributions to the shaping of the decision analysis model of a project. The changes that occur during implementation are essentially changes in the properties of these constraints, so the completion of the constraints analysis embeds these in the model.

Diagram 3

Detail of STU showing 2 environments







For example, limiting our analysis to two types of constraint on project performance:

  • The natural environment & ecosystem
  • Administrative procedures

These can change during implemtation relaxing or tightening constraints.

The natural environment and ecosystem include seasonal weather conditions and impacts of ecosystems that can affect specific processes within a project. For example a transform handling the husbandry of a crop would be associated with assumptions on timing of cultivations and expections in terms of the final production levels per hectare. It is self-evident that if temperatures, water conditions are not as expected, timing and growth of the crop will move from the projected schedules and expected production curves. If this results in a lower output then unit costs of production will rising because input costs will be applied to a smaller output.

Diagram 4

A four STU project showing 2 environments

Administrative procedures can include the methods applied to such things as equipment procurement, project tranche payments and the the like. For example if a project schedule concerning procurement is unrealistically estimated it is often the case that delays in procurement slow down a whole project waiting for equipment in otder to start operations.Another issue, which still persists, is the handling of fund releases by ministries of finance who do not recognise agricultural production years but apply standard accounting years. This can often mean in the first year of a project, funds are made available too late for a project management to arrange for resources to be applied at the appropriate time in a season. This can lead to projects having to roll over to the next year losing the initial projected activities and their outputs, wasting allocated funds.

Impact-cascades

As can be observed, the environmental changes (natural or financial) usually impact one transform and then there is a cascade effect as the change in perfomance of that transform affects downstream transform performance in terms of timing as well as expected output in quantitative or qualitative terms.

There are therefore two specific considerations in relation to reducing risks to a project with respect to changes. The first is to build into decision analysis models a way to include potential environmental and ecosystem changes and to ensure that the transforms most directly exposed to these changes are identified. It is also necessary to trace the impacts of changes in these transforms on downstream transforms.
Cascade impact analysis

In the diagram below a section of a project shows three transforms. Transform 1 can be affected by changes, external to the project, in the environment and ecosystem. Therefore the normal distribution of relevant envirionmental properties is show in the small green box and a movement from the expected position A to an actual position B leads to a change in the "New inputs" in terms of factor properties (quantitative and qualitative measures). This can alter the internal inputs to that transform process with likely impacts on the output of that transform.

Similarly, Transform 2 might be sucseptible to some aspect of administrative processes which for some reason have been changed from an expected value of C to an actual value D (see grey box) at the time of implementation of that transform. The impact cascade is the same with the external alteration in an administrative procedures being transslated into an internal change, impacting some apect of the transform process.



Therefore besides the complexity of the design process to identify a project baseline consisting of a sequence of transform processes, inputs and outputs the decision analysis needs to be extended to manage the dynamic analysis of the more significant impact-cascades that can occur during implementation. This analysis can be undertaken conveniently making use of such operations research techniques as Monte Carlo Simulation which is a robust and proven method to end up with clear analyses to rank the degrees of risk of each transform to each type of potential change and to establish a means of ranking the overall risk to a project by type.

In the box on the left, which contains a short explanation, the small green and grey graphs represent "stochastic" inputs in that the particular property of the input that affects the project transform can be represented by a range of values and a specific distribution curve that has been determined by statistical analysis of data. By using feasible ranges (usually based on past observations and analysis of records) and likely or known distributions, the process model can be used to estimate the likely impacts of change. This is how Monte Carlo Simulation works. The examples reviewed only relate to two basic types of constraints and changes in constraints but the same project model can be used to setup any number of stochastic inputs and the Monte Carlo Simulation will generate the corresponsing outputs. There are around 100 basic properties related to possible constraints and all of these can be represented by stochastic input values. In this way Monte Carlo Simulation can be used to generate comprehensive calculation of the expected outputs at individual transform level, of potential changes in conditions during implementation.

The significance of this analytical capability

The output of MCS is a series of input and output distributions and by taking single input and output values it is possible to identify a single project option. Each option is a logic feasible association of output values with input values. In the Navatec System these instances or scenarios are referred to as Logical Process Options or Logical Project Options (LPOs).

Recording design options

Simulation provides a basis for assessing the potential impacts of changes in conditions on project performance. A project team can explore a wide range of conditions that can occur during implementation without committing any funds. This process provides teams with a deeper understanding and insight of the actual vulnerabilities of any project design and gain a deeper appreciation of which design options are likely to be more resilient to the likely changes.

This information is important in providing guidance to subsequent decisions required during implementation if and when conditions change. An optimized design consists of a combination of the best operational designs for each transform. Each combination of inputs, outputs and a sub-processes is represented by a Logical Process Option (LPO). LPOs are recorded in an Accumulog1. This approach to project design does not presume that there will be a single fixed plan since this is usually unrealistic. It is assumed that any project plan selected from the design options will be subject to changes in the future as conditions change. If and when these changes occur the Accumulog contents can help guide analysis and decision making. If no equivalent circumstances to those taking place are recorded in the Accumulog, then inputting the then current data can generate the appropriate decision-support within a few seconds.

In order to coordinate oversight and implementation decision-making, Navatec System makes use of a real time audit (RTA) system3 which monitors changes specific to each project in a portfolio. This provides oversight to help ensure a prioritisation of changes in terms of their potential impacts and a practical schedule of the required decisions. This combination serves to provide a tactical support to sustain a project's momentum and to complete the cycle having maximised the likelihood of achieving the original objectives.

Operational coherence



The single Log Frame soon loses coherence with changing circumstances
The Navatec System operational structure therefore provides a due diligence-based design and it can provide logic support to project managers to respond to changes in conditions, almost in real time. This helps lower the likelihood of project failure and the invariable questioning concerning a lack of due diligence or recriminations that often accompany project performance failures.

This proactive Accumulog-based approach is distinct from the conventional approach to project planning, monitoring and evaluation based on a Log Frame which is based on a single project option. Log Frames usually carry no information of the design detail that explains why the content is the way it is. Without a full knowledge of design constraints and why the current option was selected it is difficult to provide informed advice as a result of an M&E mission. Quite often the people carrying out an M&E assignment do not possess the technical knowledge or knowhow that was necessary to design a project and to manage it. They are therefore often reduced to limiting their attention to the comparison of the indicators drawn up at project initiation with the state of indicators recorded during implementation. This has become a regular and instittionalised practice. With this information it is possible to measure "performance" but this provides no guidance on how a state of under-performance can be reversed in an optimised manner. The single Log Frame is somewhat like an initial LPO attempting to address inevitable changes in circumstances that will occur during the 1st through 2nd, 3rd implementation year or even beyond, resulting in a declining coherence between LPO assumptions and the evolving reality. Similarly, in the absense of such expert guidance on required adjustments, Log Frames can become increasingly irrelevant.

By way of contrast the Navatec System design approach, based on LPOs, does not assume that circumstances will remain the same over the life cycle and as a result all of the necessary resources are made available to facilitate LPO updates at a low cost and within a short analytical period. This has the effect of keeping LPOs and project plans completely up to date and thereby maintaining the coherence of the LPO content to changed circumstances.

The reiterative LPO simulation approach maintains coherence under changing conditions

There is no doubt that even with such analytical resources at hand it is unlikely that LPOs assumptions, that is, the quantitative data embedded in each design, can remain fully coherent with changed circumstances. Even if this were possible it is also likely that project performance expectations and realizations would not be the same as the original plan.

The LPO approach is currently the most effective means of helping managers of project portfolios and projects securing the best outcomes in accord with the original objectives. This has some fundamental advantages:
  • higher likelihood of achieving close to original objectives
  • lower likelihood of wasted resources
  • less overall risk


1  Accumulog - an Accumulog is a blockchain first identified in 1985 in an ITTTF initiative as an essential cumulative record for the support of learning and knowledge recall in learning systems. Originally prototyped in the Seel-Telesis decision support system in 1990, Navatec System is the first system to integrate this essential capability.

2  Due diligence design information - the information recorded in Accumulogs, in addition to LPOs, is gaps and needs analysis, constraints analysis, the feasibility envelope and prioritised list of feasible objectives and baseline design.

3  RTA-real time audit was devleloped to take advantae of the WWW global access to point data in projects. See RTA.Systems


APEurope.org