What evaluation tools are appropriate for DSS?
by Daniel Power
Editor, DSSResources.com
Managers evaluate many types of projects, but those involving information
technologies are often considered challenging. Evaluating DSS projects is
especially difficult and yet it must be an ongoing process for large-scale DSS
projects. Evaluation activities should be commensurate or proportionate to the
scope, complexity, and cost of a proposed DSS project. Project scope refers to
the number of potential users, the size of the project staff, the potential impacts
on existing systems, and the amount of programming or development effort that
will be required. Also, project sponsors and project managers must decide what
amount and type of evaluation is appropriate and necessary in their company's
IT management environment. A very bureaucratic or political IT environment may
necessitate additional evaluation activities.
The scope, complexity, and cost of a decision support project and the development approach should determine what evaluation activities will actually occur and in what sequence. Evaluation should determine the worth or merit of a contemplated, in progress, or completed project. Evaluation should be performed periodically from the initial idea stage to the periodic post-implementation project evaluations. A DSS project can be revised or even canceled at any stage. The resources that have been expended on a project are “sunk” costs.
Like other projects, the anticipated results and benefits of a decision support project should be quantified so that the requested expenditure can be
evaluated in comparable units. But it is difficult to quantify the results and
benefits of a DSS Project. DSS analysts are basically making estimates and
guesses.
A number of alternative tools are available for evaluating decision project projects prior to, during and after completion. The following five tools are especially useful:
Incremental value analysis assesses “soft” benefits, such as improving staff productivity, improving the speed of strategic actions, enhancing a company’s competitive advantage, or improving access to data.
The scoring approach considers intangible benefits and other considerations that are not considered credible by analysts who only focus on financial criteria. The process involves weighting factors to reflect how well the project satisfies a given factor. Points are assigned to each impact criterion. The scores are summarized; projects are ranked.
The qualitative benefits scenario approach attempts to estimate what decision making will be like when a proposed DSS is in place and hence speculates on how the company will benefit.
Cost-effectiveness analysis is a simplified CBA where one assumes that
all of the alternatives have either the same benefits or the same costs. The
analysis is simplified because only benefits or costs need to be calculated, not
both. In this analysis, the best alternative is the one with the greatest benefits or the lowest cost.
Research and Development Options approach considers the expenditures for both incremental DSS development and for maintaining flexibility to build a future DSS. Managers the evaluate whether the total value of the options outweighs any shortfall in value from the expenditures.
Once a DSS project is completed, managers need to follow up and periodically evaluate what is working well with the system and why, as well as assess problems that are being encountered.
This column is edited and updated from Power (2002) pp. 198-205.
References
Power, D. J., Decision Support Systems: Concepts and Resources for Managers, Westport, CT: Greenwood/Quorum Books, 2002. pp. 34-35.
Last update: 2020-12-06 03:42
Author: Daniel Power
Print this record
Show this as PDF file
You cannot comment on this entry
