Developing a Portfolio Approach to Capital Investment: A Case Study in Re-Engineering Resource Allocation at the U.S. Department of Veteran's Affairs
Software and methodology helped the Veteran's Administration link its funding requests to the agency's strategic objectives, resulting in dramatic time savings and greater efficiency.
By O. John Wasyluk and Daniel Saaty
In 1997, VA decision makers discovered an off-the-shelf software prioritization tool based on the Analytic Hierarchy Process (AHP) method of decision making. Through the use of AHP and decision support software, VA budget officers were able to instill discipline and consistency to their capital investment process. Over the next three years, the VA designed its organization to 1) review a variety of investment options from construction projects to information technology projects and 2) standardize the application for capital investments and develop a capital investment methodology comprised of formulation and execution reviews. Formulation review is comprised of three phases: functional development, technical review, and strategic review. The functional development phase is at the field level, where needs are realized, gap analysis completed, application proposals developed, and solutions ultimately applied. In the technical review phase, proposals receive early rounds of technical and financial scrutiny from department-wide councils or boards, as well as some initial prioritization within the owner organization. Finally, the strategic review phase includes a two step review process and prioritization by the VA Capital Investment Board (CIB). The Execution Review includes the confirmation phase and occurs once funds are appropriated and included in the president's budget. Planning assumptions for these funded proposals were made 18-24 months earlier. For example, prior to the obligation of funds, the VA Capital Investment Board (VACIB) will review the upcoming fiscal year's obligations to assure that the proposed investment is still needed and that the planning assumptions used are still valid VACIB prioritizes the projects into a single list according to how they contribute to strategic objectives.
In 1997, executives at the U.S. Department of Veterans affairs set out to meet several legislative mandates to improve their decision making process for capital in vestment Legislation such as the Government Performance Results Act (GPRA) of 1993, the Federal Acquisition Streamlining Act (FASA), OMB's Capital Planning Guide, and the Clinger-Cohen Act put pressure on government agencies to improve their ability to audit their program and project evaluations. The GPRA sets out general guidelines requiring agencies to develop strategic plans, evaluate programs based on internal and external goals, and set measurable performance improvement milestones as key components of every budget submission. Clinger-Cohen more specifically focuses on measuring performance and effectively selecting and controlling information technology investments. FASA and the OMB Capital Planning Guide provide specific guides on capital planning decision-making.
Decision makers were walked through a set of pairwise comparisons where they were asked to compare goals for their relative importance to the VA. For example, executives were asked to compare "return on investment" (ROI) to "improving customer service" to determine which contributed more to the success of the organization. Voters were then asked to discuss their positions with others. They shared what they did and did not know and their stakeholder priorities. After discussing the priorities, voters were permitted to change their votes. Subsequently, they were asked to compare ROI to "Risk" and then Risk to customer service. The software calculated the votes' geometric average for each comparison, then the averages were used to compute the AHP priorities. The AHP algorithm places the ratios of each pair of factors' relative importance into a matrix. This matrix is then multiplied by itself and the eigenvector, or priority of the goals at a given level of hierarchy, is computed. The weighted goals were then used to rate investment options for their contributions to improving performance in the VA. In addition to providing consensus, the process also had a positive impact on the group's dynamics. For the first time, senior executives left a planning meeting in consensus about what was important to the organization and how the investments would enable the VA to achieve the strategic goals.
The decision model was delivered to the project evaluation team on a laptop with radio frequency keypads to enable the team to efficiently rate each project against the objectives to which it contributed. The software allowed decision makers to use both quantitative and qualitative information to rate competing investments. For intangible objectives like improving customer service, it provided the capability to build adjectival scales such as outstanding, very good, good, and so forth. The ratings are weighted and based on the value that decision makers perceive for each rating. If numerical performance data is available, the raw data is converted to a utility score between 0 and 1, with 1 being the highest possible data point. Evaluators can then build a utility curve to determine the value of any data point in the range.
VA evaluators entered ratings using thekeypads, then were given the opportunity to discuss their scores. One of the ground rules was that ratings had to be sufficiently close to move to the next judgment. Once the ratings had been applied to the projects the software produced a final priority list justifying the incremental strategic benefits of each project.
In 1999, after implementing 20 of 28 recommendations compiled in a "best practices" survey, the VA finally integrated IT and construction into a single software model. While some projects did not address every performance goal in the hierarchy, they were scored based on which priorities they contributed to, how much they contributed, and how
The report titled, "Annual Report to Congress on Implementing the Federal Acquisition Streamlining Act," covered 15 major civilian agencies and 185 individual acquisitions. In this report, the OMB ranked the VA as having made "Excellent Progress." The OMB went on to say that the VA "is the first civilian agency to develop an agency-wide capital planning process that allows for investment trade-offs among categories of assets, such as medical and non-medical equipment and infrastructure and IT."
The VA reviews and validates its process and decision model annually in a "Lessons Learned" conference to get feedback from the field and to ensure it is current with any legislative mandates or changes to the strategic goals. In 2000, the agency further streamlined the process by instituting a Web site, a technical library, and capital investment guidebook, and developing four electronic Web-based templates to make the process more user-friendly.
TeamEC/EC 2000 2nd Edition for Groups Screen Shots
Questions for Analysis and Discussion
O. John Wasyluk and Daniel Saaty, "Developing a Portfolio Approach to Capital Investment: A Case Study in Re-Engineering Resource Allocation at the U.S. Department of Veteran's Affairs", Expert Choice, Inc., 2001, URL DSSResources.COM