ADMN641 Information Systems Design and Integration
Lesson 6 - Systems Implementation
"If you do what you always have done, you will be where you always have been." Anonymous 
Some Groundwork

Most of us are products of an educational system which rewards "correct" and "concrete" answers. Do you remember when you were very young and your teacher asked you the question, "How much is 1 + 1?" When you quickly answered, "2." you were rewarded with praise. Later in life we find that the most significant questions in life are significantly more complex, and often there are no "correct" answers. Life is really more about nebulous essay questions. The reason we need to understand this fact is that this lesson moves beyond the "simple" technical issues surrounding Systems Implementation such as hooking up wires and installing new software.

The lesson attempts to look at the other, more fuzzy  issues involved with keeping the implementation of  IS in tow with the expectations of  management and users. And there are rarely "correct" answers to accomplishing this task.  The people in the IS functional area have never done very well in understanding how to properly bring their systems "to the floor." The reason is quite simple. Personnel in IS are too busy just attempting to keep up with their current workload to worry much about the effectiveness of  systems or measuring how well they produce. They are almost forced into a mode of "drop them off and run".

The work of the IT function is constantly changing, taking on new form. Along with this most organizations have seen their IT budgets grow substantially. It is not unusual in service organizations for IT to be 20% or more of the total budget. With this type of outlay and with very little measurement of  the effectiveness of that outlay, management has a right to be concerned. Few would argue that there is no need for IT. At the same time though what methods exist for demonstrating the value of IT to the organization? In response to this outcry, there is a movement to establish "performance measures."  One could argue that performance measures are perhaps more of a "maintenance" topic. But to be effective these measures should be established during the creation and fielding of the IS system, not after it has been in place for some time.

IT Performance Measurement and Metrics

Ultimately performance metrics must be tied to business value, whether in IT or any other area of the business such as marketing or finance. While an easy statement to make, it is a much more difficult task to implement. How do we determine whether hiring another person for the help desk provides value to the organization? Should value just be measured in bottom line $ profits? If so, how could you possibly tie one to the other? The causal diagram below is an indicator of how this might be done. Even if we did see this type of activity occur, how would we know they were related? How would we know whether the increased number of help desk personnel resulted in decreased customer phone waiting time? Perhaps waiting time decreased because customers hung up out of frustration. Ultimately proving direct causality here would require a carefully controlled program of measurement (counting seconds on the phone, compiling customer surveys, etc.) and statistical testing of that data. Even performance measurement has its cost.

Causal Loop of Effects of Hiring on Bottom Line Profits
If you are thinking about implementing performance measures, a good place to start is with the U.S. General Accounting Office Document Measuring Performance and Demonstrating Results of Information Technology Investments. Another excellent resource is Performance Based Management: eight steps to develop and use information technology performance majors effectively. That document is available from the General Services Administration. Another useful source, at least in terms of an overall process with proven benefits can be found at Formula for ROI .

The Gartner Group recommends the following seven key metrics for chief information officers.  They are:
Area specific measure
applications development dollars per function
data center dollars per MIPS
central servers dollars per combined power rating
distributed computing dollar per user
IT help desk dollars per call
wide area data dollars per device
voice network dollars per minute and dollars per extension
While these measures are very concise, they may be a little short in terms of scoping an overall measurement plan. And while we typically feel more secure with "concrete", readily measurable issues such as capacity or speed, often other much less tangible measures such as morale, and long term learning prove more valuable indicators. As such macro level methods for controlling organizational functioning such as the Balanced Scorecard can be much more effective.

Itemizing Metrics
By reviewing the MIS issues and metrics below, can you fill in the space to the right where a specific metric is missing? These issues and metrics were developed by previous groups studying this metrics area. Can you think of other important measures which may be absent from this list?
The Organization

 Issue  Metric/ Performance Measure 
Issue  Metric/ Performance Measure
Experience/  Background  Number of Years in Field 
Training  Course(s) Completed, Test Score 
Motivation/Attitude Ordinal Scale 
Quantity  Number of Personnel 
Effort  Avg Hours worked per week
 Hardware / Networks
Issue  Metric/ Performance Measure
Capacity    MB per Second, GB 
Compatibility / Integration 
Technology Evolution   Generation / Sub generation Change 
Justification / Acquisition   
Issue  Metric/ Performance Measure 
Issue  Metric/ Performance Measure 
In addition to these metrics, see other example detail metrics by following this METRICS LIST link. It is obvious that many of these metrics are detail in nature and typically departmentally specific. Ultimately an organization should utilize its own processes and perhaps some group decision support tools such as GroupSystems or TeamEC to help guide them into developing these measures.
Executive Guide: Measuring Performance and Demonstrating Results of Information Technology Investments. GAO/AIMD-98-89. March 1998. [Available only in PDF version at]
Shein, Esther. Formula for ROI: IT is gauging project performance to produce tangible results for business.  PC Week Online
September 28, 1998.

Assignment (due by midnight, Saturday, 03/04/2000):

1. Examine the issues and measures posted in this reading. Can you think of some measures where none are listed by the issues. Can you think of some issues or major topical areas which should be included? Post in the IT Performance Measure Conference some additional issues and measures.
2. Mini Paper 1 is due.

(c) John H. Saunders 1999. Permission granted for use in courses at the University of Maryland