How should Decision Support Systems present apparently "Bad News"?
On Saturday, January 22, 2005, a special issue of DSS News was sent to all 1066 subscribers. The purpose of the special mailing was to showcase some rapidly approaching deadlines for submitting papers to conferences especially AMCIS 2005. I had been contemplating adding a conference announcement list to DSS News for some time and the project kept being pushed aside. With some conference deadlines approaching, I wanted to act to support the efforts of those who organize workshops, meetings and conferences relevant to building and understanding decision support systems. So I hurriedly created the special mailing.
Doing a task in a hurry is often a mistake and I apologize for the informal nature of the mailing, but Saturday, January 22, 2005 was busy and hurried. I actually initiated the mailing myself at Topica.COM, the web-based service that DSSResources.COM uses to maintain the DSS News email list and support sending the bi-weekly newsletters.
For almost 4 years, until September 2004, DSS News was sent using Topica's free email discussion lists. In the beginning, DSS News actually ran some Topica supplied ads. The free service is limited to fewer than 1000 subscribers and DSS News outgrew the service. That's good news for the DSS Community, but it meant that to grow our subscriber base we needed a new email service. After some investigation, we moved to Topica's permission-based email marketing solution. The annual cost of the service is about USD $500. Readers can take a web-based tour of the interface and find out about the main features of the hosted application at www.topica.com . Currently, DSSResources.COM revenues pay the cost for maintaining and mailing DSS News.
I was in a hurry so rather than having the DSSResources.COM Webmaster, my oldest son Alex, handle the mailing, I did it myself. Right! We were both still learning about how to use the many features of the new system. I created a Campaign which is Topica lingo for a mailing using the touted "point-and-click content entry system". I decided to try out Topica's Tracking feature and rather than sending the mailing to a more target audience I sent it to "All Subscribers". We have defined a number of audiences based upon email addresses and profile information. I also "cleaned up" the list.
At Topica "As forms are submitted, campaigns sent, messages opened, links clicked and transactions completed, valuable intelligence is being gathered, automatically enriching your Topica database. Topica provides detailed reporting to help you understand this information, so you can optimize your efforts. Daily list activity reports track the overall size of your list ... Real-time delivery reports confirm that campaigns were sent per your specifications and, more importantly, show you how recipients responded to those campaigns by detailing opens, clicks and purchases per link, per recipient, so you can see exactly who clicked what." So Topica has a data-driven decision support subsystem. Topica is primarily providing a transaction processing subsystem for subscriptions and an information dissemination subsystem. The decision support subsystem is quite rudimentary.
So what does the DSS Conferences mailing have to do with "Bad News"?
On Sunday morning, January 23, 2005, I checked the DSS Conferences Campaign Detail Performance Summary. We had 130 bounces. The spam filters are wrecking havoc on email delivery, but the "real-time" delivery reports showed another troubling number. The Subscriber Activity Reports showed that 41 subscribers were "deleted" on January 22, 2005. My initial reaction was "why did so many people unsubscribe?" Only 2 subscribers were unsubscribed on January 16 with 36 accounts disabled (mailing for DSS News vol. 6, issue 2) and 14 subscribers were deleted on January 2, 2005 (mailing for DSS News vol. 6, issue 1). I was concerned that those 41 people "unsubscribed" because of the unexpected DSS Conferences mailing. I was also blaming myself for hurrying with the mailing and doing it myself. Bad news!
In the Data-Driven DSS tradition, I decided to drill-down into the details of who unsubscribed and see if I could figure out what had happened. This was all happening on a Sunday morning. I went into the subscriber database and viewed the list of deleted subscribers. To my dismay, I noticed that my friend Murray Turoff at New Jersey Institute of Technology had been deleted on January 22, 2005. I did some additional investigation and finally decided to send Murray an email. I had 2 email addresses for him and copied the message to both figuring that the subscribed account may have been disabled.
My email (1/23/2005 10:56AM) read: "Hi Murray-- I'm writing an Ask Dan! column titled How should DSS present 'bad news'? One example that I'm planning to use relates to the web-based data I received from Topica.com today as part of managing my mailing lists. Yesterday, I sent out an email on DSS Conferences to promotes some Calls for Papers, ISCRAM 2005, ISDSS 2005, etc. and when I checked my stats this morning 41 addresses had been unsubscribed from the DSS News mailing list. Some had been removed because the address was bouncing and others through voluntary removal. One of the addresses on the unsubscribe list was yours...Any help or comments you can give me about the unsubscribe would be appreciated. It was "bad news" to see your email address on the list."
Murray responded (1/23/2005 6:44PM) "Dan I never did anything that should have removed my email address. ... I have no idea what happened. Once in a while the NJIT computer could reject mail when the mail server goes down, but I don't recall that happening. They have a spam system and it should not have rejected the DSS newsletter. I know I got one recently or in December at least." Murray's response made me feel better, but I knew I still had a problem.
After additional investigation, I determined that we were deleting subscribers after 4 hard bounces. So spam filters are causing problems for DSS News. Please let your friends know that if they haven't received DSS News in awhile that they should make sure it is "white listed" in their email spam filter.
How should Decision Support Systems present apparently "Bad News"?
Sunday, January 23, 2005 was not a good day for me. I'm emotionally involved with DSS News and DSSResources.COM and my rudimentary DSS was giving me "bad news". I felt better after receiving Murray's email, but I continued my inquiry into what we as DSS designers could and should do when a system might present negative information, also known as "Bad News". Information presentation is especially important in DSS because a decision-maker is intended as the user of the system and hence will act and react to the information. The more important the decisions that will be made using a DSS, the more attention that needs to be given to building the DSS and especially to the information displays.
I'm a DSS generalist and not a specialist in the intricacies of information display, and even a quick foray into this narrow, specialized topic shows how much knowledge needs to be "pulled together" to build "mission critical" DSS and to understand factors impacting the outcomes of using DSS. My experience demonstrated that 24-7 web-based operations can cause additional problems when apparent negative information is received by decision makers on evenings/weekends. Then the stress associated with decision making may be harder to resolve. I experienced a common problem with drill-down -- atomic facts without a context can be disconcerting. Also, the drill-down to who unsubscribed had inadequate detail. I could see the date when a person was unsubscribed, but I couldn't tell if it was the result of a rule, i.e., more than 4 hard bounces, manual unsubscribe by the administrator, or a voluntary unsubscribe. Also, it became evident that presenting an absolute number like 41 doesn't show the meaning or significance of the value. The red down arrow in the display was also disconcerting. In a DSS, it is important to create context and facilitate gathering follow-up information.
In my search for suggestions about improving DSS information displays, I turned to my library and Google. While surfing a few years ago, I had found Edward Tufte's blog (www.edwardtufte.com/tufte) and an Ask E.T. question about Executive Decision Support Systems. So I checked there. Tufte is a Professor Emeritus at Yale University, where "he taught courses in statistical evidence, information design, and interface design". Tufte was asked to share guiding principals or "best practices" in the presentation of Key Performance Indicators to the senior executives of a corporation.
Let me paraphrase and summarize Tufte's ideas (check his site):
1. "ask first of all: What are the thinking tasks that the displays are supposed to help with?"
2. build systematic checks of data quality into the display and analysis system.
3. good management information systems are boring.
4. "use simple designs to show well-labeled information in tables and graphics; displays should often be accompanied by annotation, details from the field, and other supplements."
5. "from a display, decision-makers need to learn what the story is and whether they can believe the story."
6. "most of all, the right evidence needs to be located, measured, and displayed. And different evidence might be needed next quarter or next year."
I also reviewed some user interface design articles and books and I want to recommend David Tegarden's tutorial on "Business Information Visualization" that appeared in Communications of the Association for Information Systems (CAIS) in 1999. Perhaps I can expand on what the experts have concluded in another Ask Dan?
This experience reinforced the importance of being careful in interpreting data; when data is presented in a negative way it is easy to focus on the worst interpretation of the data rather than upon multiple interpretations. Also, good information display is important in building DSS whether the information will be interpreted as negative or positive. A green up arrow can cause as many, if not more, problems as a red down arrow. Many of us are actually complacent and accepting of "Good News", even if the facts don't support that initial conclusion.
Decision makers need all of the information that is realistic to provide, intentionally or unintentionally delaying, ignoring or distorting some data because it might be "bad" news is always a mistake.
In retrospect, I am strongly reminded that who gets decision support information is important and it is especially important to help decision makers interpret the information. In general, it is impossible to completely remove the ambiguity and provide the complete decision context in a DSS (but we should try!). This information constraint is why I advocate building DSS and keeping decision makers in the loop rather than using decision automation in semi-structured, ambiguous decision situations.
This experience also reminded me that sometimes apparent "Bad News" can turn into "Good News" and vice versa. In this case, the "Bad News" resulted in this column. Please keep reading DSS News and tell your friends. Let's continue to build a strong DSS community.
References
Teagarden, D., "Business Information Visualization", Communications of the Association for Information Systems, Volume 1, Article 4, January, 1999 at URL cais.isworld.org/articles/1-4/default.asp?View=html&x=68&y=10 .
Tufte, E. "Executive Decision Support Systems", Ask E.T., at URL www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=00002r&topic_id=1 .
The above response is from Power, D., How should Decision Support Systems present apparently "Bad News"? DSS News, Vol. 6, No. 4, January 30, 2005.
Last update: 2005-08-16 21:52
Author: Daniel Power
Print this record
Show this as PDF file
You cannot comment on this entry
