e-Talk Radio: Pressman, Roger, 14 November 2000

TEXT TRANSCRIPT: 14 November 2000
Copyright 2001 Quality Plus Technologies and Carol Dekkers. All rights reserved.

Announcer: Welcome to Quality Plus e-Talk! with Carol Dekkers. This program will focus on the latest in the field of technology. All comments, views, and opinions are those of the host, guests, and callers. Now let's join Carol Dekkers.

Carol: Hi. I'm Carol Dekkers. I'm the host of Quality Plus e-Talk! And I'd like to welcome you to this week's show. I'm the president of Quality Plus Technologies, which is an acknowledged industry leader in teaching software companies how to build better software. And we've talked for a number of weeks and had a number of different guests, and I'd like to welcome to the show this week a resident of Palm Beach County, which is absolutely timely. We didn't plan it this way, but it's wonderful. He is the author of six books, including Software Engineering: A Practitioner's Approach, which is the most widely used software engineering manual, encyclopedia, book that's ever been published. He's an internationally recognized consultant and author in software engineering. Over three decades, Roger Pressman has been a software engineering manager, a professor, an author, a consultant, focusing particularly in the software engineering discipline. And he's president of RSPA, which is Roger Pressman Associates, which does a lot of consulting to establish effective software engineering practices in companies. He's also the developer of a product called Process Advisor, which is the first industry self-directed software process improvement product. And he's also the star and I believe producer of the software engineering video set that's in use at over 1,000 companies. So I'd like to welcome you to the show, Roger.

Roger Pressman: Thank you, Carol.

Carol: And we had talked a lot about some of the different things that we could talk about this evening, in terms of where software engineering is going, what are the trends, that type of thing. And then this election came up, and Roger happens to live in Palm Beach County. How has it been lately?

Roger: Well, it's actually been kind of comical. I've received well over 100 emails in the last two days from friends and colleagues all over the world, as far away as Pakistan and India, who have sent me the crazy cartoon that I'm sure everybody's seen with the connect-the-dots cartoon, where there's a straight line between the dot and Bush and very curved lines between the dots and everyone else. Everyone finds that very amusing. Everybody finds South Florida right now very amusing. It actually isn't very funny at all. It's a serious system problem, and one that I'm hoping is going to be remedied over time.

Carol: And you may even be involved in helping.

Roger: Well, I'm not sure that that's going to occur. I try to stay as far away from politics as I possibly can. But there's no question that there was a significant problem down here. I think in many ways it's being misrepresented by the media, but no question, we have a systems problem. We don't really have a software problem, because the systems down here are so archaic that software doesn't even come into play. But things better be changed, or else we could potentially have this kind of a problem again.

Carol: Right. Now, what's been interesting to me is we've had a couple of shows. We've had Howard Rubin on the show, where he's talked about cybergeography, where third world countries are getting very, very involved in software.

Roger: Sure.

Carol: And I think it's fascinating to me that this absolutely illustrates that we've got software everywhere in our lives, but not absolutely everywhere. It's not consistent, and in some places, in this election in particular, they had some very, very advanced voting systems.

Roger: Well, they had very advanced voting systems in some places. Oregon is an example where at least they had absentee ballots submitted as the only ballot you could submit. But as we can see, even in the Oregon system, there's been a very lengthy delay in getting all the votes counted. The vast majority of voting systems in this country are very old. As an example, I happened to notice this today in a major newspaper in South Florida, a very good newspaper, I might add, the Sun-Sentinel, they noted that across the country, 20% of all votes are cast using lever-based voting machines, mechanical machines that were invented about 100 years ago. And they're very error prone, and not really very reliable, and almost 30% of all votes cast in this country are cast using the punchcard system that we used in Palm Beach County, much to our dismay. The problem isn't the punching. The problem was an ergonomic problem, which we might want to talk about later, because ergonomics is a critically important issue in the design of anything, including human interfaces. Certainly, a ballot is a human interface. And this one was designed so poorly that actually when I saw it in the voting booth I laughed. Unfortunately, it turned out it wasn't a laughing matter. As we can see by the huge number of errors and double punches and all other kinds of nonsense that occurred. So we really are very archaic in this country in the way we vote. The problem, it seems to me, is a political one. It's also based on cost and timeliness and availability and a lot of other issues, which us techies can't really address. But certainly we can't continue with the system we have, or we're liable to have the problem again, as I said earlier.

Carol: Right. And I think that's interesting that you went in and you looked at this ballot. The ballot was redesigned in 1996, was it not? Or is the same ballot that was used, same style of ballot that was used…

Roger: Well, the voting mechanism, the punchcard mechanism, has been used down here for many, many, many years. The ballot was redesigned for the year 2000.

Carol: Oh, okay.

Roger: And the reason it was redesigned was, as has been reported correctly in the media, at least my understanding from the media is that it was redesigned to make it more legible for senior citizens who need larger type. And that's understandable. And secondarily, because there were nine different candidates on the ballot, not just two or three or four, but nine different candidates on the ballot, we needed what has now become known as the infamous butterfly ballot. The problem is that when people read a list, whether they're reading on the Internet or reading a computer-based report, or reading a ballot, they're used to reading the list top-down and left to right. This particular ballot, the butterfly ballot, was not set up that way. And it obviously confused a lot of people.

Carol: And what's interesting to me is one of the things that I didn't realize about Palm Beach County is that apparently in the 1996 election, there were 15,000 double punched…

Roger: Again, there's an awful lot of poor reporting, bad reporting. And one thing that I am is a numbers person. So I've spent a little bit of time looking into numbers, just for my own edification. It turns out that isn't correct. There were approximately 6,000, what might be considered to be double-punched ballots. There were another 7,000 that were not punched at all, either people forgot to punch them or whatever. Six thousand is the key number. And we compare that this year to 19,000, that's a very big difference.

Carol: Oh, and you see a significant difference in that.

Roger: Yes. I mean I don't want to sound like I'm advocating for anyone, because believe me, I'm not really very political. But the numbers are reasonably clear here. And when you think about them, it's really mindboggling. I mean, Florida had 6 million votes cast. The current differential according to CNN a few moments ago was 280 or 300 votes. That's 5/1000 of one percent. And thinking of this as a system, understand clearly that what we're talking about is five errors per 100,000 votes cast. That's a very small margin of error in any machine that counts votes or any human that counts votes. So what we've got here, I believe anyway, is that we are literally within the margin of error of any existing counting mechanism. And what that means, in my opinion, is that any time we do a count we're going to come up with different numbers.

Carol: Right.

Roger: Because we're within the margin of error. Clearly within it.

Carol: And you can never get 100%. You could approach 100% accuracy…

Roger: Not with the systems we have currently. I think we could begin to get very close to 100% accuracy, but not with the existing systems.

Carol: And there's a cost that goes with it.

Roger: Exactly. Somebody said the other day that if New York State, just New York State, were to implement a computer-based solution to polling, to voting, I'm sorry, at the polls, it would cost them $100 million. That's a significant sum of money. And when you extrapolate out to a nation-wide system, it would certainly cost well over a billion or a few billion dollars. Now, it would be interesting to know whether people thought it was worth it. And then there a lot of other issues as well. Security is a critical issue. As well as some others. But cost is nontrivial for a good system. The question is how much are we, as American citizens, willing to pay to get the kind of a voting system that wouldn't let us down like this one has.

Carol: And that's an interesting thing, because you'd really only be using it necessarily, once every two years, once every year, for some of the…

Roger: Well, that's exactly right. But then again, you can argue that the citizens are completely willing to spend a couple of hundred million dollars for a sports stadium that's only used every Sunday.

Carol: Right.

Roger: And I think there are a lot of people who are listening who would probably say it's much, the money is much better spent on the stadium for the Cardinals or whatever, than it would be spent on politicians. And I couldn't really argue that point. But the reality of it is, we are willing to expend large sums of taxpayer dollars for systems or things that are used intermittently. And the question is whether those dollars would be well spent on a voting system that undoubtedly is used only intermittently.

Carol: And we'll be back with more of Roger Pressman after these short messages.

Welcome back to Quality Plus e-Talk! My guest this week is Roger Pressman, who's an internationally recognized consultant and author in software engineering. And coincidentally, he happens to reside in Palm Beach County. We've been talking a little bit about some of the systems that are in place to do the election, to do the election results. And Palm Beach County has come up as, I wouldn't say a shining star, but definitely a shining microscope, I guess you could say. In the listening audience, particularly in Arizona, there's a mixture of optical scanning. No, there is no optical scanning, a very small, I think there's one county that has optical scanning in Arizona. But it's interesting that USA Today says that 18% of the entire country used punchcards. And Arizona predominantly used punchcards and paper, which is similar to what was used in Palm Beach County.

Roger: Well, I think one of the things that's happening, Carol, is there are a lot of people who use a punchcard system like we do down in South Florida. And a lot of the people in other states are making jokes and being very smug about the fact that well, "We didn't have any problems, and our votes got counted." And to them I say your vote was not 5/1000 of 1%. If it was, you would, I think, recognize that your system is as error prone and has as many problems as ours does in Florida. Now, our problems were compounded in Palm Beach County specifically, as the listeners I'm sure are aware, I don't see how you couldn't be aware of this, of a poorly designed ballot, which created some problems on top of the systemic problems associated with the kind of systems that we have in place for collecting data from the polling booths. But all of us I think experienced these problems. It's just that we are generally blissfully unaware of them, because the votes are never close enough to have them come out. In Florida, this time for the first time in history, the vote is so incredibly close that we're seeing these problems. And I think, I certainly wouldn't call it the shining example of anything this week. I think we're an example of a global system, or at least a national system that is seriously flawed. And we have to use the example that Florida is kind of exhibiting to try to do something about it so that if we do have another close election, maybe in another state, maybe in Arizona, where you guys are, that we won't run into these problems again.

Carol: Right. Now, you mentioned something about ergonomics. Maybe you can explain a little bit what you mean by the ergonomics that was involved.

Roger: Well, first let me digress for a moment. When we talk about ergonomics in a software or systems sense, which is what my field is, what we talk about is human factors design. We try to design human interfaces of any type so that they are easily understood, so that they require very little memorization, so that they are not error prone, so that they have kind of a useful metaphor that allows the user to interact in a natural and intuitive way. That's what ergonomics is about. Normally, today in the software engineering field, we talk about the ergonomic design of let's say an Internet-based e-commerce system, or the ergonomic design of some client-server application. In the Florida, Palm Beach County vote, and more importantly, in the national Presidential election, we here in Palm Beach County are at the epicenter of poor ergonomics. The infamous butterfly ballot was about as bad an ergonomic design as I have ever seen. It was designed by politicians, who were clueless, quite frankly. It was not done with malevolence, I honestly believe that. I think the best intentions were at hand. People tried to do a good job. But they simply screwed up, if you'll excuse the word. And it's created a firestorm down here. Had somebody with a little bit of training looked at that ballot, it would have taken them all of three seconds to recognize that it had the potential to be problematic. But apparently no one did, or if they did, they kept their mouths shut. And as a result, at least in part, we have the mess that we're looking at right now.

Carol: And in a lot of automated systems, you have this user acceptance testing. You have people test drive it. And I don't believe that necessarily happened in Palm Beach County.

Roger: Well, what happened in Palm Beach County, and the media has made much of this…What happened in Palm Beach County is, as is required by Florida law, the ballot was published in the newspaper, but comically, the most important part of the ballot was not published, and that is the holes and their placement. Because that's what created enormous confusion for many people. The problem was simply that no one had ever…The equivalency here for the those software types in the listening audience would be buttons that we use in any interactive interface. It would be very difficult to assess the efficacy of buttons if the placement wasn't shown.

Carol: And we've got to go into break, and we'll be back with more of Roger Pressman.

Welcome to Quality Plus e-Talk! If you're just joining us, you may be familiar with Quality Plus Technologies. We teach people how to develop better software that works and meets customer expectations. My guest this week is internationally recognized consultant and author in software engineering Roger Pressman, who happens to also reside in Palm Beach County, which has received a lot of attention this week. And last week. For anyone that's interested, the guest phone-in number is…a toll-free number…866-277-5369. And we've been talking about the voting systems, and a software issue, ergonomic issue, and Roger's been explaining to us some of the things that are going on in Palm Beach County. Can you tell me a little bit…I think people are probably wondering, Roger, whether demographics played a huge part, whether it's the elderly that…You know, the media has been having a heyday with. Is that really the problem?

Roger: Well, I don't believe it is. I think unlike the media depiction of Palm Beach County as a county that is in its entirety just one retirement community after another, the listening audience should recognize that the retirement community within our county is large, unquestionably, larger than average, but it still only represents about 20% of the population. The remaining 80% of the population is just like any of us, and that is working folks, young people, students, whatever. And given that, people should not have the impression that everybody who walked in a voting booth was a senior citizen. And frankly, in defense of seniors, many of them are completely capable, most of them, all of them, pretty much, are completely capable of voting. I think the problem had much more to do, as I mentioned in the last segment, with a very poor ergonomic design of the ballot. And very poor vote counting systems. Those two things, I think, have presented the problems that we have here. Not so much the demographics within the county.

Carol: Right. USA Today had a feature article, where they showed that certain areas of the country have optical scanning, very modern systems. And I find it interesting that Alaska has entirely optical scanners.

Roger: Well, I think it's fairly easy to explain why that's happened. Number one, Alaska was one of the last states to join the Union. Given that, they're young. Given that, their voting system and their voting process represents the more modern era. If you look at states in New England or Florida or the South in general, we're much older. Our voting process and the machinery which allows us to effect the vote is much older. And a lot of us are using voting systems that have been around for 40, 50, 60, 70, 80 years. The problem is that the demand for instantaneous data today, the problem is the demand for easier data collection far overcomes the ability of the voting systems to respond. In addition, I again want to emphasize this, none of this would be relevant in any way if it wasn't for the closeness of this election. The errors that we're exhibiting in South Florida, I'm sure are exhibited in every other state in the union that has older systems. The difference is, in South Florida, the difference between the two candidates is 5/1000 of 1%.

Carol: It's like a meteor hitting, or something that you could never predict to actually happen to be this close.

Roger: Yeah, it's pretty amazing. Statistically, we've seen some incredible aberrations. A professor at Carnegie Mellon and another professor at Stanford did an analysis of the Pat Buchanan vote, which is a real argument for the ergonomic problems associated with our ballot in Palm Beach County. And the professor I believe from Stanford came up with a statistical analysis that indicated that the likelihood of Pat Buchanan drawing 3,400 votes in Palm Beach County, when compared to the population and the voting patterns in every other Florida county, was about 1 trillion to one.

Carol: Wow.

Roger: And the implication of that is, and this is an independent observer, the implication of that is that there were errors made, and then we have to then say, were the errors due to stupidity? And I discount that. I honestly do. Or were the errors due to a very poor design? And I think that that's exactly what happened. This is not a political discussion, so I'm not going to even begin to suggest the potential remedy or whether there even should be a remedy. That's not my area. But there's no question that there was a very poorly designed ballot. And that reflected in people who voted for one person when they meant to vote for another. Clearly, there is no question about that. In this case, the statistics are pretty clear.

Carol: And none of the other demographics, none of the other statistics in Palm Beach County stand out. You don't have a higher occurrence of people running…you know, driving into restaurants. Or any of the other things that you would associate if this were actually an aberration in the demographics.

Roger: No, no. I honestly don't think that's the problem. I think the seniors have been probably unfairly maligned by the media. I saw a piece on MSNBC the other night which really was kind of aggravating. The entire piece was on Palm Beach County, and it was set in a retirement community, with a bunch of seniors doing line dances. And I have to tell you that, because I live here, that Palm Beach County has major industrial parks, it has all kinds of shopping malls and young people's facilities, and sports stadiums, et cetera. Why they chose to do this piece with a backdrop of seniors doing a line dance escapes me, unless they were trying to mold opinion in a specific way. And this was MSNBC, this wasn't a hack media outlet. Maybe some people might consider them a hack media outlet, but I don't know. But the reality of it is, the media can shape perceptions in a dramatic way, and all of us have to remember that we have to gather the facts and not just listen to what they tell us. Because in many cases, the facts are far different than what they tell us.

Carol: Right. It was interesting to me living in Florida to hear the polls, because on a day by day basis, I don't know what it was like anywhere else in the country, but it would be, you know, 40 to 49 to 49 to 48, and the undecided voter that I've read about for so many years wants to go with the winner. And it's almost impossible to know who the winner's going to be so that you can vote for the winner, unless you know ahead of time. And I think that really illustrates the closeness of the whole thing. Moving beyond the election, what are the solutions? What are some ideas that we as a nation could embrace?

Roger: Well, the solutions are difficult, because the solutions have to be political. And the minute you move from a technology solution to a political solution, all the rules change. I can only say that first of all, we have to understand what we need. Secondly, it would seem to me, we have to understand what the key issues are. And thirdly, we have to consider what the alternatives are. We need, and I would think most people would agree with me, easier, less error-prone modes of input for the election. I'm going to use software jargon here, but that's really what I'm saying. We need an easier mode of input, less error prone. We need better collection mechanisms and better communication of the collection that we make. It would seem to me we need a much higher degree of accuracy, which therefore means a much lower error rate. We need a high degree of security at the polling place, if in fact we have a polling place. And I believe, this is kind of a personal thing, I believe we need more time to execute the choice. And by that, I mean that although we've been sending people to polling booths for 200-odd years, it may be time in the 21st century to change that and say maybe we should have a week to conduct the election. And during that week, people can make a calm, informed decision. And then submit their vote in some way. One of the problems that we had in South Florida is the lines were so enormously long, polling places were so crowded, there was a subtle pressure on all of us to rush through the vote.

Carol: Hold that thought. We'll be back right after these messages.

Welcome back. I'm Carol Dekkers. Our guest this week has been Roger Pressman, who is a renowned author in the area of software engineering, and consultant, professor, I think you've pretty much done everything, Roger.

Roger: Well, I haven't done everything, Carol, but I've tried to do enough so it makes life interesting.

Carol: Right. We went into break, and you were talking a little bit about some of the problems, I think.

Roger: Well, yeah, we were talking a little bit about the vote, and we were also talking a little about some of the potential solutions. And what I was trying to do was to first indicate what the requirements are. Whenever we look at software systems, you have to begin by identifying requirements. The problem, of course, is that there would be a vast array of requirements. And most of them would be political. It's very hard to mold those into technology solutions sometimes. But just to summarize what I said, the requirements would be ease of input, would be high accuracy, would be high degree of security, would be better collection mechanisms, better communication of the collected results, and a personal requirement that I think is important is more time to execute a choice. The issues involved in building a better voting system for the United States of America - number one would be cost. Are we willing to spend probably the billions of dollars that would be required for something that is so intermittent.

Carol: And right now it's a county-specific system, is it not?

Roger: Right. But I think what's going to be happening, what we're going to see happening politically is there's going to be a call, I suspect, and again, I'm not a political talking head, I don't know, but it would seem to me that some of the Washington politicians are now going to begin saying we need a national system. We need a consistent national system for collecting, inputting, collating, organizing and reporting the vote. That wouldn't surprise me in the least.

Carol: And that's a massive endeavor. When you look at the different types, there's the optical scan, there's the punchcards, there's the lever machines, the paper, electronic, a mixed hole thing, a data vote, then we've got our absentee…When I look at systems that are statewide, and then you go and you transpose that into another state. If you have a system that works in Arizona, California never can take it just off the shelf.

Roger: Well, and that's why I think if this happens, there's going to be legislation that will force California to take the vanilla system off the shelf. Like I said, this is a political issue. And because it's a political issue, there will be enormous wrangling over it. And probably because there'll be enormous wrangling over it, as often happens in the USA, nothing gets done. And as long as elections aren't close, there's no problem. We can live with the system we have for another 100 years. But if we have another election in some state like we're having in Florida, then all of a sudden all the warts and flaws of the system come out, and we have everybody looking bad. And that's the question we have to ask ourselves. Are we willing…This election, to use a meteorological metaphor, is sort of like a hundred-year storm. They talk about design for a one-hundred-year storm, assuming that the flood levels will never come up except once every 100 years. Well, it's sort of the same thing. This election is a hundred-year election, in essence. There's never been an election this close, to my knowledge. Certainly, the Kennedy-Nixon was not even nearly this close. So I mean the question is, do we design for a hundred-year election, or don't we? And that's a decision that's going to have to be made by politicians. The actual implementation of a voting system, by the way, Carol, in my opinion, is not that awe-inspiring. I think it could be done relatively straight-forwardly, at cost certainly, but not…It is not as complex, in my opinion, for example, as designing a state-of-the-art air traffic control system.

Carol: Right.

Roger: And there's a system where you really, truly do want zero errors. Or as damn close to zero errors as you can get.

Carol: No kidding.

Roger: Yeah, all of us who fly kind of believe that very strongly. Were we to really understand the internal workings of the existing air traffic control system, I think a lot of us million-mile-a-year flyers wouldn't be doing quite so much flying. But we do anyway. The point is, I don't think it's that difficult to implement a nationwide, high-quality, high-accuracy voting system from a technological point of view. I think it's virtually impossible to implement one from a political point of view. And the question is, can our politicians, our quote leaders end quote, overcome that impossibility to the extent that we actually do something about it. Or do we opt to shrug our shoulders, say this is a one hundred-year election, all of us will be dead in the next 100 years, let our great-great-great-grandchildren worry about it. And that may be the end result of all this. Who knows?

Carol: And one of the things we talked about in the break was that Florida doesn't have a state personal tax. And I said do you think that makes a difference, and I was taken aback a little bit by your answer, which was that we're willing to spend millions and millions of dollars for stadiums that are only used on Sundays.

Roger: Well sure, but I would say that in Palm Beach County, there probably are a lot more Dolphins fans than there are either Gore or Bush fans. So the reality is that that's probably money well spent. At least from the voters' point of view. Although there are many voters who question that expenditure. I'm one of them, by the way. The reality is that it's really a question of what tolerance we have for pain. This, over this week, we're exhibiting a relatively painful process of seeing a system that is flawed. But it's going to go away, ultimately a count will come in, ultimately one of the two parties will concede, and we'll move on. And maybe that'll be the end of it. But I suspect we're going to see more of this before it's all over. And I just hope that some technologists, some good systems people, are integrated into the process someplace. What worries me is we're going to have a system designed by politicians, for politicians, and it's going to be a mess, and it's probably going to cost five times what it should.

Carol: And we'll be back with a close-up with Roger Pressman.

Welcome back to Quality Plus e-Talk! We're just about out of time, but we've been spending this week talking to Roger Pressman, who's an internationally recognized consultant and author in the area of software engineering. He's the President of Roger Pressman Associates, RSPA, and he is also a resident of Palm Beach County. I'd like to thank you very much, Roger, for being my guest this week. And would you like to give out your Web site address, if anybody's interested in…?

Roger: Sure. It actually has absolutely nothing to do with politics, I promise. And it's very simple. We've been on the net for a long time, so we have one of those cherished, four-letter URLs. It's www.rspa.com.

Carol: Great. Thank you. And next week's guest, we're going to be having Ed Yourdon, who is also an author, consultant, and presenter, who has been around the country for a number of years and has been the editor of American Programmer, among other things. He wrote a book called Death March of Large Systems Projects. I know I've got the title wrong. But he'll be our guest next week. And I'd like to thank you for joining us this week. This has been wonderful. Roger, I'd like to give you the opportunity to say a few words in closing, just about Palm Beach County, about software engineering…

Roger: Well, I will say that I've done many, many interviews, and this is the only one where I've talked about national issues. Normally, I'm kind of hung up talking about software and systems in kind of an esoteric way, because that's my business. The only thing that I can say to everybody is just simply that what we're seeing here is an example of a system that's flawed and that runs and operates and no one is aware of it until a series of circumstances occurs that causes us to be very painfully aware of it. I suspect that out there in the world, in our world, in the systems and software world, there are also systems that operate and exist, and in fact everybody thinks are fine, that aren't. One of the ways that we try to build high-quality systems is to use good process, good software engineering practices, good system engineering practices. And for any of you who are working in the new Web-enabled world, who are building Web applications or e-commerce applications, recognize clearly that if good process, good Web engineering practices is used, you're much more likely to build high-quality systems that won't turn out to be like the Florida voting system that is enormously error prone, but that cranked along for many years until a set of unusual circumstances occurred, and then it came crashing to its knees. It's kind of a strained metaphor, but I would suggest to you that it's a reasonable one.

Carol: And I think that's very well said, Roger. Again, thank you very much for being my guest this week. We have a number of guests coming up, Ed Yourdon being next week, Mark Paulk of the Software Engineering Institute the week after, who will round out our overall schedule this year. And the whole topic, the things that we want to talk about, that we want to bring to the public, is the whole notion of what is software engineering versus computer science. What is going on behind the scenes in system development, in development of software? What does it take to get the requirements right? And Roger, I think that you've illustrated something for us this week, which was quite unexpected when we first scheduled this show, which is how software or lack thereof can affect our very lives. So I'd like to give a final farewell to Roger Pressman. Thank you again for joining us. If you'd like information on Quality Plus Technologies, please go to www.qualityplustech.com and as a final sendoff from Quality Plus e-Talk! this is Carol Dekkers. Have a great week.

Copyright 2001 Quality Plus Technologies and Carol Dekkers. All rights reserved.