FSN: SAP versus Microsoft - which ERP makes End-Users more productive?

SAP versus Microsoft - which ERP makes End-Users more productive?

20th August 2007
A new study published this month suggests that Microsoft Dynamics NAV and Microsoft Dynamic GP make End-Users more productive than SAP All-in-One and SAP R/3. Like all research of this nature the devil is in the detail. The sample size is small, not necessarily comparable between vendors or functional areas covered by End-Users within its scope. So the results need to be viewed with a certain amount of caution. Gary Simon , FSN's managing editor, finds that the study is far more important for the questions it raises about how companies should be assessing the contribution that ERP software makes to their business rather than the actual outcome of the research.

A new research study published this month measured how end users of Enterprise Resource Planning (ERP) applications describe the impact of Microsoft and SAP applications on their business productivity. This research found that Microsoft Dynamics end users on average rated their experience with Microsoft applications more favorably than SAP users rated their experience with SAP. The findings are based on a business productivity measurement framework developed by Keystone Strategy, working under the direction of Dr. Marco Iansiti, the David Sarnoff Professor of Business Administration at the Harvard Business School , and sponsored by Microsoft.

Ever since Microsoft entered the business applications software market it has challenged the received wisdom that ERP software makes businesses more productive and efficient. Their willingness to question the status quo appears well justified, for example, a study from AMR Research ( The Enterprise Planning Spending Report 2005-2006 ) found that only 15% of employees are licensed to use their company's ERP system and that 46% of licensed ERP seats go unused.

The humble user interface (UI), can have a profound impact on the economics of investing in an ERP system. A Forrester Research study, ( Put Business Applications To The Usability Test ) quoted in the same study concludes that “poorly designed user interfaces can profoundly affect the bottom line. The expenses associated with a bad UI, over the course of the application's lifetime, may end up being many times the cost of the application itself.” It seems the costly implications of a bad UI, including increased new user training times, a decrease in productivity, and poor user adoption.

Against this background of wasted investment the Keystone Strategy asks questions that strike at the heart of the matter, for example, “How can an application have a deep impact on productivity at a company when such a limited set of employees has access to the system?”; “Why would an investment in a tool as powerful as an ERP system and with such potential to boost business performance be limited to such a small set of employees?; and finally, “Of the already limited number of licenses purchased, why do nearly half go unused?

Indeed, very interesting questions, but how does one go about measuring End-User productivity? Measuring the impact of applications on ERP end-user productivity is a complex undertaking because of the wide range of business functions, vertical industry requirements, and user types who interact with such systems. So for this study, the authors developed a business productivity framework to measure how ERP end users feel Microsoft and SAP applications impact their personal productivity.

Building on existing industry standard usability tests such as the Software Usability Measurement Inventory (SUMI), Keystone concluded that business productivity is a function of six factors, namely; Usability, Familiarity, Transactional Efficiency, Flexibility, Business Insight, and Collaboration.

Usability measures the user's perception of how easy the applications are to use, how “in command” of the application a user feels, how easy it is to navigate in the software, and how much the user enjoys using the software. Familiarity measures the user's perception of how intuitive the application feels, how easy it is to learn, how quickly they can become proficient with the application, and how comfortable they feel using it. Transactional Efficiency measures the user's perception of how easy it is to execute common and repetitive tasks, the efficiency of the interface as it pertains to those common tasks, and the speed and reliability of the software. Flexibility measures the user's perception of how easy it is to execute infrequent or unusual tasks in the application, how easily the software can be adapted to meet specific new business needs and processes, and how agile the software is in handling problems that arise unexpectedly. Business Insight measures the user's perception of how well the software enables easy and comprehensive reporting, access to real-time information, visibility into cross-departmental information, and the ability to gauge the impact of business decisions. Finally; Business Insight measures the user's perception of how well the software enables easy and comprehensive reporting, access to real-time information, visibility into cross-departmental information, and the ability to gauge the impact of business decisions.

Behind each of these six categories the researchers developed 85 questions or statements with which users registered their level of agreement. Typical statements included, “ I sometimes don't know what to do next with this software” ; “There is too much to read before you can use the software” and “There are too many steps required to get something to work.”

The depth of thinking built into the research methodology is in sharp contrast to the way that companies usually set about evaluating software purchases. Time after time companies demote “usability” to one line or question in an Invitation to Tender, typically, “Is the software user friendly?” Yet as the Keystone and other research shows, the quality of the user interface can have a profound and costly impact on user productivity if the wrong decision is made.

Habitually companies place far more emphasis in evaluation on comparing features and functions of the software. The inadvisability of this approach is illustrated by the results of the Keystone research which suggests that there is less than 8 percent difference in the transactional efficiency between typical SAP and Microsoft applications – with Microsoft apparently having the upper hand.

Pronounced differences (17 percent or more) between the two suppliers' offerings come to light around issues of usability, collaboration, familiarity and flexibility with Microsoft once again perceived to have the advantage. However the results need to be viewed with caution. The sample size was small (99 companies) of which two thirds were Microsoft and whereas 65 percent of the Microsoft sample looked at usability in the finance function only a third of the SAP sample covered the same area – the rest was sales & Marketing (a third) and Operations (a third). Whilst, the sponsor of the survey was not revealed to respondents and the individuals administering the survey did not know who the sponsor is there is no indication of the seniority or relative seniority of the users giving the answers. In a sense, the execution of this survey does not appear to match the quality of the methodology.

But the value of the study remains high. It vividly shows that buyers and users of business software have been looking in the wrong place – concentrating on functionality rather than usability. In reality, package functionality is converging – rather like 2 litre cars there is often little to choose between them in terms of raw capability. Like cars, choosing ERP software has come down to looks, usability and of course the organisation behind the product.

About Us