Integrating on-demand and on-premise data – now that’s a challenge!

14th May 2012

FSN writer, Lesley Meall looks at the challenge of integration between the Cloud and on-premise world and finds that for those brave enough do it themselves there are some Cloud-based integration tools available. But for those without steady nerves there are tools more suitable for the IT department.

Integration has always been a challenge for any business using more than one software application that collects, or generates or uses common data. Whether it’s the names and addresses of suppliers, customer reference codes, or core financial data, challenges such as data duplication, inconsistencies, errors, and disconnected islands of data (and how to avoid or overcome them) are perennial, whether you are trying to integrate multiple on-premise applications (and the associated data), integrate on-premise applications with software that is being provided ‘as a service’ and data that is living in the cloud, or integrate multiple cloud-based apps. 

‘Because they want their data to reflect a consolidated global instance of the truth, a lot of organisations have spent the past decade simplifying and standardising their processes, and driving towards integrated architectures,’ says Tony Chauhan, an analyst with the Hackett Group, a global consulting and finance strategy firm. ‘But cloud computing tends to fragment architectures, and more importantly, data, so it flies in the face of what many organisations have been working so hard to achieve.’ This doesn’t mean it can be avoided: the accessibility of SaaS means that divisional heads, departmental managers and even individuals can easily create external silos of information – and integration issues. 

How you address these will depend on a host of factors. These range from the ‘connectivity’ offered by your chosen software applications, to the financial and human resources at your disposal – and your willingness to handle things in house or pass it to an external provider with the necessary expertise. Integration means different things to different people (and different software providers). As one seasoned IT veteran observes: ‘Integrated almost always makes business lives easier; integration is almost always a costly and complex business.’ An integrated software application that can encompass accounting, ecommerce, order management, stock control and more is a more straightforward proposition than the disconnected alternative – no matter where the associated islands of data live. 

There’s a big difference between ‘integrated’ applications which can share the same underlying database or data warehouse, and ‘piping’ data between the multiple data repositories used by multiple applications. So there are lots of different data integration challenges and ways of addressing them. ‘It’s all about understanding what the traffic flows are,’ says Adrian Taylor, chief technology officer, Data Integration. The reality is more complicated, because getting integration between some or all of your islands of data could mean bi-directionally copying modified data between online environments and on-premises data repositories or warehouses, or replicating data in real time to allow applications living in the cloud and on the premises to access the same information – and more besides. 

For the international fashion label Ted Baker it was the former. Ted Baker runs its finance systems on servers in the UK and in Hong Kong and has a cloud-based budgeting and planning system, so it updates the ‘data cube’ in the latter from the transactional finance system once each day. ‘We run a routine that creates an output file and that goes to the cloud-based server each night,’ says Charles Anderson, head of finance at Ted Baker. But not all business systems (or integration challenges) lend themselves to this approach. The entrepreneur Andreas Doppelmayr needed something more ‘real time’ to link two of the applications he uses to run his retail web shop

Doppelmayr doesn’t have his own IT department, and he tries to do as much of his own IT work as he can, so he decided to use a specialist cloud-based integration as a service tool and take the ‘do-it-yourself’ approach. ‘I was really excited when I discovered Carry The One, because it made the integration process look so easy,’ he says. It’s one of a number of cloud-based services that promises to make integration into relatively uncomplicated drag and drop exercise – and the reality did not disappoint. ‘I followed the step by step guide and the initial integration between the Magento ecommerce software and E-conomic accounting system took me just a few minutes,’ reports Doppelmyr – Carry The One suggests you allow around 30 minutes. 

Even organisations with much more complex integration hurdles to leap are managing the process themselves using on demand integration tools– though some of these are aimed at organisations with in-house IT experts. Although the educational software provider Blackboard decided to take the DIY approach to integrating a couple of its on-demand and on-premise software applications with the help of the WebSphere Cast Iron Cloud Integration system from IBM, Blackboard had its own software engineers on hand to help. ‘We considered building the custom integration code ourselves,’ says Jon Lal, a senior application engineer at Blackboard, before it opted for the IBM system (which shows how ‘involved’ the business was equipped to get). 

Blackboard needed to integrate systems with overlapping data and functionality. ‘We have two CRM systems in place,’ says Lal, the back office on-premise ERP PeopleSoft and the cloud-based CRM SalesForce. ‘The sales team didn’t see the PeopleSoft CRM as a good fit for their sales force automation tool,’ says Lal, who had to integrate both systems. ‘Our goal was to get the best that PeopleSoft has to offer in order management, contracts and finance, and also to leverage everything that SalesForce has to offer.’ In practice, this meant that if a sales user logged into SalesForce and made a change to an account, that change would ‘pipe over’ so that an order person working in PeopleSoft could also see the change in that system. 

‘Integration was required to maintain our data in both systems,’ says Lal, who opted for IBM’s WebSphere Cast Iron Cloud Integration because it could meet those immediate integration needs and support Blackboard’s growth plans. ‘It allowed us to streamline our integration processes so that we could deliver the CRM project on time,’ says Lal, ‘and it also offered us the ability in the future to integrate other software products as needed.’ Not that IBM and Carry The One are the only specialists offering cloud-based data integration software; others include Appresso, Dell-Boomi, and Sesame Software. So if you do decide to rise to the data integration challenge, you will not be short of tools to help.