Analytics-driven management stands to end the key challenges that constrain change and configuration management. By applying powerful analytics to the overwhelming change and configuration data, IT Operations Analytics (ITOA) technology can turn massive amounts of information into clear, actionable insights.
Information technology (IT) organizations are struggling. With constrained budgets, tighter efficiency requirements, and a need to streamline costs, IT managers are challenged to launch new applications, maintain high levels of application availability, and deliver on strict Service Level Agreements (SLAs). It’s typical for IT teams that support a wide variety of applications running on distinct platforms that they will face complex operations to manage today’s enterprise data centers, fighting to overcome change and configuration management problems that affect performance and availability. While a variety of IT management tools have been implemented to automate and control IT operations, they were not designed to deal with the complexity and dynamics of the modern data center, leaving IT operations overwhelmed with lots of raw data, which makes change and configuration problems a chronic pain for IT operations.
Analytics-driven management stands to end the key challenges that constrain change and configuration management. By applying powerful analytics to the overwhelming change and configuration data, IT Operations Analytics (ITOA) technology can turn massive amounts of information into clear, actionable insights.
Complexity, Dynamics, and Silos
It is a major undertaking for IT operations to link, match, cleanse, and transform data across systems into useful information, before IT management spirals out of control. Today's volume of data is the result of a number of recent developments.
Complexity
Complex IT environments are regarded as part and parcel of a company’s business operations. This complexity comes out of the variety of technologies at play, a high degree of technology customization, a mix of legacy and modern software, and dynamic infrastructure. In the cloud scenario, self-service provisioning has multiplied the amount of activities occurring outside of static processes, going beyond the capabilities of IT management. IT complexity translates into higher costs and reduced agility and flexibility.
Dynamics
The pace of change has dramatically accelerated in today’s IT environments, spurred by dynamic business demands for flexibility and scalability. While being generations ahead of the typical ITIL change control process, IT organizations face greater demands, growth of server virtualization, and a shift to use more dynamic services, like the cloud. IT teams relying on traditional change management processes find it more than a little difficult to keep pace with the frequency of change required to support their environments. As a result, they can end up sifting through what can amount to petabytes of data before they are able to take strategic action.
Silos
Traditional IT management tools view the world from a bottom-up perspective, managing individual components under specific silos. While the silo method of working promotes efficiency within that specific silo, this undermines the possibilities for working collaboratively and seamlessly across an IT organization. The challenge here is figuring out how to manage changes while applications, servers, network devices, and databases are all siloed in various divisions of IT infrastructure, which creates a barrier to visibility. No matter how much you fine-tune these silos, it is not enough to combat the day-to-day availability and performance problems that crop up.
A Big Data Problem: Surge in Volumes of Performance and Event Data
Data is the lifeblood of the modern IT department. It not only provides valuable metrics on overall system health, but it can be a powerful enablement tool for initiatives like agile development and DevOps.
Yet with systems becoming increasingly heterogeneous, customizable, and distributed, monitoring technologies now captures much larger volumes of data per time period. Furthermore, the adoption of agile-style development methodologies and DevOps has dramatically increased the rate at which systems change (in many enterprises by an order of magnitude). These initiatives require an almost instantaneous understanding of how changes affect the overall system.
To continuously extract meaning from event and performance data, IT operations wade through Big Data—mountains of data generated through IT operations at all levels of the business system stack. To make matters worse, the context of this data might change as systems are updated according to business demands, putting these massive volumes of data into motion. Getting bigger and faster, data and its context is growing and changing far more rapidly than IT can cope; a potentially destructive situation if not managed properly. As complex IT environments face growth in the volume, velocity, and variety of sources of data, the real issue is what is ultimately done with the data, not the fact that IT operations collects such large amounts. IT Ops needs to bring disparate data sources (applications, web servers, databases, middleware, operating systems, etc.) generated across the IT organization together and extract valuable, actionable information from the mountains of data collected in enterprise environments.
Still Seeing High-Profile Outages
News headlines seem to have remained consistent over the last fifteen years. With downtime and outages due to configuration issues still making headlines, the chronic nature of change and CM challenges is more apparent than ever. While company names have changed, the bottom line remains the same that these challenges create critical operational issues that can even reach the news. Seemingly preventable internal glitches raise havoc for IT operations consistency.
IT’s Big Data Left Unchecked
To benefit from trends such as automation, cloud computing, and DevOps, infrastructure and operations (I&O) professionals need reliable information more than ever. Yet, IT operations still has a lot of old tools that focus on what the IT landscape looked like ten years ago. Consider the following:
- Configuration management databases (CMDB) don’t go deep enough: CMDB implementations lack the tools to discover efficiently detailed, extensive information for each of the assets that are in an environment, thus becoming more like inventory systems. With the difficulties involved in the implementation of the configuration management database (CMDB) or more recently, the content management system (CMS), this approach only becomes a robust solution when viewing it from at 30,000 feet; it does not provide the detailed information required for handling day-to-day operations.
- Service desk doesn’t see problems: Focused on processes, the service desk provides an effective platform for enablement, automation, and management of pre-defined process workflows. However, the service desk lacks an ability to see final deliverables of these workflows. For example, a service desk can drive a change lifecycle from inception of a change request to the change approval. However, the service desk has no visibility into what happens as the result of this approval once the change is actually implemented and deployed in the production.
- Deployment automation: Automated deployment tools significantly reduce the risk of deployment errors coming from fat fingers, a lack of focus, and other typical human errors. At the same time automated deployment assets become pretty complex and prone to “programming” errors as any software system, with a certain percentage of manual deployment activities taking place even in enterprises that are most advanced in terms of deployment automation. The differences between application and infrastructure deployment processes also contribute to complexities of automated deployment. Hence, environments need to be validated for consistency prior to deployment as well as post-deployment, making sure that mistakes and errors in the deployment process don't impact changes transitioned between environments.
Getting Actionable Insight
Having to deal with data sets so large and complex that it is difficult to meaningfully process, on-hand IT management tools fall short for handling IT’s Big Data. As IT controls evolve every time a change occurs in the infrastructure—whether for the deployment of new hardware or applications or some other change—IT managers need to be able to regularly evaluate the effectiveness of IT control and change management processes.
IT operations need to not only warehouse data, but also analyze, extract, and display the data in a manner that makes it actionable, turning large volumes of data into meaningful information for decision-making. This helps to reduce the cost and time for product development and optimize offerings.
Actionable insights allow IT organizations to control and manage configuration changes on a continuous basis, gaining the visibility necessary to ensure that the infrastructure is secure, compliant to SLAs, and ready to meet business-service demands. Some examples of insights could be identifying the changes and misconfigurations causing an incident, detecting misalignment of environments that can cause deployment failures, isolating unauthorized and undesired changes, uncovering a lack of synchronization between production and disaster recovery environments, and revealing inconsistency in configuration between clustered servers and many other data points that are essential to ensure smooth day-to-day operations.
IT Operations Analytics Makes IT Operations More Effective
Agile development means that changes are introduced to the systems in rapid succession, and application topology can change at a moment’s notice. IT operations doesn’t have the time or personnel to constantly update and understand the system it’s monitoring.
IT operations analytics enables enterprises to collect, index, parse, and harness data from sources as varied as system logs, network traffic, monitoring tools, and custom applications; all of this allows for the churning of this data through a set of smart algorithms in order to provide meaningful insights for IT operations. IT operations analytics and business intelligence (BI) tools are both comprehensive, including any amount of data in any format from anywhere, and they have the capability to correlate the data to provide a centralized view at the finest detail.
The difference is that while BI gives a platform to slice and dice the data as users want, IT operations analytics takes this data to another level of automated insights. IT operations analytics rely on the domain understanding of the data to provide meaningful insights. IT operation analytics give IT operations teams visibility and insight into the behavior of business systems and their performance, automatically identifying and isolating the critical events (such as changes) that have a potential for disruptions. ITOA provides immediate awareness of potential issues, facilitates for a rapid understanding of these issues, and helps IT operations determine the best course of action to restore or meet performance and availability expectations.
Analytics Address Change and Configuration Management Challenges
To provide actionable information, ITOA combines complex-event processing, statistical pattern discovery, behavior learning engines, unstructured and structured information search, topology mapping and analysis, and multidimensional database analysis.
By combining big data and high-powered analytics in the change and configuration management space, IT operations can:
- Determine root causes of failure, issues, and defects in near-real time
- Analyze configuration data to determine consistency between environments
- Calculate risk and take proactive measures to ensure stability
- Determine and rank the relative impact of multiple root causes
- Direct the results to the most appropriate individuals or communities in the enterprise for problem resolution
The ITIL change and configuration management processes have defined the steps needed to ensure a consistent approach to the delivery of IT as a service. However, while these processes have made IT more reliable, efficient, and effective at managing risk, they have not done much to help IT exploit new disruptive technologies.
In increasingly dynamic environments, the complexity of change and configuration requires a cross-domain approach to change and configuration management. As systems become more complex, established IT processes become more bureaucratic and basically slower. The efficiency of these processes is diminished and downtime and the mean time to repair (MTTR) increases, which results in customers receiving a poorer service for longer than is considered acceptable.
Data can be turned into actionable information using analytics tools, such as statistical algorithms, comparison engines, and configuration impact knowledge bases. This increased insight offers IT a greater level of visibility into the environment it is supporting. Being armed with this knowledge of not only the environment, but also what may be affecting normal operations, is invaluable, especially to operations teams seeking technology to help them work more proactively.
Industry Analysts Hail IT Operations Analytics
Turning the data generated through change and configuration management activities into actionable information can help IT operations take proactive management steps, reducing disruptions to normal service, and managing change and configuration more effectively.
Analysts and other industry experts are also voicing their enthusiasm about ITOA:
- "IT analytics is an exciting field because it represents breakthrough innovations that can bring substantial value and lasting competitive advantage to the businesses that adopt them.” This quote is from Forrester Research, Inc. in its “Turn Big Data Inward With IT Analytics” report from December 2012.
- Gartner Research VP Will Cappelli, explained in a recent report, “IT Operations Analytics Technology Requires Planning and Training,” that the “operational data explosion has sparked a sudden and significant increase in demand for ITOA [IT Operations Analytics] systems.”
- Gartner’s recent “Hype Cycle in IT Operations” report, states that “IT Operations Analytics will provide CIOs and senior IT operations managers radical benefits toward running their businesses more efficiently…”
More Than Just Change and Configuration Management
IT Operations Analytics platforms can enrich a wide variety of IT management use cases. For example, an ITOA platform can feed application performance data to a central event management system to simplify monitoring and hasten root cause discovery; give an executive dashboard real-time metrics on conversion rates and SLA misses to assess performance problem impacts; provide granularity detail on configuration changes in IT environments and address configuration drift; identify suspicious changes that can reveal security issues; and send detailed records of failed transactions to log analysis software and cross-correlate application performance problems with underlying infrastructure events.
Prospects for IT Operations Analytics
As we see, data is now the center of the IT department. The success of an organization increasingly depends on the capabilities of IT operations being able to provide a reliable support for the critical business systems and underlying infrastructure, linking business processes and IT more closely to each other. IT operations analytics can help operations ensure that the business side has stable services, supporting the introduction of new applications and infrastructure while analyzing growing amounts of operational data.
