Service Operations Center

LiveVault Service is a managed service providing continuous online backup, recovery, and electronic vaulting. The most important benefit of this managed service is to deliver complete peace of mind to the customer, through the knowledge that their data is always secure, and available when needed. With LiveVault Online Backup Service, that benefit is achieved with the Service Operations Center (SOC). The SOC monitors all critical systems, and takes a unique proactive approach to provide a wide range of monitoring and troubleshooting capabilities.

The success of LiveVault Service is dependent upon the reliability of the underlying technology used in its infrastructure and the strength of the organization that stands behind it. In considering this managed service, it’s important to know:

How reliable are the networks and the systems that back up and vault my data?
Who manages, monitors, and maintains this infrastructure?
Does the service monitor the whole infrastructure, all the time?
If the network goes down or hardware fails, what happens?
If I have a question or problem, can I get immediate help?

LiveVault Service addresses all of these concerns with its Service Operations Center. The SOC proactively monitors all critical systems 24 hours a day, 7 days a week, including: protected customer servers, Internet connections and gateways, LiveVault Online Backup Service backup servers, and LiveVault Online Backup Service web servers. In addition to monitoring, maintaining, and troubleshooting the entire LiveVault Online Backup Service infrastructure, the SOC also monitors the status of backup and restore operations in progress.

The SOC provides world-class technical operations and customer support with:

Complete infrastructure coverage
Full range of monitoring, troubleshooting, installation, and restore support services
Best-in-class network diagnostic and troubleshooting tools and technology
State-of-the-art operations center with complete redundancy and fail-over capability
Trained, experienced, and dedicated staff
Detailed escalation and crisis management policies and procedures.

Operations overview

The basic mission of the Service Operations Center is to be the eyes and ears of the customer, so that every customer is secure in the knowledge that someone is continuously monitoring backup and restore operations and that critical data is always protected. The SOC is there to ensure that all functions run as smoothly and transparently as possible. Top-of the-line hardware, state-of-the-art software, and regular preventive maintenance assure peak performance and availability. If problems do arise, the SOC identifies and manages them to a prompt and satisfactory resolution, often before the customer is even aware that the problem existed.

SOC responsibilities

The SOC has a wide array of responsibilities and competencies. One of the most important is its ability to monitor Internet traffic and connectivity, Wide Area Networks (WANs), and IT infrastructure. Components the SOC monitors include:

The Internet:
- Data transmission during backup and restore operations
- Customer access to MyLiveVault, the web management interface
- SOC access for monitoring and troubleshooting
LiveVault Online Backup Service Gateway servers, located at an offsite data center, which run LiveVault Online Backup Service Internet Gateway software
LiveVault Online Backup Service backup servers, located in an offsite data center, which capture byte-level changes and manage the actual backup process
LiveVault Online Backup Service web servers, through which customers can check the status of their data protection via MyLiveVault
LiveVault Online Backup Service intranet web servers, through which the SOC monitors customers’ data protection.

Additionally, the SOC is customer service oriented. Along with troubleshooting and technical skills, SOC personnel have years of experience with the interpersonal and communications skills necessary to manage the technical details of a crisis.

Monitor the infrastructure

The SOC proactively monitors the entire LiveVault Service infrastructure. This includes the secure connection through the Internet, the LiveVault Online Backup Service Internet Gateway servers, the LiveVault Online Backup Service backup servers in the data center, and the LiveVault Online Backup Service web servers. Most monitoring activity is concerned with detecting and diagnosing Internet connectivity problems, firewall and gateway issues, and hardware or software problems.

The SOC’s primary monitoring tool is Micromuse’s Netcool suite. This industry-leading software provides real-time updates on network faults and service availability. Netcool is used by many of the world’s leading telecommunications and Internet service providers to ensure the uptime of large-scale, network-based customer services and applications. Netcool software quickly alerts operations staff of problems so that immediate action can be taken to resolve them.

The SOC supplements Netcool with its own monitoring tools. The Enterprise Monitoring Tool (EMT) monitors the status of LiveVault Online Backup Service servers in the data centers, and the LiveVault Admin Tool is used to diagnose LiveVault Online Backup Service server problems detected by EMT.

Troubleshoot problems

The SOC troubleshoots any problems that may be detected and serves as a point of escalation for issues that require involvement of other parties. When an alert occurs, a LiveVault Online Backup Service service operator first attempts to determine the precise nature of the problem—whether it is network, hardware, or software related. Depending on this determination, the SOC then manages the problem to resolution, escalating if necessary to other appropriate parties.

Since all LiveVault Online Backup Service servers consist of Dell hardware, the SOC uses Dell OpenManage to access those LiveVault Online Backup Service servers remotely and find out if the servers are operational. With this and other best-of-breed troubleshooting tools, such as HP OpenView, SOC service operators can troubleshoot all critical infrastructure components.

Restore operations

Another important function of the SOC is to monitor and manage customer restore operations. At any time, a customer can initiate a restore operation through MyLiveVault or by contacting Customer Service. Whenever a restore request is initiated, the SOC is alerted and begins to monitor and manage it for optimum results. For every restore initiated, the SOC automatically determines the best and fastest possible delivery mechanism depending on file size, and lets the customer know if they should use an Internet restore for smaller files, or overnight delivery of a CD or NAS for very large data sizes.

For all restore operations, customer data is encrypted and protected using the latest digital technology, regardless of whether the restore is delivered over the Internet or by physical media. Moreover, data center personnel who handle tapes and other media are bonded, and only perform tasks that the SOC has preauthorized. The SOC is responsible for coordinating with the data centers to get the correct tapes and ensure that the NAS or CD units are correctly written, protected, labeled, and shipped.

SOC facilities

The LiveVault Online Backup Service SOC is designed and equipped for maximum security, reliability, availability, and performance. The primary SOC facility is housed in a safe and secure environment, located in LiveVault’s Marlborough, Massachusetts corporate headquarters, in the heart of New England’s “technology corridor.” This site is protected by a security system with round-the-clock monitoring. Dedicated video monitors display signals from cameras facing all entrances that have access to the SOC. In addition to SOC site security during normal operations, there are provisions for operation of a virtual SOC—including a redundant secondary operations center—in case power outages, bad weather, or other emergency temporarily impedes travel or interrupts operations at the primary site.

In addition to a safe and secure physical environment, all LiveVault Online Backup Service hardware and software is carefully selected for optimal performance and reliability. All LiveVault Online Backup Service servers use certified Dell hardware. This means that every Dell workstation and server in the network has been tested and certified by Dell to ensure optimum performance and reliable operation of LiveVault Online Backup Service products and services on this hardware. LiveVault Online Backup Service software is designed to recover automatically from network failures, through a connection integrity feature that automatically restores the secured connection from the customer server to the LiveVault Online Backup Service data center if a network interruption occurs. After a network interruption, a job that was in progress picks up where it left off. For restore operations, multiple levels of redundancy, active testing and fault-tolerance ensure that the file version a customer is looking for is available and correct.

The SOC contains workstations, projection screens, security monitors, and a control console to provide a complete operations support environment. Three large projection screens display video inputs from a variety of sources, including any of the management workstations, satellite TV signals, and other inputs. The main center screen is typically reserved for displaying and tracking Netcool alarms. This allows operators to drill down and use other utilities to diagnose and/or troubleshoot a problem while continuing to monitor the alarm status on the main projection screen.

SOC organization and staffing

By monitoring critical systems 24 hours a day, 7 days a week, the SOC ensures that customer data is always safe, always backed up, and always available for immediate restore. The SOC is staffed by teams of experts who combine specialties in network technology with strong customer support skills. As a proactive team, these experts know how to diagnose and troubleshoot problems, when and where to escalate them, and how to communicate with customers. The SOC personnel are also armed with an extensive array of custom-developed policies and procedures, providing an appropriate detailed plan for them to follow in every conceivable scenario.

The LiveVault Online Backup Service total service strategy

The Service Operations Center is a key component of the LiveVault Online Backup Service total service strategy, in which the SOC coordinates with LiveVault Online Backup Service Customer Service, engineering, and product organizations, with data center personnel, and with external network carriers and equipment vendors to provide comprehensive customer service and protection through every stage of LiveVault Service. In providing total customer service and support, Customer Service may escalate a customer request to the SOC that, in turn, may involve other LiveVault Online Backup Service or outside organizations to resolve the issue. Whenever an issue is escalated, the SOC takes full responsibility, and manages the crisis by working its resources until the issue is fully resolved, while sending regular status updates to the customer.

Continuous coverage, comprehensive support, complete protection

In selecting an Internet-based online backup service, two major considerations are its technology infrastructure and its support organization. Realizing that an online backup service requires the complete reliability of these two components, the LiveVault Online Backup Service Service Operations Center is dedicated to maintaining the health and well-being of the LiveVault Service infrastructure, and providing continuous data protection and total peace of mind for customers.

The LiveVault Online Backup Service Service Operations Center is part of a total service strategy designed to provide comprehensive data protection and customer service. With round-the-clock operation, trained and dedicated staff, well-defined escalation procedures, coordination among LiveVault Online Backup Service and third-party support organizations, state-of the-art facilities, best-of-breed equipment and tools, and complete operational redundancy, the LiveVault Online Backup Service SOC ensures high availability, continuous protection, and timely support under all conditions to ensure that your data is always protected and always available.