Managing a Linux system can be as easy as connecting to a remote system and run some of the system tool available with most Linux distributions. It gets difficult when you have to manage multiple remote systems, and many Linux admins do work in environments with hundreds or even thousands of systems.
Managing a Linux system can be as easy as connecting to a remote system and run some of the system tool available with most Linux distributions. It gets difficult when you have to manage multiple remote systems, this can start with a few web servers you are maintaining and go all the way to thousands of systems, especially in enterprises and server farms.
Since the introduction of remote server management, both the technology from vendors and the requirements of IT organizations have changed dramatically. Many of the management technologies that have been available in server platforms for years are moving to the enterprise desktop bringing system management not only to enterprise data centers, but also to the small business and home office environment.
For years Linux system management was and is still possible using available open-source tools and system utilities that ran on the system and collected information that a system administrator could pull using a management console remotely. When it comes to choice, the Linux system administrator has a range of solutions to choose from, however, the need to converge on a consistent solution has emerged. IT professionals need standardization to improve operational efficient in heterogeneous environments and to perform certain commodity functions in a common and standard based way.
From the perspective of a system administrator, regardless of how many systems he is managing, the perfect manageability solution should fulfill the following basic requirements:
True Out-of-Band Manageability
Provide standardized ways of representing and transmitting management data independent of the system hardware, management console, or the state of the managed system. With current management tools, administrators might have several different management consoles and tools to manage systems and devices from different vendors and with different operating systems leading to confusion or an increased need for training.
Secure and Reliable Manageability
Use secure and reliable protocols for communicating between management consoles and managed systems or devices. A system administrator needs to be able to manage systems or devices regardless of their physical location and beyond firewalls and proxies.
Give the system administrator full control over the system by means of remote power management and redirection of system messages and boot devices over the network.
In-Band Access to Manageability Features
A system administrator should be able to access hardware and system alerts, inventory data and health sensors from the installed OS, for example during initial setup and the provisioning phase.
The Perfect Solution
A “traditional” and common out-of-band management solution involves connecting each system’s serial console port to a console server. This implementation allows the monitoring of hardware self-test information and console. This solution however does not scale well. Just imagine all the cables and console servers needed for a typical size HPC cluster or a web server farm. Another type of management solution is a remote access card with its own processor, memory, battery, network connection, and access to the system bus.
A baseboard management controller (BMC) is a specialized microcontroller embedded on the motherboard of many computers, especially servers, which allowed IT administrators to access, diagnose and restore frozen servers. The first service processor technologies were vendor specific and added to IT management complexity. As a result, demand for vendor-independent service processors emerged. Intelligent platform management interface (IPMI) was introduced to provide a way to manage servers in today?s complex data centers.
The IPMI specification defines a set of common interfaces to computer hardware and firmware which system administrators can use to monitor system health and manage the system. IPMI operates independently of the OS and allows administrators to manage a system remotely even in the absence of the OS or the system management software, or even if the monitored system is not powered on. IPMI can also function when the OS has started, and offers enhanced features when used with the system management software.
On Linux, several open-source implementations exist to abstract IPMI information to a level that will make it easy to use:
OpenIPMI: An IPMI client (http://sourceforge.net/projects/openipmi)
ipmitool: A Command-line interface to IPMI (http://sourceforge.net/projects/ipmitool/)
ipmiutil: IPMI client-side utilities (http://sourceforge.net/projects/ipmiutil/)
GNU Freeipmi: IPMI libraries and tools (http://www.gnu.org/software/freeipmi/)
A mature implementation is OpenIPMI which consists of two main parts: A device driver that goes into the Linux kernel, and a user-level library that provides a higher-level abstraction of IPMI and generic services that can be used on any operation system.
IPMI has some disadvantages when compared with emerging management technologies based on a common data model and secure and routable protocols. Additionally, IPMI is a server technology which does not cover the complete spectrum of systems and devices wanting to be managed.
Console and IDE redirection extend the control and provide powerful tools for system rescue, diagnosis and monitoring. The ability to view console message over the network and being able to reboot a system from a remote CD-ROM to fix a problem on the local hard disk require support for such virtual device in the kernel. To be able to use the virtual, remote drive under Linux, the OS has to be aware of the device.
Both features and many other manageability features are no luxury items anymore. Until recently, such features were only available on’ expensive’ server hardware usually affordable only by IT departments if big enterprises. Many desktop platforms available on the market today have these features and more making it possible to for a hobbyist to start managing her systems the professional way.
The common data model
The type and structure of management information that describes a managed system is known as a data model. The data model standard used in many management applications is the Common Information Model (CIM) which is defined by the Distributed Management Task Force (DMTF). CIM provides a common definition of management information for systems, networks, applications and services, and allows for vendor extensions. CIM’s common definitions enable vendors to exchange semantically rich management information between systems throughout the network. Because it defines the management data in a common way, it enables management tools from a variety of vendors to be platform independent.
CIM is the underlying data model for management initiatives such as Web-based Enterprise Management (WBEM), ” Desktop and mobile Architecture for System Hardware” (DASH) and” Systems Management Architecture for Server Hardware” (SMASH).
DASH is a DMTF Management Initiative that represents a suite of specifications which standardize the manageability interfaces for mobile and desktop hardware. The DASH suite of specifications defines the interfaces for management in the form of protocols and profiles for representing mobile and desktop hardware. SMASH on the other hand is an initiative is a suite of specifications that deliver architectural semantics, industry standard protocols and profiles to unify the management of the data center.
The trend recently is to move hardware management to the hardware and make it available to the IT professional directly without the mediation of an operating system. This is moving away from times when the hardware data was only accessible through special interfaces in the OS kernel. The System Management BIOS (SMBIOS) is an example where management information is available, however only through software interfaces on the local host. To retrieve such information, a software agent running on the system is needed.
The hardware management information alone is not enough and to complete the picture, the same principles need to be used in the host OS and in user space. This is becoming reality in the Linux world with the increasing popularity of management standards such as CIM and interoperable protocols such as WS-Management. Together they provide a complete picture, in-band and out-of-band.
WS-Management is emerging as the preferred programmatic interface for system management. WS-Management is a web services-based specification and provides the security and routability characteristics available in web-based protocols and makes it possible to build distributed system management solutions that are OS-independent. It provides a common way for systems to access and exchange management information across the entire IT infrastructure?hardware, software, and applications.
The WS-Management specification promotes interoperability between management applications and managed resources by identifying a core set of web-service specification and usage requirements to expose a common set of operations that are central to all systems management.
Support for WS-Management on Linux becoming reality with different open-source implementations already available. One of the first implementation for Linux that provides both client and service support and interacts with existing manageability technologies such as CIM is Openwsman (http://openwsman.org). Openwsman exposes system management information on the Linux operating system using the WS-Management protocol.
Using openwsman it is possible to manage any system or device with a WS-Man stack over the network. This includes managing windows systems with Vista which has a WS-Management stack and exposes many interfaces and resources that can be managed over the wire. Recent desktop platforms from Intel known as vPro also have WS-Management as part of the Intel Active Management Technology.
The core hardware architecture of Intel AMT is resident in firmware. The micro-controller within the chipset’s graphics and memory controller (GMCH) hub houses the Management Engine (ME) firmware, which implements various services on behalf of management applications running on the local OS. Additionally, flash memory houses system BIOS, code used by the management engine, and a third-party data store (3PDS) that enables applications to store information as needed in non-volatile memory.
As you see, the figure shows the how manageability technologies or moving into their own space in the hardware.
While the majority of Intel AMT features are accessible over the network and without the presence of an operating system, some operations can be initiated from the OS running on the managed system or in some scenarios the Manageability Engine can initiate communication with the OS for example to deliver notifications to services subscribed to the notification manager in the AMT subsystem of the manageability engine. Communication between the local host operating system (OS) and the ME is accomplished by means of the Manageability Engine Interface driver. The Manageability Engine Interface driver is bi-directional, as either the host OS or Intel AMT firmware can initiate transactions.
The Intel Manageability Engine Interface driver and the Local Manageability Service are now available from the AMT open-source project (http://openamt.org). Additionally, the MEI driver has been submitted to the mainline kernel and is undergoing a review from the kernel developer community.
A sample application (
wsmid_identify) provided as part of the openwsman client can communicate with different systems and devices. In the case of AMT the data is retrieved independent of the power state of the system. The identify request is the most basic operation in WS-Management and serves to identify the version of the protocol and the stack being used on a certain device or OS and shows how interoperability helps one management application manage different type of systems.
The conclusion is that hardware management interfaces are moving out of the kernel and will in the future be available directly from the hardware, regardless of the power state. The role of the kernel is not gone, special kernel modules make it possible to access hardware and system management information in the firmware and management boards. On the other hand, user space management agents complement hardware information through similar interfaces and protocols. In short, management technologies are converging and already now it is possible to manage heterogeneous environments with different OSes using one management console application.