OpenUSM is a modern approach to Server Management, Insight Logs Analytics and Machine Learning solution integrated with monitoring & logging pipeline using Docker containers & Redfish. It is 100% container-based platform-agnostic solution which can be run from laptop, server or cloud and works seamlessly on any of Linux or Windows platform with Docker Engine running on top of it. It follows “Container-Per-Server(CPS)” model. For each server management tasks, there are Python-scripts which when executed builds and run Docker containers, uses Redfish API to communicate directly with out-of-band monitoringtool, collects iDRAC/LC logs and pushes it to ELK(Elasticsearch, Logstash & Kibana) stack for further log analytics and Machine Learning. OpenUSM is currently hosted at https://github.com/collabnix/openusm
OpenUSM today support both Linux and Windows Platform. It has already been validated on Linux OS like Debian, Ubuntu and CentOS system. You can find extensive documents here.
Under this blog post, I will showcase how to get started with OpenUSM on Docker for Windows Platform.
Tested Platform:
- Microsoft Windows 10 Enterprise
- X64 based PC
Pre-requisite:
- Installing Python 2.7
- Installing Winsyslog
- Installing Docker for Windows
- Configuring Docker for Windows
Installing Python 2.7
To install Python 2.7, the simplest way is to use Chocolatey. Chocolatey is software management automation. Chocolatey works with over 20+ installer technologies for Windows, but it can manage things you would normally xcopy deploy (like runtime binaries and zip files). You can also work with registry settings or managing files and configurations, or any combination.
Run the below command to install Chocolatey on your Windows 10 laptop:
Set-ExecutionPolicy Bypass -Scope Process -Force; iex ((New-Object System.Net.WebClient).DownloadString(‘https://chocolatey.org/install.ps1’))
If you face any issue, do refer this link.
Now install python using choco as shown below:
choco install python
You can verify if python is installed or not using the below command:
PS C:\> C:\Python27\python.exe -V
Python 2.7.15
Installing WinSyslog
For OpenUSM to work, syslog server is required. You can install any Syslog server available in the internet. For this demo, I will use WinSyslog which is too easy to setup. Once you install it, it will open up the window as shown below:
Click on Options section under File Menu and you shall see the below window:
As shown above, you will need to enter your local laptop IP address and port to fetch logs for OpenUSM. You can test it by clicking on “Send” and you shall see “Syslog Messages send successfully to 192.168.1.6” if it goes well.
Installing Docker for Windows:
Open https://docs.docker.com/docker-for-windows/install/ and click on “Install from Docker Store” to open up the below page to download Docker for Windows CE Edition.
Docker CE for Windows is Docker designed to run on Windows 10. It is a native Windows application that provides an easy-to-use development environment for building, shipping, and running dockerized apps. Docker CE for Windows uses Windows-native Hyper-V virtualization and networking and is the fastest and most reliable way to develop Docker apps on Windows. Docker CE for Windows supports running both Linux and Windows Docker containers. You can install either of stable or Edge release from the below link.
Double-click Docker for Windows Installer to run the installer. When the installation finishes, Docker starts automatically. The whale 
in the notification area indicates that Docker is running, and accessible from a terminal.
You can verify Docker version either by visiting “About Docker” in the top menu:
Or you can open a command-line terminal like PowerShell, and try out below Docker command to check the version –
Configuring Docker for Windows for OpenUSM
We need to perform few of configuration changes related to Docker for Windows before we proceed with setting up ELK stack. First we need to enable share drives for ELK stack to work. Docker for Windows provides you a simplified approach to enable this feature. Click on Whale Icon > Shared Drives > Select “C:” local drive which will be made available to your Docker containers which run ELK Stack.
Once you select and click on “Apply” it will restart Docker as well as Kubernetes(if enabled earlier). This should be good enough for OpenUSM to work smoothly.
Cloning the OpenUSM Repository
git clone https://github.com/collabnix/openusm
cd openusm/logging/
Setting up ELK Stack
Docker for Windows is a development platform and comes with docker-compose installed by default. All you need is to run the below command to bring up ELK stack… Awesome, Isn’t it?
docker-compose up -d
You can verify if ELK has come up or not by running the below command as shown:
Open up http://127.0.0.1:5601 to access Kibana UI as shown below:
Sending HW Sensor Logs to ELK Stack
It’s time for an action now. I assume you are connected to your Lab infrastructure using VPN in order to access the server. It just takes few seconds to send sensor logs(Fan, Temperature etc.) of server sitting in your datacenter to ELK stack using the below python script.
PS C:\Users\Ajeet_Raina\openusm\logging> C:\Python27\python.exe .\sensorexporter.py -i <idrac-ip> -ei 127.0.0.1 -eu elastic -ep <password>
This script uses Redfish to talk to remote server, fetches the logs and send it to syslog server which we configured earlier, pushes it to Logstash and elasticsearch and get it displayed via Kibana UI – all in just few seconds. Isn’t it cool?
Visualizing the logs under Kibana UI
When you open kibana UI for the first time, the index pattern mightn’t come up. Click on “Index Pattern” under Management tab on the left hand side. Next, click on “Create Index Pattern”. Search for Fan* and temp*. By now, you should be able to see temperature and Fan speed logs under Discover tab.
Click on “Discover” tab to see the overall logs fetched directly from iDRAC IPs.
Click on “Visualize” tab to add filter. In the below example, I have chosen iDRAC IP, Minimum and Maximum Reading as shown below:
Click on “Dashboard” to add specific filters for Fan speed, choose your type of visualization(I selected “Pie Chart” option) and select the metrics to display it as shown below:
In my next blog post, I will talk about Elastic’s Machine Learning “anomaly score” and how the various scores presented in the dashboards relate to the “unusualness” of individual occurrences within the data set of fan speed and temperature as a whole. Stay tuned !
