Why Sentry Software Chose OpenTelemetry for Hardware Sentry

A quick walk down Memory Lane!

Hardware Sentry® has been detecting server hardware failures for BMC Software's customers since 2004 - time flies by! At that time, the product was known as Hardware Sentry KM for PATROL. It was a module for the BMC's monitoring agent: the PATROL Agent. For almost 20 years, Hardware Sentry has remained in this form through the various evolutions of BMC's monitoring ecosystem: ProactiveNet, Performance Management, and TrueSight.

Aiming for the cloud

Recently, BMC introduced BMC Helix Operations Management, a new SaaS platform based on open standards. BMC Helix is backward compatible and can consume the data collected by a PATROL Agent. However, the core components of BMC Helix (VictoriaMetrics and Grafana Dashboards) require a  specific data format, which cannot be provided by a KM for PATROL without breaking the existing consoles and the views in TrueSight. We therefore decided to move away from the original PATROL design and embrace the new capabilities offered by the new Helix platform.

Considering that VictoriaMetrics is 100% compatible with Prometheus and that BMC Helix exposes a REST API endpoint to push metrics using the Prometheus Remote Write protocol, we designed the first alpha version of our “next-gen” monitoring product as a Prometheus exporter. Users just had to configure VictoriaMetrics' vmagent to scrape the exporter and push the metrics to BMC Helix. Except for a few timing hiccups, it was working fine.

Embracing the OpenTelemetry revolution

During our research to make the configuration and the architecture simpler, solve timing hiccups, remove the vmagent, and add filtering and processing capabilities, we discovered OpenTelemetry (OTel) and its Collector. The OpenTelemetry Collector is a vendor-agnostic proxy that can receive telemetry data in multiple formats, filter and transform this information, and export it to one or more back-ends. It is composed of three components:

• Receivers to get data into the collector.
• Processors to define how the received data is processed. OpenTelemetry processors help address availability and performance issues by reducing the number of outgoing connections to transmit data and by preventing memory errors.
• Exporters to define where to send the received and processed data. OpenTelemetry exporters provide powerful options for reporting telemetry data. They allow exporting metrics from an instrumented application to another third-party system (Prometheus, BMC Helix, Datadog, Splunk, Dynatrace...etc.).

Thanks to the OpenTelemetry Collector, we have full control of our data and the ability to send data to multiple destinations in parallel through configuration. The tools, APIs, and SDKs are available for most programming languages (at Sentry, we opted for Java and Go), which provide high interoperability across different languages and environments. Because OpenTelemetry is supported by a large community, extensions are regularly added to the collector to provide more capabilities (health check, service discovery, data forwarding, etc.) therefore, providing more value to our end users. And icing on the cake, OpenTelemetry is adopted and natively supported by observability leaders like BMC Software, Datadog, Grafana, Prometheus, New Relic, Splunk, etc.

Now that Hardware Sentry is based on OpenTelemetry and can help IT administrators report and optimize the carbon footprint of their servers, it becomes obvious that the solution should benefit to the majority, and not be limited to BMC Helix and Prometheus users!

Meeting the challenge of carbon neutrality

Our developers were really excited by this new project. In a few months, they created a custom distribution of the OpenTelemetry Collector: the Hardware Sentry. This first version focused on the ability to push metrics to Prometheus and BMC Helix. We also developed dashboards for Grafana and BMC Helix which leverage the collected data to report hardware issues in the monitored systems, as well as their electricity usage and CO₂ emissions.

Our team is working now on the integration of other major observability platforms that natively support OpenTelemetry, starting with Datadog and Splunk. Like in Grafana and BMC Helix, we will be providing dashboards and predefined alert rules to help users get the solution working out-of-the-box, with minimal configuration.

Hardware Sentry KM was vendor-agnostic for the monitored systems (HP, Cisco, Dell EMC, Huawei, etc.). Now Hardware Sentry is vendor-agnostic for the observability platform as well!

Achieving carbon neutrality while saying goodbye to vendor lock-in and embracing vendor-portability is now possible with Hardware Sentry. Are you ready to dive in?

Learn more about OpenTelemetry Discover our solution Talk to an expert

If you have any question about Hardware Sentry, reach out to our experts .