Data Recording

How to Build Better Software
How to Build Better Software 2548 1422 KRONOMETRIX

A better way to design software

Companies are pushing lots of updates to fix different vulnerabilities and software defects at an alarming rate. Many developers are focusing heavily on coding, writing software in sprints, based on user stories, using scrum or other agile methodologies but still fail to deliver robust software.

Programming is not coding!
What is the difference and why is it important to know?

We would like to present three main things how anyone can use to learn build better and more reliable software.

Precise Designs

Being agile does not mean you should forget what and how your software should do. These must be written down in a precise form, not based on plain English prose, static diagrams, user stories or some sort of pseudo-code. 

And that means using formal methods, like TLA+ which will help you validate and find very subtle design bugs before implementing or writing a single line of code, not found through any other method or technology. Formal methods are mathematically techniques for the specification, development, analysis, and verification of software and hardware systems.

Examples

There are many implementations of formal methods used in industry: TLA+, Alloy, Petri Nets and may other implementations. Several companies are using formal methods. For example TLA+: Amazon Web Services uses TLA+ to design its own products (S3, DynamoDB, EBS), Microsoft (Xbox), Intel (microprocessor design),  European Space Agency (Rosetta spacecraft using OpenComRTOS operating system)

Programming vs Coding

“Coding is the easy part of the programming”. Every time you plan to build a new program, there are three main things involved:

  • first you must decide what the program should do
  • then one should think how to do it, for example what algorithm to use
  • and then do it, by selecting a programming language and start coding

Without thinking very carefully about the first two parts , what and how the program will work, and writing down these into a precise format, what we call a specification, nobody can answer what means for the program to operate correctly. We call these the two main parts, the programming part.

Programming means working on a high level aspect of your program, at the algorithm level rather than the code level.

Coding is the last part of building the program, after the high level designs are in place. This requires to select a programming language and start the low-level part of the program, the implementation.

Scalability Analysis

Last but not least you should quantify the performance of your application. If you can’t quantify it, you will not be able to guarantee it. This means using a mathematical model, like Universal Scalability Law (USL) to validate your performance results and conduct a proper scalability analysis. 

There are two main benefits using USL for our program:

  • Validate our performance test results. Many companies are conducting lots of performance tests without making sure the results are also correct. The USL model helps us on this part
  • Project the application performance for different payloads
The USL model

There are different ways to use USL, from using Excel/OpenOffice, R or any other data numerical application.

At Kronometrix

Building software is not easy. With a small budget we managed to build a robust platform which can operate without human interventions for years, use less electricity vs. other similar solutions. Using USL, TLA+ have helped us to achieve these goals. Get started today with our free Kronometrix data analytics platform. If you have any questions or what to know more about our products and services please contact us!

Kronometrix Data Recording Update
Kronometrix Data Recording Update 1864 1424 KRONOMETRIX

Kronometrix Data Recording
version 1.20

December 2022

Kronometrix Data Recording is an automatic data collection package for ICT enterprise, industrial IoT, weather and environmental devices. To simplify how to get started we have created several packages, free of charge, which can be used to capture and transport data for analysis.

Check out our free Kronometrix
data recording packages

Select your application to get started

Version 1.20

We are working to release a new update for Kronometrix Data Recording which will include the following new features:

sockrec – The network IO Socket Data Recorder Improvements

New features and capabilities to receive data from various sensors and devices, like Vaisala WXT weather transmitters:

  • capability to filter incoming data using a JSON configuration
  • discard invalid or bogus not allowed data
  • reprocess data if there are transmission errors
Raspberry Pi fixes
  • inventory data fixes: number of cores and the processor type
  • corrections to fix invalid device name for certain USB disks, microSD cards
Storage data recorder bug fixes
  • fixes for CloudLinux operating system
Support for new IAQ sensors
  • Siemens QPA2052
  • Honeywell C7355A, C7355B
  • Schneider Electric SLXRHS1N
Security Updates
  • Expat 2.5.0 update
  • CPAN Dec 2022 update
  • OpenSSL 1.1.1s has been already fixed in 1.19.1 November 2022 release

Download you package based on your configuration and industry, from operating systems to industrial IoT installations. For 24×7 contracts you will automatically receive 1.20 after release.

Get started today with our free Kronometrix data recording packages! If you need more packages, options and support check our business catalogue or contact us!

Happy Winter!

Capture the right data with Kronometrix Data Recording
Capture the right data with Kronometrix Data Recording 2578 1906 KRONOMETRIX

Ready packages to capture the right performance metrics from your systems and applications

Kronometrix Data Recording is an automatic data collection package for IoT, enterprise ICT, web applications, weather and environmental devices. Available as an open source project, for many computer systems, the data recorders can be used to capture the right performance metrics from different data sources. Learn more about Kronometrix data recorders capabilities below.

Kronometrix Data Recording Capabilities

To simplify how to get started we have created several packages, free of charge, which can be used to capture and transport data for analysis. These packages are offered without support or any other service level agreements.

Check out our free Kronometrix packages

Need support and SLA? Visit our business catalogue to get more packages.

How does it work?

There are different ready made packages for several computer systems or applications. Each package includes all necessarily modules to start and capture data:

  • binaries and data recorders
  • the transport utilities
  • main configurations
  • manual pages and documentation

Select your package, download and install it for your system. After installation, you need to configure it to get your data. Each package includes a short description how to get started.

There are different package, for different industries and applications.

Download you package based on your configuration and industry, from operating systems to industrial IoT installations. Note: we are currently working to improve some of these packages, soon to be available for download.

Requirements

Check the prerequisites section, of each package to see what are the minimum settings to get started.

Installation

Follow the short installation notes how to install your package. For example this is how you will need to install the Raspberry Pi package:

# dpkg -i kdr-stable-raspbian-armv8.deb
The software is found under /opt/kronometrix directory. Current raw data is saved under log directory. The configuration files are found under etc directory. 

See your data

After installation, you have access to the data recording package, including the recorders and the captured raw data from your systems or applications. To visualize, analyze and consolidate all your data you can subscribe to Kronometrix SaaS or directly use the raw data with your own data analytics and visualization applications.

Get started today with our free Kronometrix packages! If you need more packages, options and support check our business catalogue!

Prevent Silent Data Corruption
Prevent Silent Data Corruption 1584 1224 KRONOMETRIX

Storage capacities have increased substantially, but their error rates remain unchanged and high

Companies are storing and placing their data on different networks, like cloud storage providers or private, on-premises systems and devices totally unaware of the real danger of silent data corruption – “undetected data corruption, also known as silent data corruption, results in the most dangerous errors as there is no indication that the data is incorrect.”

In the past, using older storage technologies, the probability of data corruption was very small because of the volume of data stored. Today, we store massively more data, from audio, video, numerical data, to documents using private or public social networks, without any ideas how safe our data is.

There can be many error sources beyond the disk storage itself, from loose cables, unreliable power supplies to DMA parity errors, external vibrations or other soft errors. Silent data corruption is difficult to detect and report, usually resulting in “cascading failures, in which the system may run for a period of time with undetected initial error causing increasingly more problems until it is ultimately detected”.

Jeff Bonwick, the technical leader of ZFS stated that, “The fast database at Greenplum, which is a database software company specializing in large-scale data warehousing and analytics, faces silent corruption every 15 minutes. As another example, a real-life study performed by NetApp on more than 1.5 million HDDs over 41 months found more than nearly half a million silent data corruptions, out of which more than 30,000 were not detected by the hardware RAID controller. Another study, performed by CERN over six months and involving about 97 petabytes of data, found that about 128 megabytes of data became permanently corrupted.” (Wikipedia)

Kronometrix Data Integrity

Developed with data integrity in mind, Kronometrix is using OpenZFS technology to detect, correct and prevent silent data corruption errors, supporting:

  • long-term data storage: large datastore sizes with zero data loss
  • data integrity: using hierarchical checksum of all data and metadata
  • native data compression: supports archival years of raw data
Intelligent Fabric

Kronometrix connects to a wide variety of data sources: everything from IoT devices, ICT enterprise to weather and environment sensors. In addition to multifaceted data ingress, the distributed data fabric provides high-speed transport for data consolidation, analysis and visualization in real-time.

Built as a ready solution, Kronometrix contains all needed software components, from data collection, to analysis and visualization, on top of core computing capabilities:

  • multi-processor configurations
  • in-memory processing, supporting RAM or flash memory
  • integrated storage and network capabilities

Please visit Kronometrix Data Fabric for more informations about key features and capabilities. If you have any questions please contact us.

Kronometrix CloudView Managed Service
Kronometrix CloudView Managed Service 17280 11520 KRONOMETRIX

Find out if your applications and services are properly configured and optimised for your business and customers

A lot of organizations are spending considerable time and energy, to detect and correct improper service configurations, which have a direct impact to the main operation and functionality, as poor performance or outages. These activities, many times, continue as a fire-fighting exercise, without a strategy and direction, exhausting everybody and getting very expensive.

Kronometrix CloudView brings all necessarily tools and utilities to build and define a direction, to simplify all these and get the job done.

For example, when we deployed Kronometrix CloudView for Nexetic, one of our customers, we have received several alarms from various parts of the applications and systems. Kronometrix uses a special algorithm, based on sampling the prescribed frequency and duration of trigger events, to eliminate the typical flood of unnecessarily repetitive alarms. Having a relatively high number of alarms was an indication for us, that certain things were not properly configured or correctly set to run.

Using the alarm list, we were able to navigate and detect the most important parts of the system and services which required immediate attention. Working with the devops team, we were able to take one issue at the time, perform changes to immediately improve the performance and ultimately the availability of the service.

How it works

To help you run your business smoothly in the cloud, we continuously monitor your applications, services, and operating system instances: no matter whether it be public, private or a hybrid cloud nor which vendor you have selected to provide the infrastructure. Moreover, when things go wrong or become inefficient, we let you know what the problem is and how best to fix it.Kronometrix Cloudview is an enterprise service, built on top of Kronometrix SaaS designed to track your cloud infrastructure usage and keep you aware of changes before they happen. It includes:

  • Real-time performance analysis of your cloud instances,  operating systems, storage and enterprise services
  • End-user performance and availability of web applications and websites, Internet services
  • Validity of X.509 security certificates 
  • A complete managed services support package, 8×5 or 24×7 for continuous service operations, including incident management and root-cause analysis

The service includes a managed service package, which will allow the Kronometrix Operations team to access and remotely operate your services. Our operators are highly skilled professionals with over 25 years of UNIX/Linux experience. Kronometrix will measure and analyse all your services. In this way we will help you to stop fire-fighting and focus on whatever matters most to your business, by keeping all your critical services running.

If you have any questions or need more information about our service,  please visit Kronometrix CloudView, check out CloudView FAQ or contact us.

SNMP Data Recorder
SNMP Data Recorder 1243 880 KRONOMETRIX

It is simple now to capture SNMP data!

Kronometrix Data Recording can work with various systems, applications, or devices using one or many data communication protocols, like MODBUS, HTTP, TCP or UDP. During this autumn, we have added support for SNMP devices, implementing a new data recorder, called snmprec. Based on a modern and scalable interface, the new recorder is designed to capture the right data from various devices, like:

  • Network equipment: switches, routers, firewalls
  • Storage devices: NAS and SAN
  • Utilities: HVAC, UPS, power metering equipment

The recorder does not require extra libraries or additional software to run. It has all needed components to connect and capture data from different devices or systems. Part of Kronometrix Data Recording, the recorder is available for various operating systems, or it is integrated to our IoT Gateway, for business installations.

How does it work ?

The snmprec data recorder is implemented as a CLI based application, easy to run interactively or continuously, for longer periods of time. This way it is simple to use for different activities from troubleshooting or debugging to performance analysis and capacity planning. The recorder, like any other Kronometrix data recorders, is offering immediate access to original raw data.

Configuration File

The recorder keeps all its settings, under a simple JSON configuration file. It is a text based file, simple to change and modify, using any text editor. The configuration supports any type of SNMP device and metric. You can add your own devices or use the built-in default supported systems.

Execution

You can easily start the recorder from Linux/UNIX terminal, like this:

What’s Next ?

We are working to improve the current functionality of SNMP data recorder, to support authentication, and different object types, like Counter32, Counter64, TimeTicks, as defined in RFC 2578. We are planning to create a number of default SNMP objects in Kronometrix, supporting printers, network equipment (firewalls, switches) and different other utility devices, like power metering, HVAC or UPS. Later, during Q1 2022 we will improve as well our data visualization capabilities, to fully support the SNMP data. If you want to stay up to date and have access to the latest development builds, please join our free development server.

Add your devices

If you work with different SNMP devices, we can help you to have them supported and ready for Kronometrix usage. Please let us know your equipment, and we will do the rest.

More information

Kronometrix Data Recording is an open-source software for ICT Enterprise, IoT, Weather and Environment industries. It supports a number of ready data recorders, which can be deployed on virtual or bare-metal servers, or ready systems like ARM single-board computers. Find out more about our data recorders or Kronometrix IoT Gateway. If you still have questions and need to ask more about our products and services don’t hesitate to contact us.

Kronometrix Data Recording Capabilities
Kronometrix Data Recording Capabilities 960 619 KRONOMETRIX

A complete solution to capture the right data for your business!

Kronometrix Data Fabric offers a complete solution to capture correct performance data from various systems, devices or applications, using the following options:

Data Recorders

A simple and efficient set of data recorders, built as an open-source software, which can help to capture data from computer systems, web applications, Internet or enterprise services.

How does it help?

Kronometrix data recorders can easily deploy to bare-metal or virtual computer systems, offering immediate access to the original raw data. The recorders are available as a software package for different operating systems, designed for:

  • software developers, ISVs and companies which require immediate access to their application’s performance data
  • IaaS or SaaS companies which are running their business on cloud, to help to capture and transport data from different providers for consolidation and analysis
  • banking and financial institutions, security agencies which require full transparency and control to the data recording process (to understand what parameters are being collected, where and how are stored)
Features

The data recorders have a small, configurable runtime system, without complicated licenses and rights to use. By default the data recording package includes many data recorders, designed to collect data from ICT enterprise to industrial equipment. Main features:

  • Support for raw data
  • Industrial IoT readiness
  • Service to Cloud communication
  • Conservative in CPU and memory usage on different architectures
  • Easy to change or add new data recorders to collect new data
  • Runs without human intervention
  • Easy to detect data transmission problems
  • Community Support

Please check here, for a complete list of the main features.

Getting started

Kronometrix data recorders are free, open-source software. You can download, modify or build your own products on top of it. To get support, join our free, unlimited development Discord server. If you still have questions or want to find out more about Kronometrix data recorders, please visit our project or contact us.

IoT Gateway

Kronometrix Iot Gateway is is a ready solution for enabling device -to-device or device-to-cloud communication, available as s a software image for single-board computer, like Raspberry Pi. The image contains all needed components, from operating system to data recorders and transport utilities.

How does it help?

Kronometrix IoT Gateway is ready to deploy, closer to sensors and devices for edge and fog computing, offering support for:

  • IoT solution providers, to capture various data and simplify the transport to cloud functionality
  • software developers, ISVs and companies which work with various sensors and devices and require to connect, capture and process real-time data for analysis
  • building facility owners to easy the process of capturing various smart building data, like indoor air quality, ventilation or energy consumption
  • different solution providers, to enhance and bring new functionality to their own product and services, by offering data recording and transport capabilities
Features
  • Real-time data collection from various devices
  • Support for ARM64 SBC like Raspberry PI 3B/3B+
  • Device to device communication
  • Device to cloud communication
  • Data caching, buffering and transport capabilities
  • Rich connectivity: USB, MODBUS, SNMP, SERIAL
  • Data filtering capabilities
  • Support for original, raw data
  • Remote management options
  • Support for legacy or non-internet connected systems
  • Commercial support 8×5 or 24×7 including SLA
Get access

Kronometrix IoT Gateway is a commercial product, available to purchase for single-board computers, like Raspberry PI, including commercial support and different SLA options.

More information

Find out more about our data recorders or Kronometrix IoT Gateway. If you have any questions please don’t hesitate to contact us.

Avoid Certificate Expiration
Avoid Certificate Expiration 960 641 KRONOMETRIX

Find out how Kronometrix can keep track of your security certificate settings and prevent a data breach

Are your services and applications properly configured and secure for your business and customers? When more and more companies and services are starting to automate the renewal process of the security certificates, using providers like, Let’s Encrypt it is more important than ever to ensure your X.509 security certificates are properly configured and installed.

The attackers pulled data out of the network in encrypted form undetected for months because Equifax had crucially failed to renew an encryption certificate on one of their internal security tools.

March 2017. Equifax data breach

“The attackers were able to gain access to multiple Equifax databases containing information on hundreds of millions of people. But how were they able to remove all that data without being noticed?

Like many cyberthieves, Equifax’s attackers encrypted the data they were moving in order to make it harder for admins to spot; like many large enterprises, Equifax had tools that decrypted, analyzed, and then re-encrypted internal network traffic, specifically to sniff out data exfiltration events like this. But in order to re-encrypt that traffic, these tools need a public-key certificate, which is purchased from third parties and must be annually renewed. Equifax had failed to renew one of their certificates nearly 10 months previously — which meant that encrypted traffic wasn’t being inspected.

The expired certificate wasn’t discovered and renewed until July 29, 2019, at which point Equifax administrators almost immediately began noticing all that previously obfuscated suspicious activity; this was when Equifax first knew about the breach.”  http://bit.ly/equifax-securitybreach

A simple example, how such large organization, like Equifax failed to notice  and maintain application basic security configuration – X.509 certificate expiration, something no one was really paying attention to, which allowed the attackers to penetrate even deeper in the infrastructure, without being noticed for months.

How Kronometrix helps

To help you run your business securely in the cloud or private network, Kronometrix continuously monitors the validity of X.509 security certificates of all your applications or services. When one of your security certificate approaches the expiration date, an alarm will be issued to notify your operation personnel.

Kronometrix checks several metrics for each X.509 security certificate to ensure the certificate is valid, that it is properly installed and that it is operational. Expired, missing or unavailable data from a X.509 security certificate will automatically raise an alarm. All information is displayed visually using a Kronometrix application, called CertCheck, which offers access to all your certificates, including the following information:

  • total number of X.509 certificates
  • certificates already expired
  • certificates which will expire in the next 30 days
  • valid number of certificates

In this way, Kronometrix will automatically keep track of your X.509 security certificates, across all your services and applications without supplementary manual labour work, additional software licenses or 3rd parties applications.

How it works

From data recording to visualization and reporting, Kronometrix provides a complete unified solution to ensure all your enterprise security certificates are never out of date. A data recorder, called certrec, which is part of Kronometrix Data Recording will collect from all your services and applications the X.509 certificate data. A transport utility will convert raw data, into Kronometrix data messages and will deliver the results to a backend system for analysis and alarms.

This is a standard feature of Kronometrix, available as a service or for on-premises installations. If you have any questions or need more information about this, please visit Kronometrix Distributed Data Fabric, or contact us.

Ecological Revitalization
Ecological Revitalization 1991 987 KRONOMETRIX

See how Pikanore is using Kronometrix to build sustainable solutions for a clean environment

Weather and environment are essential core components of Kronometrix data fabric. From data messages, summary statistics to UI dashboards and raw data, Kronometrix provides a complete solution to capture, analyze and visualize real-time data, regarding:

  • present weather conditions
  • city air pollution
  • building indoor air quality

Ecological Revitalization

Pikanore Ltd. has selected Kronometrix Analytics to provide continuous weather and environment measurements from their workplaces.

Revitalizing properties for ecological purposes refers to the process of cleaning-up and returning contaminated land to a state that supports functioning and sustainable habitat. This is a very important mission to protect human health and the environment. 

Different technical activities, like excavation and earthmoving equipment can significantly disrupt existing habitat during cleanup. Measuring the impact of these activities is essential to ensure normal functioning of the clean-up process and alignment with current rules and environment regulations.

How does it help?

Kronometrix collects data from different weather equipments, reporting current weather conditions around different workplaces: wind speed and direction, air temperature, air pressure, humidity, precipitation accumulation and offers access to the original raw data to build different other weather reports, if required. Having access to original historical raw data, offers maximum flexibility to build and analyze any weather conditions using our own in house applications. Kronometrix helps us to consolidate and integrate different data from weather to air quality in a simple, and efficient way.

Such way, Kronometrix is helping Pikanore to ensure all their workplaces comply with the current environmental regulations and their clean-up operations do not harm the environment.

More information

Please visit Kronometrix customers to learn more about some of our customers and their success stories, or contact us.

Kronometrix IoT Gateway
Kronometrix IoT Gateway 900 629 KRONOMETRIX

Kronometrix IoT Gateway is a simple to use, automatic data collection and transport utility with a small system footprint that supports raw data collection from Industrial IoT, enterprise ICT, web applications, weather and environmental devices. The system, contains all needed software and hardware components:

  • Single Board Computer
  • the operating system
  • the data recording package
  • and all configurations

A MicroSD card, includes all software modules and packages, available from www.kronometrix.com. Running on top of Raspberry PI SBC, Kronometrix IoT Gateway (K1 Internal Technical Name) is very simple to use and run or embed to other different projects.

How does it work ?

You can connect Kronometrix IoT Gateway to your network and get access to its main features using a screen and a keyboard, or remotely using SSH2 protocol or a serial connection. You can select what data recorders you want to run, using the main configuration file kronometrix.json, from ICT enterprise to IoT:

  • measuring the performance and availability of several web sites or applications using HTTP/HTTPS protocol
  • fetching data from different Internet & enterprise services, like: IMAP, POP3, LDAP, SMTP, HTTP, MariaDB, Memcached, Redis, Isilion storage or any TCP based service
  • checking the performance and availability of different UDP services, like NTP, DHCP
  • measure indoor or city air pollution using different industrial MODBUS sensors and devices
  • fetch data from various weather equipment using USB or direct SERIAL connections

A future version will offer a simple web management interface to simplify even further the management and usage of Kronometrix IoT gateway.

Recorders

A central part of our IoT gateway are the data recorders. Recorders are not software agent(s), nor they should be confused with them. They are independent, light software probes designed as CLI to test and fetch certain minimal data points from different parts of a system(s), more or less same way how a sensor operates. The recorders share a common runtime engine, powered by Perl5. The recorders do not know nor have any knowledge of any transport layer. The output of each data recorder is a simple CSV based raw data file, stored locally on the gateway for a number of days. For more information you can read here. By default the sampling rate of a data recorder is 60 seconds, which is configurable.

Transport

The original raw data can be transported for further analysis and visualisation to Kronometrix SaaS, or any other Kronometrix installation or even to 3rd parties analytics or business intelligence platforms. The transport utility keeps tracks of any raw data changes and periodically sends the update to one or many backend destinations, using HTTP or HTTPS protocol. The transport utility uses a JSON based configuration file.

The data recorders and transport utilities are built together as a binary package for several operating systems. For systems or installations which do not require a complete IoT gateway there is always a possibility to manually install the data recording package.

Raw data

All recorded observations are stored as raw data. Raw data is produced by a recorder, which fetches data from a system, device or sensor, data which has not been modified, altered or changed in any way. All collected metrics are variable measured sequentially in time, called time series. All these observations collected over fixed sampling intervals create a historical time series. To easy the access to all this set of data we store the observations on commodity disk drives, compressed, in text format, like CSV format.

Time series let us understand what has happened in past and look in the future, using various statistical models.

What’s New ?

There are new features and improvements available on the latest version of Kronometrix IoT Gateway 1.4. Some of the very important features:

  • Computer system data recorders, sysrec, diskrec, nicrec have been updated to support latest CentOS and Debian Linux operating systems
  • NTP data recorder, ntprec, improvements and fixes when fetching  data from one or many NTP servers
  • IoT data recorder, rs485rec: support for data correction, error handling fixes and better logging features
  • Internet and enterprise data recorder, svcrec, performance improvements when handling thousands TCP or UDP services. We have added support to allow custom UDP probes, if necessarily.
  • Software stack updates
  • Security updates

Home
Data Analytics Platform
Industries
Knowledge Base
API
News
About

Kronometrix Oy (SDR Dynamics Oy, 2608375-6)
Upseerinkatu 1
02600 Espoo,
Finland

support@kronometrix.com
sales@kronometrix.com
+358 10 286 1946

Facebook
LinkedIn
YouTube

© 2023 Kronometrix Oy. All rights reserved.

Terms of Service   Privacy Policy

Get in touch

If you have any questions about our products and services, please contact us.