Posts

MAP Rain model infographic

MAP Rain model infographic shows some of the analytics solutions built using the MAP Rain model

MAP Rain is a generic real-time model built using MAP for processing, primarily, radar and forecast rainfall data. The model delivers all the underlying process of fusing the historic, current, Hyperlocal (our own 1-hour forecast) and forecast rainfall data together for any area, resolution, or size of cutout (the area over which we process the rainfall data). Built into MAP Rain are lots of methods to re-sample, aggregate, extract and re-process the rainfall data. These ensure we have the flexibility to apply rainfall data to any point or polygon in the overall cutout area. All these methods are available as API calls.

We use MAP Rain to build a broad range of applications that have rainfall data at their heart. The number of such applications continues to rise and we can deliver all these applications to multiple customers on one instance of MAP.

MAP Rain Infographic

More information on MAP Rain

MAP Solar infographic

Overview of the solar model built with MAP to predict solar irradiance


The objective of the project was to provide a way to optimise the integration of battery storage and PV solar power. By predicting solar irradiance, and hence solar power from PV installations, gives solar plant operators the ability to maximise revenues by charging and discharging batteries at the optimum time.

The project integrates near real-time satellite imagery, AI and an external Python solar calculation library with the core MAP framework to deliver an innovative and accurate way to predict solar irradiance data.

MAP Solar Infographic

More information on MAP solar

Key Partners in the InnovateUK funded collaborative project
Open Energi
National Solar Centre
Cornwall Council

MAP IoT Entity infographic

Overview of the MAP IoT Entity model used to build IoT applications

MAP IoT Entity model gives developers the ability to create their own Entities or Things in MAP. It makes use of the core MAP framework and integrated analytics stack to hide all the complexity of creating calculations, associations, alerts. More importantly, it does this at scale and in near real-time. This allows developers to focus on creating the underlying algorithms they want and in developing their dashboards/user interfaces. The MAP IoT Entity model takes a lot of the principles of the original low-code MCE Calculation Engine and applies them at scale – but you will need to be a developer to use MAP!

MAP IOT Infographic

More information on the MAP IoT model

Using Data Blocks and Data Versioning to deliver real time analytics

Introduction

This is part of a series of articles where we describe the way the Meniscus Analytics Platform (MAP) works. Theses articles jump into the features that make MAP different to other analytics applications by providing an Integrated Analytics Stack delivering real time analytics. In this article we discuss Data Blocks and Data Versioning.

In delivering real time analytics, disk IOPS (Input/output Operations Per Second) is one of the main rate limiting steps in achieving the calculation speeds required when processing high volume and high velocity raw data. An example of such a data is radar rainfall data where new values covering a large area arrive every 5 minutes.

To help reduce disk IOPS, we developed the concepts of Data Blocks and Data Versioning into MAP to drastically speed up data access, increase calculation speed and reduce the volume of data written back to the database.

Data Blocks

Rather than loading and persisting all data for an Item, data can be broken up into chunks called Blocks. So, only the chunks of data that are demanded for a query, or as an input to a calculation, are loaded from the database (i.e. delay loading), and only the chunks of data that actually change need to be persisted. Blocks are typically used with unbounded, time-related data such as sample arrays, where the size of a Block is limited and the maximum number of Block samples depends on the size of a sample. This provides efficiencies in real-time processing, whereby data changes are localised and typically at the end of the data.

Data Blocks are transparent to the user. It is purely an internal mechanism to reduce traffic to/from the database. When requested or persisted, Data Blocks are held in memory for a time. This ensures future retrieval is temporarily faster as the data is expected to be in demand.

Data Versioning

Data Blocks are complimented by the MAP concept of Data Versioning. All Item data in MAP is versioned, including Blocks (as such referred to as child data). A version is simply a unique timestamp. It allows users to query for the relative age of data. Specifically, when it last changed, and for calculated Items when the last calculation started and completed. A client application can then tell if data has changed without having to load the data itself. There are additional non-data versions on an Item. I.E when its properties or list of child items last changed.

It is this versioning technique that allows MAP to efficiently detect when calculated items need recalculating (referred to as dirtying as calculation).

About MAP

MAP is an Integrated Analytics Stack providing a framework for users to create and deploy calculations at scale using any source of raw data. MAP is based on IOT principles and uses Items as the underlying building blocks to store either RAW or CALCulated data. So, users create an Entity Template or Thing using these Items and then replicate this template hundreds of thousands of times using an ItemFactory.

For more information on MAP then click here

Meniscus Launches Analytics Platform “MAP” on FATHOM Store

Platform turns disparate data into real-time information for the water industry

Huntingdon, Cambridgeshire – June 20, 2016 — Meniscus, an analytics software company that provides high-performing, flexible and scalable cloud-based tools for the development of bespoke applications, today announced a new partnership with FATHOM Water Management, Inc. (“FATHOM”). Meniscus will offer its proprietary analytics platform MAP through the FATHOM Store, an online marketplace designed to enable simple, fast and risk-free adoption of new technologies for water utilities.

With MAP, customers can:

  • Monitor chemical costs, use, and optimization
  • Monitor process based energy costs, use, and optimization
  • Integrate real-time rainfall data into the organization

Mike Everest, managing director of Meniscus, commented, “As a cloud-based service delivering process-based energy, chemical, and weather based analytics services, working with FATHOM is key for us. First, it provides access to data stored on the FATHOM cloud servers, which simplifies the delivery of our services, and secondly, it provides exposure to many water companies in the U.S. We look forward to the opening of the FATHOM Store to help accelerate the adoption of our platform by water utilities across the country.”

The FATHOM Store is a simple, secure data integration platform that connects water utilities with technologies that help streamline business processes, recover revenue and better engage with customers. FATHOM’s expert team thoroughly vets each product submitted for consideration to the Store and selects only the top-performing tools like MAP for inclusion. Through open application program interfaces (APIs), technology companies can exchange data with FATHOM, as well as utilities, to deliver maximum value.

Utilities can now easily access and acquire MAP without the risks common to technology adoption.

“Nationwide, more than 56,000 water systems are searching for technologies that can help them make the most efficient use of ever-dwindling resources. Unfortunately, finding, vetting and integrating new technologies demands a level of investment most cannot afford to make – until now,” said Trevor Hill, CEO of FATHOM. “Through the FATHOM Store, we are connecting technology innovators, like Meniscus, directly to the utilities that need their support most. By mitigating the time and financial commitment required to modernize water utilities’ processes, we are propelling the Smart Grid for Water into reality.”

The FATHOM Store is available at https://store.gwfathom.com.

About Meniscus

The vision behind Meniscus of turning raw data into valuable information was the idea of Managing Director Mike Everest.  Meniscus ensures use of all the data collected to identify problems early and to help businesses operate more efficiently and at lower cost, and so the idea of automatically calculating performance metrics from operational data was born. Meniscus is focused on adding value to existing data and delivering data analytics to the connected world.  Visit www.meniscus.co.uk for more information.

About FATHOM

FATHOM is a software-as-a-service company helping water utilities do more with declining resources. With rapidly deployable, risk-free, cloud-based solutions that address all aspects of the meter-to-cash verticals for water utilities, FATHOM increases revenue, decreases costs and delights customers. Water utilities trying to simplify the adoption of technology can use the FATHOM Store as a marketplace to acquire and access new technology. FATHOM was built by a water utility, for water utilities and delivers a risk-free solution to more than 160 community water systems across the United States. There is strength in numbers. To learn more, visit www.gwfathom.com