Preprint
Machine Learning

Modeling temporal aspects of sensor data for MongoDB NoSQL database

Nadeem Mehmood(Università di Camerino), Rosario Culmone(Università di Camerino), Leonardo Mostarda(Università di Camerino)
March 31, 2017Journal Of Big Data45 citations

45

Citations

1

Influential Citations

Journal Of Big Data

Venue

2017

Year

Abstract

Proliferation of structural, semi-structural and no-structural data, has challenged the scalability, flexibility and processability of the traditional relational database management systems (RDBMS). The next generation systems demand horizontal scaling by distributing data over autonomously addable nodes to a running system. For schema flexibility, they also want to process and store different data formats along the sequence factor in the data. NoSQL approaches are solutions to these, hence big data solutions are vital nowadays. But in monitoring scenarios sensors transmit the data continuously over certain intervals of time and temporal factor is the main property of the data. Therefore the key research aspect is to investigate schema flexibility and temporal data integration aspects together. We need to know that: what data modelling should we adopt for a data driven real-time scenario; that we could store the data effectively and evolve the schema accordingly during data integration in NoSQL environments without losing big data advantages. In this paper we explain a middleware based schema model to support the temporal oriented storage of real-time data of ANT+ sensors as hierarchical documents. We explain how to adopt a schema for the data integration by using an algorithm based approach for flexible evolution of the model for a document oriented database, i.e, MongoDB. The proposed model is logical, compact for storage and evolves seamlessly upon new data integration.

Analysis

Why This Paper Matters

The proliferation of heterogeneous data from IoT sensors has exposed the limitations of traditional relational databases, particularly in handling temporal data with evolving schemas. This paper tackles a practical pain point for AI practitioners working with real-time sensor streams: how to store and manage data that changes structure over time without sacrificing the scalability benefits of NoSQL systems. By focusing on MongoDB—a widely used document database—the work provides a concrete blueprint for integrating temporal sensor data in a way that is both flexible and efficient.

The significance lies in bridging the gap between schema-on-write (relational) and schema-on-read (NoSQL) paradigms. The proposed middleware approach allows developers to maintain logical consistency while adapting to new data sources, which is essential for AI pipelines that ingest streaming data from diverse sensors. This is particularly relevant for applications in smart cities, industrial IoT, and environmental monitoring where data formats evolve rapidly.

Technical Contributions

  • Middleware-based schema model: Introduces an intermediate layer that converts raw ANT+ sensor data into hierarchical MongoDB documents, preserving temporal ordering and enabling efficient querying.
  • Algorithm-driven schema evolution: A novel algorithm that automatically adjusts the document structure when new sensor types or data fields are added, without requiring manual schema migrations.
  • Compact storage design: The hierarchical document structure reduces redundancy and storage overhead compared to flat or relational representations, though no specific compression ratios are given.
  • Temporal data integration: Explicitly models time as a first-class property, ensuring that temporal queries (e.g., time-range aggregations) remain performant in a document-oriented database.

Results

The abstract does not include quantitative results such as storage savings, query latency improvements, or throughput comparisons. The authors claim the model is "logical, compact for storage" and "evolves seamlessly," but no benchmarks or case study metrics are provided. This is a notable gap for practitioners evaluating the approach for production systems. Future work should include empirical evaluations against alternative NoSQL models (e.g., time-series databases like InfluxDB) and relational baselines.

Significance

This paper contributes to the growing body of work on schema-flexible data management for IoT and big data. For AI practitioners, the ability to dynamically evolve data schemas without downtime is critical for continuous learning systems that ingest new sensor types. The middleware approach could be extended to other document stores (e.g., Couchbase, Amazon DocumentDB) and integrated with stream processing frameworks like Apache Kafka or Flink. However, the lack of performance validation limits immediate adoption; the community would benefit from open-source implementations and benchmark results.