A detailed HBase and Spring Boot implementation for weather time-series storage, row-key design and low-latency historical retrieval.
52p
system report
02
column families
2s
retrieval target
01 / Overview
What this package actually contains.
This project connected distributed storage, batch processing, time-series serving and forecasting in one weather analytics stack. The Hadoop and HBase reports show two different but complementary parts of the system: batch MapReduce analytics and low-latency historical retrieval.
Objectives
01Design a row key that matches station/date/hour retrieval patterns.
02Separate stable metadata from measurement columns.
03Wrap HBase access in Spring Boot query and dashboard services.
Data flow
From raw records to a usable service.
The pipeline is the main story: ingestion, storage, processing and serving.
01→
Weather CSV
Station, timestamp and nineteen measurement fields.
02→
HDFS + MapReduce
Cleaning, aggregation and batch transformations.
03→
HBase
stationId#date#hour row key with info and measurement families.
04→
Spring Boot
Query, range and trend endpoints with server-rendered views.
05
Forecasting
Monthly SARIMA forecasts and Chart.js dashboards.
02 / Package highlights
What this work demonstrates.
Designs the stationId#date#hour key for prefix scans and historical look-back.
Separates info and measurement column families for clearer time-series access.
Connects HBase storage with ZooKeeper, Maven, Spring Boot and a browser-based interface.
03 / Implementation
Where the engineering complexity sits.
01
Query-first schema design
The table design began with the access pattern. A station’s observations are commonly requested as a time range, so the station identifier became the row-key prefix. Date and hour were appended in sortable form, enabling prefix scans and range scans without a secondary index.
info family stores station/date/hour context.
measurements family stores precipitation, temperature, humidity, pressure and wind fields.
Scanner caching and batched puts control RPC overhead.
02
Service layer
Spring Boot services translated HTTP queries into HBase scans and returned typed weather records to Thymeleaf pages and Chart.js views. The implementation includes station search, date-range filtering, annual trend aggregation and cluster status screens.
04 / Key code
Code that defines the pipeline.
Only verified source excerpts and pseudocode taken from the project papers are shown.
Verified project codeJava
HBase write and prefix retrieval
Original Java excerpts showing the time-series row key and station-prefix scan.
WeatherDataImporter.java + WeatherService.java
String rowKey = stationId + "#" + date + "#" + hour;
Put put = new Put(Bytes.toBytes(rowKey));
put.addColumn(
Bytes.toBytes("info"),
Bytes.toBytes("station_id"),
Bytes.toBytes(stationId)
);
put.addColumn(
Bytes.toBytes("measurements"),
Bytes.toBytes("max_temp"),
Bytes.toBytes(fields[6].trim())
);
// Historical look-back for one station
Scan scan = new Scan();
scan.setFilter(
new PrefixFilter(Bytes.toBytes(cityCode + "#"))
);
scan.addFamily(Bytes.toBytes("info"));
scan.addFamily(Bytes.toBytes("measurements"));
Verified project codeJava
Prefix scan for station history
Original Java excerpt from the HBase weather service.
WeatherService.java
Scan scan = new Scan();
scan.setFilter(
new PrefixFilter(Bytes.toBytes(cityCode + "#"))
);
scan.addFamily(Bytes.toBytes("info"));
scan.addFamily(Bytes.toBytes("measurements"));
try (ResultScanner scanner = table.getScanner(scan)) {
for (Result result : scanner) {
String[] keyParts =
Bytes.toString(result.getRow()).split("#");
String stationId = keyParts[0];
String date = keyParts[1];
String hour = keyParts[2];
// Map column values into a Weather DTO
weatherRows.add(mapWeather(result, stationId, date, hour));
}
}
Operational notes
The stack was built with Docker Compose and ZooKeeper so the distributed components could be started consistently.
The final implementation combined Hadoop batch results with HBase retrieval rather than forcing one storage engine to do both jobs.
Visualisation was kept at the serving layer through Spring Boot and Chart.js.
Limitations / next steps
The current cluster and dataset are coursework-scale; production tuning would require larger partitions and monitoring.
A cloud-native deployment would benefit from object storage, managed HBase compatibility and stronger observability.
05 / Downloads
Files in this folder.
Open the original report, presentation or source archive. PDF cover pages with private details were removed.
PDF
HBase system report
中文 · 52 pages · 8.0 MB
Sanitized 52-page report on architecture, storage design and implementation.
