KEYWORDS: Sensors, Environmental sensing, Data storage, Data analysis, Environmental monitoring, Data visualization, Data processing, Data acquisition, Visualization, Humidity
Environmental sensors play a crucial role in modern society. They provide strong support for environmental protection, resource management, and sustainable development by monitoring and perceiving various parameters in real-time. With the development of big data and artificial intelligence technologies, intelligent management of environmental data has become possible. To enhance the level of intelligence in environmental monitoring, this paper studies the performance of environmental sensor data operations in data warehouses such as Hive, ClickHouse, and Doris, and constructs an environmental sensor data analysis system based on a real-time data warehouse. The system employs Flink for real-time collection of environmental data, ClickHouse for data storage, Spark for data analysis, and front-end and back-end technologies for data visualization. The system achieves automatic collection, processing, and analysis of environmental data, thereby improving the efficiency and accuracy of environmental monitoring.
KEYWORDS: Artificial intelligence, Data storage, Data processing, Data acquisition, Data analysis, Databases, Data transmission, Statistical analysis, Data visualization
With the continuous development of water transportation in China, ships entering and leaving the ports have become increasingly significantly. The Automatic Identification System (AIS) provides powerful data support for port management by transmitting critical information such as ship location, heading, and speed in real-time. Based on data warehouse technology, a port ship management platform is constructed, and it uses AIS data analysis to achieve dynamic monitoring for ship trajectories and operations. The platform employs a refined layered design that encompasses key aspects such as data acquisition, processing, storage, and visualization, optimizing the data processing workflow and significantly enhancing the stability and flexibility of the platform. It utilizes message queues for real-time data acquisition and performs immediate processing through stream computing. The processed data is intelligently stored in corresponding data storage services based on its granularity. With a front-end and back-end separation design approach, the platform presents data to users in an intuitive and clear graphical format. The analysis of AIS data not only facilitates the efficient management of ship arrivals and operations in ports, but also contributes to the optimization of berth allocation, channel management, and personnel scheduling, ultimately enhancing the overall efficiency of port management.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.