Data Systems and Preprocessing
Data systems are computerized systems that store educators, students and school data and permit users to access information, manage and analyze data. These systems are referred to by various names, including student information system (SIS), learning management system decision support system and data warehouse.
The goal of design of a data system is to improve the manner that information within an organization is collected to be stored, retrieved, and analysed. It involves determining the most efficient mechanisms for storage and retrieval, designing schemas and data models, and implementing robust security measures. Data system design also involves identifying the most effective tools and technologies to use for processing, storing and delivering data.
Big sensor data systems are based on a collection different data sources, including wireless and mobile devices, as well as wearables, telecommunications networks, and public databases. Each of these sources provides the same set of sensor readings, each with their specific metric value. The primary challenge is to find a resolution that is suitable for the data, and an algorithm for aggregation that lets the sensor data be represented in a single form with the same metric.
For a successful data analysis it is vital to ensure that the data can be understood correctly. Preprocessing includes all activities that prepare data for analysis and transformations such as formatting as well as combination and replication. Preprocessing can be batch-based, or stream-based.
