Office of
Materials and
Road Research
Minnesota Road
Research
MnROAD Project
Mn/DOT Home
FAQ
Topics A to Z
Search Mn/DOT
Info@DOT
MnROAD Data Collection
Mn/ROAD data are collected in a variety of ways.
Online sensors are instruments for which data are collected by
an automated ("online") process. A
network of fiber-optic and
copper wire connect sensors and computers which poll the instruments
on a regular basis and return data to the Maplewood site for
analysis.
Other data (considered offline) are collected on a periodic
basis through manual or automated processes. This includes weather
data, traffic data, falling weight deflectometer (FWD) data, and
others.
ONLINE Sensor Data Collection
The following manual and automated steps are used to collect this type of data:
- The TCS (Test Control Software) program is run on a laptop or
desktop computer, where a "test file" is generated. This test file
contains information such as what types of tests should be run
on what data channels for that megadac. It also contains
the transform equations which need to be applied to the raw
voltage data returned by the sensor in order to convert the
information to engineering units, and the identification "tag"
(cell, instrument type, and sensor sequence) to be attached to
each item of data.
Test files have the filename XSCCNNNN.LST, where X is one of the letters F, H, R, S, T, or X, S is either S (static), or D (dynamic), CC is the cabinet number, and NNNN is the test file number. For example, the first set of test files generated for dynamic cabinet (megadac) 11 are FD110000.LST, RD110000.LST, TD110000.LST, HD110000.LST, SD110000.LST, and XD110000.LST.
- The test files are given to the System Administrator
(on a floppy disk or directly placed on the computer
via the ftp program. A copy of the test
file is stored in the directory /tcs2/YYMMDD.
- The test files are copied from the server
to the appropriate local site protocol
converter (PC) using the ftp program, and placed in
the directory /u/mnroad.
- SOME STEP is taken to install the new test
- Polling software, started from the script /u/mnroad/start,
is run continuously on each of the five local site PCs.
- Static Sensors
Static sensors (sensors which take measurements only periodically) are controlled by the program statpoll.out. This program reads the configuration file /u/mnroad/statsystem.fvs, which contains a list of all static megadacs connected to that protocol converter, along with various parameters such as what test file should be used.
At the appointed time (every fifteen minutes), the program goes through the list of static megadacs configured, and sends the test through the appropriate port to the cabinet. It then waits for the test to run and reads the data returned from the megadac. The transforms are performed, converting raw voltages to engineering units, and the result is written to a file on the protocol converter /u/raw/SCCNNNN, where S stands for static, CC is the cabinet number, and NNNN is a sequence number (obtained from the file /u/mnroad/statsystem.fvs).
- Dynamic Sensors
Dynamic sensors (sensors which take measurements on a continuous basis, and are triggered by traffic) are controlled by the program dynpoll.out. This program reads the configuration file /u/mnroad/balsystem.fvs, which contains a list of all dynamic megadacs connected to that protocol converter, along with various parameters such as what test file should be used.
At the appointed time (every fifteen minutes), the program goes through the list of dynamic megadacs configured, and sends the test through the appropriate port to the cabinet. Instead of collecting one sample only (as the static sensors do), the megadac continuously scans (at a rate of 2000 per second) each channel returning peak and trough data to the protocol converter. The transforms are performed, converting raw voltages to engineering units, and the result is written to a file on the protocol converter /u/raw/DCCNNNN, where D stands for dynamic, CC is the cabinet number, and NNNN is a sequence number (obtained from the file /u/mnroad/balsystem.fvs).
dynpoll.out also makes copies of the values used in the dynamic balancing process (such as baseline, trigger level, noise, and average peak. The current values for these settings are stored in /u/mnroad/DCCsensors.dat, where CC is the cabinet number. (This file is read by and also updated by the dynpoll.out program). Copies of these values are periodically saved in the file /u/raw/DCCNNNN.aux, where D stands for dynamic, CC is the cabinet number, and NNNN is a sequence number.
- Static Sensors
- The program converter2, automatically runs from the local
site RISC (LSR) Unix computer every two hours, changes the file
format of the data. (NOTE: it does not change the actual values).
Data are read across a network file system (NFS) mount to LSR. Data are read from /pcXX/u/raw/SXXX, and /pcXX/u/raw/DXXX and written to /u/final/YY_Mon_DD_HH on the local hard drive. The dynamic balancing files /pcXX/u/raw/DXXX.aux are also converted and written to /u/final/YY_Mon_DD_HH.aux.
As each file is successfully converted, the raw files are moved to the directory /u/archive. (If an error occurs during conversion, the raw file remains in /pcXX/u/raw).
- The program to_toby.ksh, automatically runs from LSR every
hour on the half hour, moves the files from the local site to the
Unix computer toby. Sensor data Files, converted by converter2
and stored in /u/final, are moved to the directory
/toby/data, which is a NFS mount to /data on toby.
Raw data files found in /u/archive are combined into a
tape archive (tar) file, and this file is moved to the directory
/toby/u.tape, which is a NFS mount to /data_arch/u.tape>
on toby. Files are not removed from LSR unless they are
successfully moved to their desired location.
- The program mnrload, is run on a Windows NT Application Server
which looks for all loader files, which have been copied here,
located in /data,
and inserts the information into the NT Oracle database. From the
identification "tag" on each record, the appropriate data table
and the minimum and maximum expected values are determined. Based
on that information, the appropriate database table is used, for
example, thermocouple information (tag XXTCXXX) is placed
in the THERMOCOUPLE_VALUES or THERMOCOUPLE_ERRORS
table depending on the value for that particular item.
After each file is processed, it is moved to the directory /data/done, and compressed to minimize disk usage.
- The program loadaux is currently being run manually
on an "as-needed"
basis. This program reads dynamic sensor balancing information in the files
DXXXXXX.aux, and inserts the data into the database table
DYNAMIC_SENSOR_BALANCING. When sufficient Oracle database storage
is obtained, this information will be automatically put in the
database as part of the mnrload process.
OFFLINE Sensor Data Collection
- Weather Station Data
- Frost Pin Elevations
- FWD Data
- Open Standpipe Data
- Pavement Condition Data, Concrete Faults
- Pavement Condition Data, Pave Tech Van
- Pavement Condition Data, Rutting Measured with Dipstick
- Pavement Condition Data, Rutting Measured with Straight Edge
- Pavement Condition Data, Skid Machine Tests
- Rainfall Data
- Time Domain Reflectometer (TDR) Data
- Tipping Bucket Data
- Weigh-In-Motion (WIM) Data