Knorr IMET Data Quality Control Report

 

 

Knorr IMET Data Quality Control Report

 

 

 

 

Jennifer Lovell and Shawn R. Smith

 

 

 

 

Research Vessel Surface Meteorological Data Center

Center for Ocean Atmospheric Prediction Studies

Florida State University

 

December 20, 2000

Report 01-03

Version 1.0

 


 



 

1.0 INTRODUCTION:

This report summarizes the quality of surface meteorological data collected by the research vessel Knorr (identifier: KCEJ) IMET system during twelve cruises which occurred over a two year span, beginning 28 January 1996 and ending 24 March 1997. The data sets were provided to the Florida State University Research Vessel Surface Meteorological Data Center (RVSMDC) in electronic format by M. Lamont (WHOI) and were converted to standard RVSMDC netCDF format. The data were then processed using an automated screening program, which added quality control flags to the data, highlighting potential problems. Finally, the Data Quality Evaluator (DQE) reviewed the data and current flags, whereby flags were added, removed, or modified according to the judgment of the DQE and other RVSMDC personnel. Details of the RVSMDC quality control procedures can be found in Smith et al. (1996). The data quality control report summarizes the flags for the Knorr IMET surface meteorological data, including those added by both the preprocessor and the DQE.


 



 

2.0 DATA VARIABLES:

The Knorr IMET data are expected to include observations averaged every minute on these cruises. Values for the following variables were collected:

Time

(TIME)

Latitude

(LAT)

Longitude

(LON)

Platform Heading (Gyrocompass)

(PL_HD)

Platform Course

(PL_CRS)

Platform Speed Over Ground

(PL_SPD)

Platform Speed Over Water

(PL_SPD2)

Platform Relative Wind Direction (IMET)

(PL_WDIR)

Platform Relative Wind Speed (IMET)

(PL_WSPD)

Ocean Relative Wind Direction

(DIR)

Ocean Relative Wind Speed

(SPD)

Earth Relative Wind Direction (IMET)

(DIR2)

Earth Relative Wind Speed (IMET)

(SPD2)

Atmospheric Pressure

(P)

Air Temperature

(T)

Sea Temperature

(TS)

Relative Humidity

(RH)

Atmospheric Radiation

(RAD)

Precipitation

(PRECIP)


 



 

3.0 CRUISE IDENTIFIERS AND DATES:

Note: The Cruise Identifiers and Cruise Dates were assigned to the Knorr cruises by the DQE for cruise identification in the quality control report. The beginning and ending dates of each cruise were determined by the Knorr's departure and return dates to port successively.

                                                                                                                        Cruise Identifiers                                                                Cruise Dates

1996

96-A

96-B

96-C

96-D

96-E

96-F

96-G

96-H

96-I

01/28/96 – 01/30/96

02/09/96 – 03/11/96

03/13/96 – 03/26/96

04/03/96 – 05/08/96

05/21/96 – 06/19/96

06/20/96 – 06/22/96

06/27/96 – 08/08/96

10/24/96 – 11/01/96

12/13/96 – 01/27/97

1997

97-A

01/30/97 – 02/01/97


 



 

4.0 1996 FLAG SUMMARY

Statistical Information:

Details of each 1996 cruise are listed in Table 1 and include cruise dates, number of records, number of values, number of flags, and total percentage of data flagged. A total of 4,954,790 values were evaluated with 306,147 flags added by both the preprocessor and the DQE resulting in 6.18% of the values being flagged.

Table 1: Statistical Cruise Information

Cruise Identifier

Cruise Dates

Number of Records

Number of Values

Number of Flags

Percent Flagged

96-A

96-B

96-C

96-D

96-E

96-F

96-G

96-H

96-I

01/28/96 – 01/30/96

02/09/96 – 03/11/96

03/13/96 – 03/26/96

04/03/96 – 05/08/96

05/21/96 – 06/19/96

06/20/96 – 06/22/96

06/27/96 – 08/08/96

10/24/96 – 11/01/96

12/13/96 – 01/27/97

4,084

45,361

20,159

51,329

38,766

3,200

59,796

10,424

64,989

69,428

589,693

262,067

667,277

736,554

60,800

1,136,124

198,056

1,234,791

1,331

107,196

45,455

60,968

17,029

373

33,246

24,323

16,226

1.92

18.18

17.34

9.14

2.31

0.61

2.93

12.28

1.31

 

Summary:

The 1996 IMET data from the Knorr proves to be of fair quality with 6.18% of the reported values flagged for potential problems. The distributions of flags for the remaining variables are detailed in Table 2.

Table 2: Number of Flags and Percentage Flagged for Each Variable

Variable

B

G

H

J

K

S

Total Number of Flags

Percentage of Variable Flagged

LAT

LON

PL_HD

PL_CRS

PL_SPD

PL_SPD2

PL_WDIR

PL_WSPD

DIR

SPD

DIR2

SPD2

P

T

TS

RH

RAD

PRECIP

 

1

 

 

 

15,701

 

 

 

 

 

 

 

595

 

11,589

29,612

 

 

 

 

 

 

 

 

88

 

 

 

 

801

3,687

 

 

 

 

 

 

 

 

 

 

 

 

 

2

 

10

 

 

 

 

 

1,119

85,019

84,899

7,476

7,476

52

30

1,276

1,241

58

1,235

929

1,387

 

 

 

 

 

113

5,696

5,696

7,577

6,511

3,015

1,896

6,330

5,087

5,971

 

 

159

 

 

44

9

7

34

42

283

230

116

301

147

185

185

1,352

61

26

791

0

1

44

9

7

16,967

90,757

90,878

15,283

14,191

3,368

2,073

7,793

7,909

11,078

12,885

30,567

2,337

0

0.00*

0.01

0.00*

0.00*

5.54

29.65

29.68

4.99

4.64

1.10

0.68

2.55

2.58

3.62

4.21

9.98

0.76

Total Number Of Flags

57,498

4,576

12

192,197

48,051

3,813

306,147

 

Percent Of All Values Flagged

1.16

0.09

0.00*

3.88

0.97

0.08

6.18

*Percentages<0.01

 

B-flags:

Longitude was assessed one B-flag during the 96-D cruise. This flag occurs on a data value when the Knorr crossed the Prime Meridian. This longitude was incorrectly coded.

Platform speed over water (PL_SPD2) received 15,701 B-flags for negative data values. The sensor on this vessel will record negative values when the ship is moving astern relative to the water surface. B-flags retained to mark negative PL_SPD2.

Temperature (T) received 595 B-flags for very low temperatures. The Knorr was near the coast of Newfoundland and was experiencing low temperatures ranging from –10 degrees Celsius to –16 degrees Celsius.

Relative humidity (RH) was assessed 11,589 B-flags over five cruises. The flagged values were above 100%, but below 102%. This is likely due to the sensor not being tuned to high relative humidity values.

Radiation (RAD) received 29,612 B-flags over eight different cruises. These values were between zero and negative one Wm-2. These physically unrealistic negative radiation values are likely the result of the instrument not tuned to low radiation values.

G-flags:

Earth relative wind speed (SPD) received 88 G-flags during the 96-E cruise. The DQE felt these values were realistic, as they were approximately one to five ms -1 greater than the climatological value and were left in place to highlight extreme wind speed values.

Temperature (T) was assessed 801 G-flags over three different cruises. In each cruise, the flagged temperature did not exceed one degree Celsius greater or less than the climatological value; therefore, the DQE feels these are realistic, though extreme, temperatures.

Sea Temperature (TS) had 3,687 G-flags over five different cruises. During the 96-B cruise, the flagged sea temperatures were approximately one degree Celsius greater than the climatological value; the DQE feels these are realistic, though extreme, sea temperatures. The other four cruises, 96-C, G, H, and J, the flagged sea temperatures were approximately one to four degrees Celsius lower than the climatological value and were left in place to highlight extreme values.

The G-flags were left in place to highlight values that are greater than four standard deviations from the climatological mean (da Silva et al. 1994).

H-flags:

Pressure (P) was given two H-flags during the 96-E cruise. The data were relatively normal when a sudden drop in pressure, approximately three millibars, occurred. At this point, an H-flag was placed to show the beginning of the discontinuity. After this event, the values recorded did not resemble normal pressure data and were subsequently K-flagged. When the pressure increased, approximately three millibars, and resumed to the normal data trend, a second H-flag was placed to highlight the end of the discontinuity.

Sea temperature (TS) received ten H-flags over four cruises. Many times when the Knorr was either arriving or leaving port, the sea temperature would have large decreases or increases respectively. If data values were recorded between two H-flags, they were assessed K-flags and should be used with caution.

J-flags:

Platform speed over water (PL_SPD2) received 1,119 J-flags over four different cruises. During each of the four cruises, the J-flags were placed on data values that flat-lined on one value for a period of time.

Platform relative wind direction (PL_WDIR) and platform relative wind speed (PL_SPD) received a total of 169,918 J-flags. Two major problems occurred with these two variables. First, most of the J-flags highlight when the variables were very similar in data trend, almost exact. The second problem was that the variables flat-lined on zero many times. These data values were J-flagged and should not be used.

Earth relative wind direction (DIR) calculated by Knorr and earth relative wind speed (SPD) calculated by Knorr were assessed a total of 14,952 J-flags during the 96-H cruise. These J-flags were given to highlight stair stepping with ship motion and were related to a change in platform heading (PL_HD).

Earth relative wind direction (DIR2) calculated by DAC and earth relative wind speed (SPD2) calculated by DAC were assessed a total of 82 J-flags during the 96-G, H, and I cruises. During the 96-G cruise, the data values flat-lined on zero and were subsequently J-flagged. In the 96-H cruise, (DIR2) and (SPD2) were given 61 J-flags to highlight a large discontinuity in the data. During the 96-I cruise, (PL_WDIR) and (PL_WSPD) flat-lined on zero. Platform relative wind direction and platform relative wind speed are used to calculate the DIR2 and SPD2. When these values hold constant at zero (see J-flags PL_WDIR and PL_WSPD) the earth relative winds are then incorrectly calculated. Consequently, the DIR2 and SPD2 were J-flagged and should not be used.

During the 96-B, C, D, E, and I cruises, J-flags were given to pressure (P), temperature (T), relative humidity (RH), and atmospheric radiation (RAD) since the data values flat-lined on one value for a period of time. Note: RH and RAD had missing data during the 96-I cruise, so they were not J-flagged for the above reason.

Pressure was given 129 J-flags during the 96-H cruise to highlight large spikes that were possibly the result of instrument malfunction.

Sea temperature (TS) was assessed 58 J-flags during the 96-H cruise. Occasionally, when the ship speed was very low, the sea temperature would increase very rapidly, approximately four to nine degrees Celsius in one minute. The problem may be related to poor water flow into the seawater intake. The sea temperature would resume to previous data trend when the ship's speed increased.

K-flags:

K-flags were used to note signatures of ship motion in certain meteorological variables. Variables such as earth relative wind direction (DIR), earth relative wind speed (SPD), atmospheric pressure (P), and temperature (T) showed stair steps in the data. These stair steps were related to a change in platform course (PL_CRS), heading (PL_HD), and/or platform speed (PL_SPD) and should not exist in earth relative data. Subsequently, the data were flagged as suspect.

The platform speed over water (PL_SPD2) received 113 K-flags during the 96-F cruise. These K-flags highlight values that were extremely noisy compared to the surrounding data. Platform speed over water also received K-flags during the 96-H cruise since the data did were slower than the platform speed over ground (PL_SPD).

During the 96-B, C, and D cruise, the platform relative wind direction (PLWDIR) and platform relative wind speed (PL_WSPD) were almost identical as stated in the previous J-flags section. During the 96-D cruise, the two variables started to show a normal data trend, but were K-flagged as cautionary because the variables had been erroneous for many months.

The ocean relative wind direction (DIR) calculated by Knorr, ocean relative wind speed (SPD) calculated by Knorr, earth relative wind direction (DIR2) calculated by DAC, and the earth relative wind speed (SPD2) calculated by DAC had stair steps occurring throughout the data sets. The cause was likely due to flow distortion. Flow distortion is the disturbance of airflow from other objects or instruments upstream from the anemometer. The significance of the stair stepping varied throughout the data set; therefore, the earth relative winds should be used with caution.

Pressure (P) continued stair steps throughout the data sets. There were some stair steps in the pressure data that were a result of a change in either forward speed or direction. These stair steps were associated with approximately a ½ millibar (mb) increase in pressure relative to both the forward speed and direction change of the ship. Other K-flags were given to highlight pressure values that increased approximately ½ millibar but did not have any meteorological or ship relative data to prove or disprove the increase. These data should be use with caution.

Temperature (T) was assessed several K-flags due to radiational heating of the ship. When the platform relative wind speed was low, ~3 ms -1 or less, significant increases in temperature were occurring during daylight hours. The second problem in the temperature (T) data was a ventilation problem, which occurred when the platform wind direction (PL_WDIR) was from around 180 degrees. This likely affected the flow of the air before reaching the bow-mounted thermometer. In these instances, significant increases in temperature were flagged as cautionary.

Sea temperature (TS) was given 5,971 K-flags to highlight large increases. When the platform speed over water (PL_SPD2) would either suddenly decrease/increase, the sea temperature would increase/decrease respectively. This might be due to a malfunctioning intake valve, making it possible that the water is not being measured correctly.

Precipitation (PRECIP) received 159 K-flags during the 96-E and 96-G cruises. The data contained small increases that did not resemble spikes, so the values were flagged as cautionary.

Spikes:

Isolated spikes occurred in most of the variables throughout the data. Spikes are a relatively common occurrence with automated data, caused by various factors (e.g. electrical interference, ship movement, etc.). These individual points were assigned the S-flag.

Missing Data:

Missing data were not a serious concern during the 1996 cruises, but in the 96-I cruise, precipitation and radiation were missing for the entire cruise.


 



 

5.0 1997 FLAG SUMMARY

Statistical Information:

Details of 1997 cruise are listed in Table 3 and include cruise dates, number of records, number of values, number of flags, and total percentage of data flagged. A total of 78,869 values were evaluated with 1,972 flags added by the preprocessor and the DQE resulting in 2.50% of the values being flagged.

Table 3: Statistical Cruise Information

CTC

Cruise Dates

Number of Records

Number of Values

Number of Flags

Percent Flagged

97-A

01/30/97 – 02/01/97

4151

78,869

1,972

2.50

Summary:

The 1997 IMET data from the Knorr proves to be of good quality with 2.50% of the reported values flagged for potential problems. Table 4 details the distribution of flags among the variables.

Table 4: Number of Flags and Percentage Flagged for Each Variable

Variable

B

S

Total Number Of Flags

Percentage of Variables Flagged

LAT

LON

PL_HD

PL_CRS

PL_SPD

PL_SPD2

PL_WDIR

PL_WSPD

DIR

SPD

DIR2

SPD2

P

T

TS

RH

RAD

PRECIP

 

 

 

 

 

29

 

 

 

 

 

 

 

1,925

 

 

 

 

 

 

1

2

5

4

3

 

 

 

 

3

0

0

0

0

0

29

1

2

5

4

3

0

0

1,925

0

3

0

0

0.00

0.00

0.00

0.00

0.00

0.70

0.02

0.05

0.12

0.10

0.07

0.00

0.00

46.37

0.00

0.07

0.00

0.00

Total Number of Flags

1,954

18

1,972

 

Percentage of All Values Flagged

2.48

0.02

2.52

*Percentages < 0.01

B-flags:

Platform speed over water (PL_SPD2) received 29 B-flags during the 97-A cruise for negative data values. This occurs when ship speed is very low, close to zero, and the water is flowing in the opposite direction of the ship's forward motion. During this time of slow speed, the ship will appear to have negative ship speed.

Temperature (T) received 1,925 B-flags for very low temperatures. The Knorr was near the coast of Newfoundland and was experiencing low temperatures ranging from –10 degrees Celsius to –16 degrees Celsius.

Spikes:

Isolated spikes occurred in all of the variables in the data. Spikes are a relatively common occurrence with automated data, caused by various factors (e.g. electrical interference, ship movement, etc.). These individual points were assigned the S-flag.

 


 

6.0 FINAL DISSCUSSION:

The Knorr's platform speed over ground (PL_SPD) data was extremely noisy. The DQE recommends the final user to apply a smoother on the data.

 


 

REFERENCES:

Smith, S.R., C. Harvey, and D.M. Legler, 1996: Handbook of Quality Control Procedures
     and Methods for Surface Meteorology Data
.WOCE Report No. 141/96,Report WOCEMET
     96-1, Center for Ocean-Atmospheric Prediction Studies Florida State University,
     Tallahassee FL 32306-2840

da Silva, A.M., C.C. Young and S. Levitus, 1994: Atlas of Surface Marine Data 1994, Volume 1:
      Algorithms and Procedures.
NOAA Atlas Series.

Slutz, R.J., S.J. Lubker, J.D. Hiscox, S.D. Woodruff, R.L. Jenne, D.H. Joseph, P.M. Seurer and
       J.D. Elms, 1985: COADS - Comprehensive Ocean Atmosphere Data Set,
      CIRES/ERL/NCAR/NCDC, Boulder, Colorado.