Annual Survey of Manufactures Sampling and Estimating Methodologies => DESCRIPTION OF SURVEY SAMPLE The annual survey of manufactures (ASM) contains two components. The mail portion of the survey is a probability sample of about 64,000 manufacturing establishments selected from a total of about 216,000 establishments. These 216,000 establishments represent all manufacturing establishments of multiunit companies and all single-establishment companies mailed schedules in the 1987 Census of Manufactures. This mail portion is supplemented annually by a Social Security Administration list of new manufacturing establishments opened after 1987 and a list of new multiunit manufacturing establishments identified from the Census Bureau's Company Organization Survey. For the current panel, all establishments of companies with 1987 shipments in manufacturing in excess of $500 million were included in the survey panel with certainty. There are approximately 500 such companies collectively accounting for approximately 18,000 establishments. For the remaining portion of the mail survey, the establishment was defined as the sampling unit. For this portion, all establishments with 250 employees or more and establishments with a very large value of shipments also were included in the survey panel with certainty. A total of 12,100 establishments were selected from this portion of the universe with certainty. Therefore, of the 64,000 manufacturing establishments included in the ASM panel, approximately 31,000 are selected with certainty. These certainty establishments collectively account for approximately 80 percent of the total value of shipments in the 1987 census. Smaller establishments in the remaining portion of the mail survey were sampled with probabilities ranging from 0.999 to 0.005 in accordance with mathematical theory for optimum allocation of a sample. The probabilities of selection assigned to the smaller establishments were proportional to measures of size determined for each establishment. The measures of size depend directly upon each establishment's 1987 product class values and the historic variability of the year-to-year shipments of each product class. Product classes displaying more volatile year-to-year change in shipments at the establishment level were sampled at a heavier rate. This method of assigning measures of size was used in order to maximize the precision (that is, minimize the variance of estimates of the year-to-year change) in the value of product class shipments. Implicitly, it also gave weight differences in employment, value added, and other general statistics, since these are highly correlated with value of shipments. Individual sample selection probabilities were obtained by multiplying each establishment's final measure of size by an overall sampling fraction coefficient calculated to yield a total expected sample size. The sample selection procedure gave each establishment in the sampling frame an independent chance of selection. This method of independent selection permits the rotation of small establishments out of a given sample panel without introducing a bias into the survey estimates. The nonmail portion of the survey includes all single-establishment companies that were tabulated as administrative records in the 1987 Census of Manufactures. Although this portion contained approximately 134,000 establishments, it accounted for less than 2 percent of the estimate for total value of shipments at the total manufacturing level. This portion was not sampled; rather, the data for every establishment in this group were estimated based on selected information obtained annually from the administrative records of the Internal Revenue Service and the Social Security Administration. This administrative-records information, which includes payroll, total employment, industry classification, and physical location of the establishment, was obtained under conditions which safeguard the confidentiality of both tax and census records. Estimates of data other than payroll and employment for these small establishments were developed from industry averages. The corresponding estimates for the mail and nonmail establishments were added together, along with the base-year differences, as defined in the Description of Estimating Procedure section, to produce the figures shown in this publication. => DESCRIPTION OF ESTIMATING PROCEDURES Most of the ASM estimates for the years 1988-1991 were computed using a difference estimation procedure. For each item, a base-year difference was developed. This base-year difference is equal to the difference between the 1987 census published number for an item total and the linear ASM estimate of the total for 1987. The ASM linear estimate was obtained by multiplying each sample establish- ment's data by its sample weight (the reciprocal of its probability of selection) and summing the weighted values. These base-year differences were then added to the corresponding current- year linear estimates, which include the sum of the estimates for the mail and nonmailestablishments, to produce the estimates for the years 1983- 1991. Estimates developed by this procedure usually are far more reliable than comparable linear estimates developed from the current sample data alone. However, the 1992 sample estimates for the purchased service items, shown in file MC92SF3C, are strictly ASM linear estimates developed only from ASM establishments that reported the specific item. The remaining estimates in file MC92SF3C, showing the breakdown of expenditures for new machinery and equipment and costs of parts (separated into purchases from foreign sources and purchases from domestic sources), were computed as ratio estimates. To do this, linear estimates of the new machinery detail items were developed from the ASM establishments and were ratio adjusted to the corresponding census total for new machinery. In a similar fashion, the ASM linear estimates of the detailed purchased materials items were ratio adjusted to the corresponding census total for cost of parts. => ADJUSTMENTS OF HISTORICAL DATA The goal of the ASM is to produce estimates that represent the employer portion of the manufacturing universe. During census years, ASM estimates generated from the same data set have typically been low in comparison to the census data. In 1987 and 1992, the ASM estimates for shipments at the all manufacturing level were 2.0 and 3.5 percent lower than the census values, respectfully. For 1992, the initial ASM estimate for shipments was approximately 3.7 percent low. Preliminary analyses of the 1992 Census of Manufactures data also suggested that the 1988-91 ASM estimates understated the true levels. An interagency task force, comprised of representatives from the Bureau of Economic Analysis (BEA), the Federal Reserve Board (FRB), and the Census Bureau was formed to investigate this issue and provide recommendations. After examining the evidence, the consensus opinion was that the understatement in the ASM estimates from 1988-91 was sufficient to warrant a revision to the ASM historical time series. The task force recommended that the ten general statistics (GS) variables published in the Statistics for Industry Groups and Industries publication (AS-1) be revised for 1988-91. The major factors contributing to this decision were as follows: 1. The classification of new single-establishment companies continued to be a problem between 1988 and 1992. The number of totally unclassified companies in our master universe file grew significantly through 1991 and then declined sharply in 1992 due to special census processing. A portion of these unclassified companies became classified as manufactures for the first time in 1992, despite having been in operation for up to 4 years. These cases are not represented in the originally published ASM estimates for 1988-91 because of their lack of classification. In addition, it appears that a significant number of new manufacturing companies were initially classified as nonmanufacturing and, therefore, not subjected to our birth supplementation. Beginning with the 1993 ASM, we have introduced procedures to address both these issues. 2. The 1992 ASM was processed within the census survey processing system. While similar to the regular ASM processing system, there were subtle differences in data collection, editing, imputation, and analyst review that cannot be totally eliminated. This suggests that the 1992 ASM estimates derived using census files are not totally comparable to the 1988-91 ASM estimates. Our analysis suggests the understatement in the 1992 ASM estimates because of processing differences to be between 1 and 1.5 percent. For the 1997 ASM, we are redeveloping components of the processing system. 3. The originally published 1989 ASM estimates appear to be uniformly low in comparison to other comparable time series and our own internal analysis. This suggests that the original 1989 ASM sample was not as representative as expected or that there were undetected operational errors in producing the estimates. 4. As with all sectors of the economy, the manufacturing sector is continually changing. Despite tightly controlled procedures for maintaining the ASM sample, some deterioration in coverage and representation was unavoidable. The revised estimates for 1988-91 were derived through a series of distinct steps. Essentially, a "smoothing" procedure was used to allocate the difference between the 1992 ASM estimates and the corresponding census values over the 5 year span. However, before applying this procedure, revised ASM estimates for 1989 and 1992 were developed to address problems 2 and 3 described above. -> Generating Revised 1989 Estimates The 1989 ASM estimates were separated into industries for which the ASM estimates reliably represented the population and industries for which the estimates were suspect. In the suspect industries, the change from 1987 to 1989 was re-estimated using the information reported by establishments in these 2 years. A change ratio estimator was used rather than the difference estimator usually used for ASM estimation. This procedure was not as rigorous as the usual estimator, but offered the advantage of not being as sensitive to the factors that were felt to have caused the original understatement. After the re-estimation, the total value of shipments for all manufacturing industries in 1989 was 0.8 percent higher than originally published. -> Adjustment to 1992 ASM Estimates As noted above, the initial 1992 ASM estimates were derived within the census processing system rather than the usual ASM system. The initial difference between the 1992 ASM estimate of shipments and the census value was approximately 3.7 percent. At the four-digit SIC level, an estimate of the processing bias was determined and the initial 1992 ASM estimates adjusted accordingly. This adjustment was felt to improve the comparability between the 1992 and the 1988-91 ASM estimates. The adjustment procedure raised the initial 1992 ASM estimate for shipments approximately 1.6 percent, thereby reducing the understatement of the 1992 ASM estimate of shipments to approximately 2.1 percent at the all manufacturing level. Similar differences existed between the 1992 ASM estimates of the other general statistics variables and the 1992 census values. -> Historical Revision This 2.1 percent underestimate of the census by the 1992 ASM is defined as the "drift" and is felt to be an accurate measure of the cumulative degradation in the quality of the estimates due to erosion in the representativeness of the sample and processing error. To improve the historical ASM time series, the difference between the adjusted 1992 ASM estimate and the census value was computed for each GS variable at specific industry group levels. This difference was divided by five and the result added to the industry group level estimate for that variable for 1988. Twice the difference was added to the re-estimated industry group level estimate for 1989. Three times the difference was added for 1990, and four times the difference for 1991. Revised estimates for other variables and for other levels of aggregation (geographical) were not calculated. The percentage revision in each variable for each year was calculated for each industry group level and that percentage revision was made to each four- digit SIC within the industry group. The chart below shows the ASM originally published inventories and shipments estimates at the total manufacturing level and the comparable estimates after the adjustment process for 1988-91. The 1987 and 1992 values represent census statistics, not ASM estimates. Year End-of-year inventories Value of shipments (billion dollars) (billion dollars) Published Adjusted Published Adjusted 1992 374.9 374.9 3,004.7 3,004.7 1991 379.9 387.1 2,826.2 2,878.2 1990 392.5 397.9 2,873.5 2,912.2 1989 380.5 384.5 2,792.7 2,840.4 1988 361.5 363.3 2,682.6 2,695.4 1987 332.5 332.5 2,475.9 2,475.9 => QUALIFICATIONS OF THE DATA The estimates developed from the sample are apt to differ somewhat from the results of a survey covering all companies in the sampled lists but otherwise conducted under essentially the same conditions as the actual sample survey. The estimates of the magnitude of the sampling errors (the differences between the estimates obtained and the results theoretically obtained from a comparable, complete-coverage survey) are provided by the standard errors of the estimates. The particular sample selected for the ASM is one of a large number of similar probability samples that, by chance, might have been selected under the same specifications. Each of the possible samples would yield somewhat different sets of results, and the standard errors are measures of the variation of all the possible sample estimates around the theoretical, comparable, complete-coverage values. Estimates of the standard errors have been computed from the sample data for selected statistics in this report. They are presented in the form of relative standard errors (the standard errors divided by the estimated values to which they refer). In conjunction with its associated estimate, the relative standard error may be used to define confidence intervals (ranges that would include the comparable, complete-coverage value for specified percentages of all the possible samples). The complete-coverage value would be included in the range: 1. From one standard error below to one standard error above the derived estimate for about two-thirds of all possible samples. 2. From two standard errors below to two standard errors above the derived estimate for about 19 of 20 of all possible samples. 3. From three standard errors below to three standard errors above the derived estimate for nearly all samples. An inference that the comparable, complete-survey result would be within the indicated ranges would be correct in approximately the relative frequencies shown. Those proportions, therefore, may be interpreted as defining the confidence that the estimates from a particular sample would differ from complete-coverage results by as much as one, two, or three standard errors, respectively. For example, suppose an estimated total is shown as 50,000 with an associated relative standard error of 2 percent, that is, a standard error of 1,000 (2 percent of 50,000). There is approximately 67 percent confidence that the interval 49,000 to 51,000 includes the complete- coverage total, about 95 percent confidence that the interval 48,000 to 52,000 includes the complete-coverage total and almost certain confidence that the interval 47,000 to 53,000 includes the complete-coverage total. In addition to the sample errors, the estimates are subject to various response and operational errors: errors of collection, reporting, coding, transcription, imputation for nonresponse, etc. These operational errors also would occur if a complete canvass were to be conducted under the same conditions as the survey. Explicit measures of their effects generally are not available. However, it is believed that most of the important operational errors were detected and corrected in the course of the Census Bureau's review of the data for reasonableness and consistency. The small operational errors usually remain. To some extent, they are compensating in the aggregated totals shown. When important operational errors were detected too late to correct the estimates, the data were suppressed or were specifically qualified in the files. As derived, the estimated standard errors included part of the effect of the operational errors. The total errors, which depend upon the joint effect of the sampling and operational errors, are usually of the order of size indicated by the standard error, or only moderately higher. However, for particular estimates, the total error may considerably exceed the standard errors shown. The concept of complete coverage under the conditions prevailing for the ASM is not identical to the complete coverage of the census of manufactures, as the censuses have been conducted. Nearly all types of operational errors that affect the ASM also occur in the censuses. The ASM and the censuses, are conducted under quite different conditions, and operational errors can be better controlled in the ASM than in the censuses. As a result, for many of the census figures, the errors are of the same order of size as the total errors of the corresponding annual survey estimates. The differences between the census and ASM operating conditions also disturb, to some degree, the comparability of the ASM and census data. Any figures shown in the files in this publication having an associated standard error exceeding 15 percent may be of limited reliability. However, the figure may be combined with higher-level totals, creating a broader aggregate, which then may be of acceptable reliability.