Tipultech logo

Breadth of a sample

Author: Dr Simon Moss

Introduction

Consider a researcher who wants to examine whether or not popping bubble-wrap tends to alleviate stress. The researcher must determine the target population. For example, the researcher could derive the sample from:

In other words, the researcher could limit the sample in relation to numerous extraneous variables, such as organization, industry, management level, gender, clinical status, and so forth. Sometimes, the decision is apparent. For example, suppose the researcher specifically needs to ascertain whether or not the effect of popping bubble-wrap on stress differs between males and females. In this instance, the researcher cannot limit the sample to females only. Likewise, suppose the researcher wants to ascertain whether or not the effect of popping bubble-wrap differs between managers and other employees. This researcher cannot limit the sample to managers only.

Usually, however, the researcher does not need to examine the effect of management level, gender, industry, and so forth. That is, these variables are not incorporated into the hypotheses. Hence, the researcher needs to decide whether or not to limit the sample in relation to these extraneous variables. In particular, the researcher needs to consider each extraneous variable in turn. For example, the researcher might first decide whether or not to include both males and females. The researcher might then decide whether or not to include both managers and other employees, and so forth. The remainder of this document describes the process that researchers should undertake to reach these decisions.

Step 1. Identify spurious relationships

The first step is to ascertain whether or not the extraneous variable could yield a spurious or erroneous relationship. To achieve this objective, first specify the principal categories or levels that pertain to this extraneous variable. For example, the principal categories of gender are male and female. The principal categories of industry are manufacturing, mining, finance, education, and so forth.

Second, for each hypothesis or research question, specify the dependent and independent variables. For example, in the previous example, the dependent variable is the level of stress and the independent variable is the time devoted to popping bubble-wrap.

Third, for each hypothesis or research question, determine whether or not the extraneous variable is likely influence both the dependent and independent variable . For instance, the level of stress but the time that is devoted to popping bubble-wrap is likely to differ between males and females. In contrast, both the level of stress and the time that is devoted to popping bubble-wrap is likely to differ across industries. For example, the manufacturing industry might be more likely to promote stress as well as involve packaging, which provides employees with an opportunity to pop bubble-wrap.

In other words, sometimes the extraneous variable influences both the dependent and independent variables that pertain to some hypothesis. In this instance, the data analysis will reveal the dependent and independent variables are related to one another even if they are actually independent. To demonstrate, consider the extract of data that is presented in the table below. The top half of this table represents the extent to which employees in the manufacturing industry pop bubble wrap and experience stress. The bottom half of this table represents the extent to which employees in the service industry pop bubble wrap and experience stress. Both variables are measured along a 10-point scale, where 1 denotes a negligible degree and 10 denotes a pronounced degree.

Industry Popping Stress
Manufacturing 9 8
Manufacturing 7 7
Manufacturing 8 9
Manufacturing 8 7
. . .
. . .
Services 3 2
Services 1 4
Services 4 5
Services 2 2
. . .
. . .

Suppose that popping bubble wrap does not really alleviate stress. As a consequence, in the manufacturing industry, popping bubble-wrap will not correlate with stress. Likewise, in the service industry, popping bubble-wrap will also not correlate with stress. In other words, these variables will not be related to one another within each industry.

Nevertheless, in the manufacturing industry, the extent to which employees both pop bubble-wrap and experience stress is high. Conversely, in the services industry, the extent to which employees both pop bubble-wrap and experience stress is low. When the observations from these two sectors are combined, elevated levels of popping bubble-wrap will often coincide with elevated levels of stress. Similarly, negligible levels of popping bubble-wrap will often coincide with negligible levels of stress. In other words, the variables will be artificially related to one another across the industries.

In short, when both industries are considered together, popping bubble-wrap appears to be correlated with the level of stress that participants experience. This finding erroneously suggests that popping bubble-wrap might alleviate stress. In other words, when industry is not controlled, an erroneous or artificial relationship between the dependent and independent variable emerges. An erroneous or artificial relationship that arises when both the dependent and independent variables are influenced by an extraneous variable is called spurious.

In contrast, when industry is controlled, a different pattern of results emerges. Specifically, when the manufacturing and service industries are explored separately, popping bubble-wrap does not appear to be correlated with the level of stress that participants experience. This finding accurately suggests that popping bubble-wrap does not alleviate stress.

<"color: rgb(255, 0, 0)&">Summary. Sometimes an extraneous variable, such as industry, influences both the dependent and independent variables. Hence, statistical procedures will yield a significant correlation between the dependent and independent variables, even when these measures are actually unrelated. Such a relationship is called spurious.

Step 2. Determine whether or not spurious relationship can be controlled statistically

The previous section revealed that extraneous variables can yield a spurious, misleading relationship unless controlled. One of two approaches can be adopted to control these extraneous variables. This section reveals that extraneous variables can be measured and then controlled statistically. The following section reveals the sample can be limited to one level of these extraneous variables.

To control extraneous variables through statistical procedures, two steps need to be undertaken. First, the extraneous variable must be measured. Second, this variable must be designated as a covariate in the context of ANCOVAs and MANCOVAs or incorporated as a predictor in the context of multiple regression analysis.

To illustrate the statistical procedures that can be undertaken to control a variable artificially, consider the extract of data in the following table. Suppose the researcher decided to assess the effect of popping bubble-wrap on stress after controlling gender. To achieve this goal, the researcher could conduct a multiple regression analysis, where stress is the criterion variable and popping bubble-wrap and gender are the predictors.

Gender Popping Stress
Male 9 8
Female 7 7
Female 8 9
Male 8 7
Female 3 2
Male 1 4
Female 4 5
Male 2 2
. . .
. . .
Female 3 2

This process could yield output that resembles the following table. The significance level associated with popping is less than 0.05 and thus significant. This finding suggests that popping bubble-wrap alleviates stress, even after controlling gender. In other words, the observed relationship between popping bubble-wrap and stress cannot be ascribed to the spurious effect of gender.

. B SE Beta t sig
Constant 1.5 1.5 . 1.0 .15
Popping -4.4 1.1 -1.5 4.0 .01
Gender 2.4 1.8 1.0 1.25 .12

Unfortunately, not all extraneous variables should be controlled statistically. In particular, several obstacles can undermine this approach. First, the extraneous variable cannot always be assessed readily and accurately. For example, level of management is difficult to measure when the sample comprises many organizations. A title of senior manager in one organization might connote different responsibilities to a title of senior manager in another organization.

Second, when some extraneous variables are controlled statistically, power might plummet. That is, the likelihood that effects will attain significance can diminish inordinately. To illustrate, suppose the researcher decides to measure and control several extraneous variables, such as gender, management level, industry, size of organization, and profit of organization. The inclusion of these variables in the context of multiple regression raises the number of predictors from 1 to 6. As a consequence, the number of participants that need to be assessed to ensure that power is acceptable increases from perhaps 40 to 140.

This reduction in power partly depends on the level of measurement and the number of categories that pertains to each extraneous variable. For example, sometimes the extraneous variable is numerical, such as the profit or revenue of the organization. When this extraneous variable is incorporated into a multiple regression and thus controlled statistically, power does not diminish appreciably. Indeed, an additional 10 or so participants would restore the level of power.

Likewise, sometimes the extraneous variable comprises to categories, such as gender. Again, when this extraneous variable is incorporated into a multiple regression and thus controlled statistically, power does not diminish appreciably. An additional 15 or so participants would restore the level of power.

Unfortunately, some extraneous variables comprise many categories or levels. Power diminishes appreciably when statistical procedures are utilized to control these extraneous variables. To demonstrate the source of this reduction in power, consider the extract of data that is presented in the table below. Suppose the researcher wants to control the effect of industry. These data could be subjected to a multiple regression analysis, where the criterion variable is stress and the predictors are popping bubble-wrap and industry. Unfortunately, in the context of multiple regression, the predictors must be numerical or dichotomous. That is, categorical predictors cannot be included, unless these variables comprise only two levels, such as gender. In other words, industry cannot be included in this multiple regression analysis.

Industry Popping Stress
Manufacturing 9 8
Mining 7 7
Finance 8 9
Education 8 7
. . .
. . .
Services 3 2
Manufacturing 1 4
Mining 4 5
Education 2 2
. . .
. . .
Finance 3 2

To circumvent this restriction, categorical variables can be converted into a series of dichotomous variables. In particular, each dichotomous variable pertains to one of the categories. For example, to represent industry, the dichotomous variables could pertain to manufacturing, mining, finance, education, and so forth. These columns are presented in the following table. In each column, a 1 denotes the participant pertains to that category and a 0 denotes the participant does not pertain to that category. The dichotomous variable associated with one of the categories, such as services, needs to be discarded to prevent a problem called singularity.

Manufacturers Mining Finance Education Popping Stress
1 0 0 0 9 8
0 1 0 0 7 7
0 0 1 0 8 9
0 0 0 1 8 7
0 0 0 0 7 4
0 0 0 0 5 7
0 0 0 0 3 2
1 0 0 0 1 4
0 1 0 0 4 5
0 0 0 1 2 2
. . . . . .
. . . . . .
0 0 1 0 3 2

To conduct the multiple regression, the criterion variable is stress and the predictors are popping, manufacturing, mining, finance, and education. Each additional dichotomous variable further reduces power. In other words, when extraneous variables that comprise many categories are controlled statistically, power diminishes to a pronounced extent.

<"color: rgb(255, 0, 0)&">Summary. To counteract spurious relationships, one of two approaches can be adopted. First, extraneous variables that can be measured and do not comprise too many categories can be controlled statistically. Otherwise, the sample needs to be limited

Step 3. Determine whether or not a limited sample would unduly restrict variance

To reiterate, extraneous variables can be controlled statistically. This approach, however, is unsuitable if the extraneous variable cannot be measured or entails many categories. In this instance, the sample needs to be restricted. Specifically, the researcher must ensure the entire sample is equivalent in relation to this extraneous variable.

To illustrate, suppose that industry influences both the dependent and independent variables. Accordingly, to prevent spurious relationships, the researcher could assess only one industry, such as manufacturing. Likewise, suppose the size of organizations influences both the dependent and independent variables. Hence, to prevent spurious relationships, the researcher could assess only one organization.

This restriction, however, can present some shortcomings. The principal difficulty revolves around the variability of other important variables. To illustrate, suppose the researcher decides to assess the employees in one organization only. Conceivably, this organization might have recently introduced regulations in which the popping of bubble-wrap is prohibited or discouraged. Variability in relation to the time devoted to popping bubble-wrap is thus curtailed. This reduction in variability can diminish power exorbitantly. In other words, the likelihood that effects will attain significance can decline. To accommodate this issue, researchers should thus:

Step 4. Ascertain convenience of limiting the sample

To summarize, researchers should specify all the extraneous variables that could influence both the dependent and independent variables. These extraneous variables could yield spurious relationships and should thus be controlled. If these variables can be measured and do not entail too many categories, they should be controlled through statistical procedures, such as ANCOVA or multiple regression. Otherwise, provided that variability remains sufficient, the sample should be restricted to one level of the extraneous variable.

In some instances, neither approach to control extraneous variables is applicable. For example, sometimes the extraneous variable comprises many categories& however, attempts to limit the sample to one level of these extraneous variables curtails the variability. In this instance, other factors can be considered to facilitate the decision. Specifically, the logistics of each option should also be considered.

For example, sometimes the scales or questionnaires that are utilized to measure the variables cannot be applied to all categories or levels of the extraneous variable. To illustrate, a researcher might want to utilize the performance appraisal system of an organization to gauge the productivity of participants. These systems tend to vary markedly across organizations. Hence, the researcher should probably restrict the sample to one organization.

Nevertheless, when the sample is restricted to one category or level of an extraneous variable, the availability of participants might diminish inordinately. To illustrate, researcher who decide to restrict the sample to one organization might not be able to attract enough participants. In this instance, the extraneous variable should be controlled statistically instead.

<"color: rgb(255, 0, 0)&">Summary. Statistical control, in which all levels of the extraneous variable need to be utilized, can present some logistical problems& the measures that are administered might not apply to all levels of the extraneous variable. Limiting the sample, however, can also present some logistical problems& the sample size might not be sufficient.

Step 5. Replication and generalization

This document has identified the conditions in which the sample should be restricted. One obvious problem, however, arises when the sample is restricted



Academic Scholar?
Join our team of writers.
Write a new opinion article,
a new Psyhclopedia article review
or update a current article.
Get recognition for it.





Last Update: 6/1/2016