Lived School Experiences that Encouraged One Person to Become a Creative Writer:
     Study III of VI
          Part III

Dan Lukiv
M.Ed., English and Creative Writing
McNaughton Centre, Quesnel, BC, Canada
E-mail: lukivdan@shaw.ca

Further Research

Once I have completed Study VI, I will research all six phenomenological studies through a methodology I call Theory from Phenomenology (Lukiv, 2004a). I have formulated this methodology by drawing on grounded theory principles (2004a), although what I call covering theory is not true grounded theory (Baker, Wuest, & Stern, 1995; and Stern, 1995). The purpose: to see if I can synthesize, through an inductive process, the themes into a covering theory that could direct teachers interested in creating classroom settings that may encourage some students to take up creative writing as a vocation. I will call this Study VII. I will confirm, if possible--if the original six participants are available--the truth of the theory through participant review by all six participants, just as I will have confirmed the truth of the themes in each of the six phenomenological studies through participant review by each respective participant.

But my series of studies of the phenomenon of what school experiences have encouraged creative writers will not end at Study VII. I wish to conduct one more study (VIII), a survey of a sample of creative writers, to investigate the truth of the theory according to other writers. If teachers are going to extrapolate my covering theory, to apply it in their classrooms, to "extend known experience [based on the theoretical expression of those experiences of the six participants]...to arrive at a useful conjecture [a useful mode of practice]" (McMillan & Schumacher, 1997, p. 7), then they may want to see that the theory applies to more than just the six original participants.

For quick reference to the aforementioned eight studies, consider the following:

         Study I: hermeneutic phenomenology, Arthur (poet), 2002;
         Study II: hermeneutic phenomenology, Thomas (poet), 2004;
         Study III: hermeneutic phenomenology, Elizabeth (fiction writer), 2005;
         Study IV: hermeneutic phenomenology, [a fiction writer?], 2005?;
         Study V: hermeneutic phenomenology, [a dramatist?], 2006?;
         Study VI: hermeneutic phenomenology, [a dramatist?], 2007?;
         Study VII: theory from phenomenology, 2008?; and
         Study VIII: survey research, 2010?.

This path, from my point of view, looks epic in scope, but all great journeys begin with shorter ones. If Study I was Point A on my phenomenological landscape, then Study II placed me at Point B, and Study III places me now at point C. I look forward to the journey ahead...the great adventure of my researching writers' relevant lived school experiences and sharing my findings with other researchers and classroom teachers. As in pure scientific research, my work may gain me surprisingly new, even astonishing insights (Kiener, 2002) worth sharing.

Naturally, I want my studies to help teachers encourage some students to become creative writers, thereby contradicting the "widespread perception that knowledge created by [researchers] is not used in practice" (Boland et al., 2000, Learning Theory, Knowledge Representations, and Knowledge Transfer, para. 1). I witnessed this perception repeatedly as many students in my recent M.Ed. cohort expressed concern over "ivory tower" researchers whose work lies cocoon-like, educationally dormant, without influence in everyday classroom activity. Transformational knowledge, on the other hand, "moves" from the researcher to the classroom teacher who actually uses that knowledge to help him or her instruct students.

Shriberg (2002) speaks of "transformational leaders [who] focus themselves and all stakeholders on long-term, shared personal...commitments" (p. 45). He speaks of leadership at the university level that promotes ecological sustainability for all concerned--for all university faculty, other staff, and students (2002). His language reminds me that education researchers should help teachers apply their research in the classroom. Such researchers would become transformational leaders.

I could, as a transformational leader, see that my studies are published in a number of printed settings, to ensure that a wide variety of teachers have access to their direction. That means publication must go beyond the journals that, generally, only education professors read. To date, actually, I have had Studies I and II or parts thereof published in Academic Exchange Extra, Mentor: The Online Publication for Nova Scotia's Educators, Arts North, Connected Magazine [online supplement], SchoolNet Africa, The Journal of Secondary Alternate Education, and The Teachers.Net Gazette.

Also, as a transformational leader, I could run workshops for teachers, helping them apply my studies' direction. If wisdom, in one sense, is knowledge applied (Draw Close, 2002), then my efforts could help teachers make wise choices in their efforts to encourage students to take creative writing seriously. I could run workshops at my own school, McNaughton Centre, and at my school district's yearly professional development days. There are provincial- and national-level conferences, too, that invite researchers to "display" their work.

Publication of my studies and workshops about their findings will offer teachers opportunities to think about how to "extrapolate." Studies I through VI, also Study VII, will not offer generalizations. None of the studies will represent random samplings. Neither will my survey, Study VIII, which will, for a number of reasons I will shortly explain, use a non-random sample of purposive/information-rich respondents (Patton, 1987; see, e.g., Arnold & Ketter, n.d.). Naturally, I will ensure confidentiality/anonymity for each respondent (Ferber, Sheatsley, Turner, & Waksberg, 1980c; and Ferber, Sheatsley, Turner, & Waksberg, 1980a; see, e.g., Arnold & Ketter, n.d.; and McElrath & McEvoy, 2001), a subject given considerable weight in graduate courses in surveys (see, e.g., Godfrey, 2002).

From these studies or workshops, teachers who want to encourage some students to take up creative writing as a vocation may gain a sense of what sorts of activities, what sorts of extrapolations, make sense. The survey's results and conclusions may add considerable weight, or usefulness, to the theory of Study VII as an intellectual point of reference for teachers. With respect to the theoretical direction, teachers may gain a sort of class smarts, like the street kid who learns street smarts, in terms of their creating a classroom "event in time" (Waterman, 1998), even a variety of "extrapolated" classroom events.

Extrapolation, here, does not relate to an if x, then y theory (Borgatti, 1996a), which would relate to external validity (McMillan & Schumacher, 1997), the validity of generalizing or applying the theory to creative writers in general: Population external validity (McMillan & Schumacher, 1997) would relate my respondents to the larger pool of other creative writers, and ecological external validity (1997) would relate my respondents' relevant experiences in school to similar experiences of those in that larger pool. Rather, I am speaking of an if x, then perhaps y theory, especially if the theory rings true not only for all six participants, but for many survey respondents too. Extrapolation draws on the if x, then perhaps y theory when I refer to that larger pool. Such a theory would not describe independent and dependant variables, but it would describe at least a "sense of process" (Borgatti, 1996a).

By sense of process I refer to activities, events, and teacher-student interactions that have encouraged my participants and the survey's respondents to become creative writers. To say that all creative writers have experienced such activities, events, and teacher-student interaction would constitute generalizability. The mathematics of that statement means z = f(x₁, x₂, x₃,..., x_n), for which z (becoming a creative writer) is a function (f) of n variables, namely x₁, x₂, x₃,...and, x_n. Such a bold statement, if global, would require a random sample (Ferber et al., 1980c) of creative writers from a sampling frame (Ferber, Sheatsley, Turner, & Waksberg, 1980b) beyond the writers of my hometown, Quesnel, beyond my province, British Columbia, and, quite frankly, beyond Canada, my country. Certainly, many studies use random samples (see, e.g., Barbados Study, 1997), but such samples are not always possible (McMillan & Schumacher, 1997). My study will be a case in point.

How could I acquire the sampling frame for just my hometown? The Quesnel Writers' Group, which I helped establish in 1979, presently has a list of members who write, but many people in the area who presently do not belong to the group also write. I, presently, am one example. As for others in my area: Who are they? Are some "invisible" because they write under pseudonyms? I cannot comprehensively answer either question. Therefore, a random sample of my area is not possible (Trochim, 2002a).

The difficulty in answering these questions grows exponentially as the geographical area increases. Other variables add difficulty. The Burnaby Writers' Society, of BC, advertises that "membership...is open to anyone, anywhere" (Burnaby, 2003, p. 487). That means members may not be established writers, and they may not live in Burnaby. The question would remain, who makes up the sampling frame of established writers in Burnaby?

Who makes up the sampling frame for British Columbia, for Canada? I know much-published creative writers who belong to no writers' groups, guilds, or leagues at the municipal, territorial, provincial, or national level. Will the Canadian Poetry Association (Canadian Poetry, 2003, p. 487), the Canadian Authors Association (Canadian Authors, 2005), the Writers Guild of Canada (Writers Guild, n.d.), and The League of Canadian Poets (The League, 2003, p. 490) enable me to establish a comprehensive sampling frame at the national level? No. Not even The Quesnel Writers' Group can enable me to list such a frame at the hometown level. Other researchers, however, have taken random samples from available sampling frames, such as Adams, Matto, and Harrington (2001), who randomly sampled from "social workers nationwide (NASW [National Association of Social Workers], 1999), from...65,996 members in their database who indicated they were in direct clinical practice" (p. 365), to study "convergent and discriminant validity of the Traumatic Stress Institute Belief Scale (TSI)-Revision L (Traumatic Stress Institute, 1994) as a measure of vicarious trauma in...clinical social workers" (p. 363). Such a study affords excellent possibilities for external validity, for generalizing to a large body of social workers.

In view of what I have discussed about sampling frames, random sampling necessary for externally valid (i.e., external population and ecological validity) generalizability (McMillan & Schumacher, 1997), within the bell curve-probability realm of statistical interpretation, becomes an impossibility--the decision about what to do about non-respondents possibly biasing the statistical pool notwithstanding. Non-random sampling becomes the reasonable alternative (Developing a Survey, n.d.; see, e.g., Hare & Skinner, 1999).

Hence, extrapolation, the fruit of non-random sampling, replaces generalizability. Researchers who make generalizations from non-random samples can mislead the public, especially when these generalizations enter the public domain. For example, Brown and Booth (in press) discuss a famous study by Arlie Hochschild (1997), who concluded from a non-random sample that many mothers and fathers avoid the demands of domestic life by spending many hours in the workplace (Brown & Booth, in press), a conclusion that implied a nation-wide phenomenon. Hochschild's research, published in her book entitled The Time Bind, has been referenced dozens of times in academia, but other studies have not supported her research. Why not? As Brown and Booth (in press) point out, "the generalizability of her findings is limited by her nonrandom sample of a small number of workers in a single firm" (p. 6). People who make decisions based on grandiose generalizations may find themselves embarrassed according to Matthew 15:14 (New World Translation): "If, then, a blind man guides a blind man, both will fall into a pit." Therefore, I want to reflect some modesty in my use of the word extrapolation. In the words of Williams (1996), "good researchers recognize and acknowledge the limitations of their work" (First Project: Experiments, para. 7).

For my survey, non-random sampling, as in the case of my purposive/information-rich (Patton, 1987) respondents, will define a group of successful--established--writers, and will, I hope, lead me to some modest conclusions. By successful, I mean my respondents will have proven their commitment to their vocation through their own letters of recommendation--their own substantial credits, as in the case of Arthur, Thomas, and Elizabeth. For poets and fiction writers, I refer to a substantial body of published works. For dramatists, I refer to a substantial body of produced or published plays.

I won't resort to snowball sampling, although researchers frequently "snowball" (see, e.g., McElrath & McEvoy, 2001). I wouldn't want to snowball through fringe writers, with little professional experience, referring to likewise fringe writers. Ironically, researchers often use snowball sampling to locate fringe groups (Qualitative Field Research, n.d.). Although I could use snowball sampling to locate established writers through other established writers, the process isn't necessary. Established writers are easy to find. I will purposively choose such writers, just as I have purposively chosen them for Studies I, II, and III.

Therefore, the survey will not require a screening question to determine which respondents are appropriate for the survey (Trochim, 2002a); however, one contingency question (Trochim, 2002b), to root out biased respondents, could be: Have you read any of my published research studies (I to VI) or articles that refer to them? I would not mention that the studies have explored experiences in school that encouraged some to take up creative writing. That reference could bias the respondent to think that some school experiences should have encouraged him or her.

A "yes" to my question would mean the respondent would not continue with the survey, due to possible bias effects from his or her reading my research or references to it. Just as history effects (McMillan & Schumacher, 1997) could affect respondents in particular ways that adversely affect the internal validity of a study, bias effects could also adversely affect its internal validity.

The survey will focus attention on the covering theory, which will likely be a short, simple statement that "covers" the themes from the six studies. In fact, "a simple statement of the observed relations" (McMillan & Schumacher, 1997, p. 8) is preferable to a complex statement, and brevity is preferable to wordiness. Such a statement not only helps the reader understand the theory but also helps the survey avoid certain problems. For example, a lengthy theoretical statement might, if the survey asks respondents to read the statement and respond to its various parts with respect to their own experiences, open the door to recency and primacy effects (Krosnick, 1999).

Variety amongst respondents will depend on a non-random, stratified purposive sampling; some researchers might synonymously use the term "maximum variation sampling" (McMillan & Schumacher, 1997, p. 398); others, the term quota sampling (1997). McCaffery et al. (2003) "purposively selected [a] sample of women from four ethnic groups: white British, African Caribbean, Pakistani and Indian" to "examine[ ] attitudes to human papillomavirus (HPV) testing" through qualitative methodology (para. 1 [Abstract]). I will want to contact poets, fiction writers, and dramatists, likely in equal proportion. How many in each stratum? I'm not sure, but I will keep the number modest, in line with this statement: "It is better to do a small study well than a large study poorly" (McMillan & Schumacher, 1997, p. 299). But I won't keep the number too modest. Too few respondents may translate into my obtaining too little data (see, e.g., Cicero-Reese & Black, 1998) to address the truth of my covering theory beyond my six participants' experiences, although I should point out that some studies with small samples have yielded useful information (Decker, Bailey, & Westergaard, 2002).

At any rate, whatever the number of respondents, I will need to consider the implications to the study with regard to nonresponse (Trochim, 2002d). For example, will those from my sample who do not respond describe a group largely made up of individuals who believe no experiences in school encouraged them to become writers? That exists as a possibility; one Canadian novelist I spoke to told me that he felt no experiences in elementary or high school encouraged him to become a writer. Another possibility: Could some respondents describe a group discouraged (Birney, 1966) rather than encouraged by experiences in school to become creative writers? For me to find answers to these and other relevant questions may require my interviewing nonrespondents about why they did not respond to the survey.

Given the non-random stratified purposive sample of respondents for my study, each stratum will define a type of writer rather than a statistically representative sample of a type. I refer to McCaffery et al. again: "As is usual in qualitative research, participants were not selected to be statistically representative of their ethnic and socioeconomic group; rather, they were selected to represent a range of [participants]" (2003, Discussion, para. 6).

There exist many research directions I could take to survey creative writers and analyze their responses. I could take a quantitative approach, using statistical analyses of respondent's answers to Likert-type, dichotomous, nominal, and/or ordinal questions (Trochim, 2002b). Such questions, according to an interview guide or self-administered questionnaire, would define a structured format (Trochim, 2002h). I could determine statistical significance of the responses with regard to an analysis of omnibus correlations--shotgun style (McMillan & Shumacher, 1997)--of variables defined by questions based on various parts of the covering theory, or, for a more first-order (Lukiv, 2004a; and Pogson et al., 2002) analysis that tosses out reference to a covering theory, of variables defined by questions based on the themes from the six studies.

I could, on the other hand, follow a qualitative approach that analyzes respondents' answers to open-ended questions that allow them to describe relevant experiences. Many researchers unable to randomize from a sampling frame consider qualitative survey data to gather information, not for the purpose of generalizability, but for direction, or for the sake of new insights. Some researchers design a survey to allow for open-ended responses (Trochim, 2002i) to specific questions, and refer to this type of survey as semi-structured (2002i). Even some mail-in surveys have used this open-ended format (see, e.g., Alvarez, & Deverell, & Penn 2003).

Open-ended questions, although more avante garde than traditional closed-ended types (true and false, Likert, multiple choice, ranking) can provide relevant, reliable, valid information (Krosnick, 1999). This information can breath humanity, a sense of real people with real thoughts and feelings, into a study (see, e.g., Alvarez, & Deverell, & Penn 2003). Abe, Talbot, and Geelhoed (1998) created survey-opportunities for respondents who had participated in a program designed to help international undergraduate and graduate students adjust to university to open-endedly talk about "their experiences with campus life" (p. 542). These opportunities provided valuable information (1998). Wollin, Dale, Spenser, & Walsh (n.d.) used open-ended prompt questions to gather information about people with multiple sclerosis (MS) and their family members. For example, "the information about MS I would like to receive now is..." (Study Design, para. 2). Miller, MacKeigan, Rosser, and Marshman (1999) qualitatively surveyed patients with respect to their demands affecting physicians' prescriptions, using an open-ended forum.

Krosnick (1999) refers to "a number of recently rediscovered studies [that] found that the reliability and validity of open-ended questions exceeded that of close-ended questions (e.g. Hurd 1932, Remmers et al. 1923)" (1999, Questionnaire Design, para. 3). In the name of reliability and validity, I would need to code comments to open-ended questions carefully (Developing a Survey, n.d.). Likely, I would group codes to form themes. Themes could be compared or contrasted to the covering theory. New insights could lead to extensions or modifications or contradictions that lead to modifications of the theory (Glaser, 1992; and Glaser & Strauss, 1967). Findings, descriptive in nature, would "explore relationships...in an explanatory way" (McMillan & Schumacher, 1997, p. 296).

The survey will require rigorous groundwork. That means I must 1) establish face and content validity (McMillan & Schumacher, 1997); 2) consider focus group input (1997) ; 3) pre- and field-test the survey (Ferber et al. et al, 1980c; and Ferber et al., 1980a); 4) consider survey (Trochim, 2002e) and researcher, even subject, or respondent (McMillan & Schumacher, 1997), effects; 5) standardize the survey form within a particular mode of delivery (1997); 6) discuss reliability concerns (1997); and 7) decide on a methodology for analysis and interpretation (1997). Studies that do not address these points draw criticism. For example, Keith's (n.d.) study about "an administrative look at the ramifications of accommodating various departments' views of how administrative literacy is to be defined" (para. 1) uses a survey instrument, but does not discuss points "1" to "5." Morasco (n.d.), however, discusses pilot-testing his character evaluation survey, thereby establishing at least one element of the survey's internal validity.

The preceding paragraph describes the need for "rigorous, systematic, and objective methodology to obtain reliable and valid knowledge relevant to education activities and programs" (Eisenhart & Towne, 2003, Changes over Time Based on Public Input: Scientifically Based Research Standards, Ai). My goal: "to build systematically on the findings of [my] research" (2003, Bv).

Unlike studies that use already established surveys that publish explicit information about content validity and reliability (see, e.g., Comley & Banks, 2000; Forsythe, 2004; French, n.d.; and Miller, 1997), my study will require discussion about my self-constructed survey's validity and reliability issues. I should add, however, that established, commercially available surveys do not necessarily preclude problematic validity or reliability issues (McMillan & Schumacher, 1997).

Point 1) Face and Content Validity

I will need to determine whether the survey truly studies respondents' experiences in school that encouraged them to take up creative writing (Palmquist, n.d.c). I could establish face validity by presenting my survey and methodology to a researcher familiar with survey research, to confirm that the survey appears reasonable in its purpose and design (Palmquist, n.d.d). Content validity (Reliability and Validity, 2003) should logically establish itself.

Let me explain: I will present the covering theory to respondents, preparing them to make their statements of convergence (some or all of their experiences, if any exist, that encouraged them to become creative writers agree with one or more parts of the theory) and/or divergence (some or all of their experiences, if any exist, that encouraged them to become creative writers contradict one or more parts of the theory). Because the theory will have already passed the rigors of Study VII, designed to ensure validity, and because convergence and divergence "reflect[ ] the specific intended domain of content" (Palmquist, n.d.e), namely similar or "contradictory" experiences in school, content validity should naturally assert itself; however, just as researchers ask appropriate colleagues/others to verify that content validity exists in their studies (see, e.g., Hanson, Randolfi, & Olson-Johnson, 2002; Howard & Weiler, 2003; and Arnold & Ketter, n.d.), I will, in the name of modesty (New World Translation, Proverbs 11:2) , likewise do the same.

Point 2) Focus Group Input

At the pre-field stage, respondent focus groups allow the researcher to gather information about a topic (How to Conduct Pretesting, 1997). My six studies will have already established themes of relevance that define the phenomenon (the topic) of lived school experiences that have encouraged the studies' six participants. Study VII will have established a covering theory that defines the phenomenon. Therefore, I will not require a respondent focus group.

Point 3) Pre-Field- and Field-Testing

Still at the pre-field stage: Some researchers employ cognitive pretesting (Krosnick, 1999), referred to by some researchers as cognitive laboratory interviews (How to Conduct Pretesting, 1997). Respondents, in face-to-face interviews, describe verbally "whatever comes to mind as they formulate responses" (Krosnick, 1999, Pretesting, para. 4) to the survey. Using formal, hermeneutic phenomenological methodology, my six studies will have already described "whatever comes to mind" for the six participants as each responded to my research question: "What, if any, lived school experiences encouraged you to become a creative writer?"; however, with respect to the survey, cognitive pretesting could reveal what problems respondents have "with the questionnaire" (How to Conduct Pretesting, 1997, p. 4). The researcher would record field notes, during the interviews, which detail such problems. Contact summaries and self-addressed memos, describing the researcher's observations, conclusions, and concerns, would provide him with direction about modifications needed in the survey.

In recent years, pretesting (pre-field- and field-testing) has new direction beyond conventional, simplistic pretesting (Krosnick, 1999), in which researchers' "impressions of the respondents' experiences in answering the questions" (1999, Pretesting, para 1) tend to vagueness. Researchers can lack concrete knowledge "about what went on in respondents' minds when answering questions" (para. 2). Behavior coding (1999; see, also, How to Conduct Pretesting, 1997), part of this new direction, would be a valuable tool to help me analyze transcriptions of a number of face-to-face taped survey interviews at the field-testing stage. The transcriptions could reveal parts of the survey that respondents find confusing. Krosnick (1999) points out that "Presser & Blair (1994) demonstrated that behavior coding is quite consistent in detecting apparent respondent difficulties and interviewer problems. Conventional pretesting also detects both sorts of problems, but less reliably" (Pretesting, para. 5). Behavior coding addresses "the respondent's ability to form an adequate response" (How to Conduct Pretesting, 1997, p. 5)--and highlights problems that both respondent and interviewer might have with particular questions (1997). Contact summaries, memos, and field notes would help me focus on "respondent difficulties and interviewer problems."

Another pretesting tool, respondent debriefing (How to Conduct Pretesting, 1997), like other elements of pretesting, helps establish internal validity. As part of the field-testing, respondent debriefing helps the researcher "determine whether concepts and questions are understood by respondents in the same way that the survey [researcher] intended" (How to Conduct Pretesting, 1997, p. 6). An interview guide would assist the researcher, especially if he had "a clear idea of potential problems so that good debriefing questions [could] be developed" (p. 7). Field notes, memos, and contact summaries would offer further assistance.

My limited budget for the study (Ferber et al., 1980b) will dictate my conducting respondent debriefing, behavior coding, and cognitive pretesting myself. My budget will also dictate to what extent I will be able to use interview debriefing, which refers to third parties' querying interviewers about problems they see in the survey (How to Conduct Pretesting, 1997). In view of my budget, I could modify interview debriefing, turning it into a self-debriefing. Also, I could use a peer debriefer, a colleague, to query, interview debrief, me about problems I see in the survey. In both self- and interview debriefing, an interview guide along with field notes, contact summaries, and memos would enable me to clarify questions and direction in the survey that still need reworking. In the end, pretesting, likely ending with interview debriefing (or self-debriefing), should help minimize the number of confounded results.

Point 4) Survey, Researcher, and Subject/Respondent Effects

Many effects in survey research confound results. Not surprisingly, then, considerable criticism exists in the world of surveys (Deary et al., 2002). Therefore, I must fully address these effects, thereby enhancing the survey's validity. Pretesting enhances the same; in fact, "3" and "4" overlap, in view of the fact that pretesting attempts to root out survey effects.

Surveys that use unbiased questions (Ferber et al., 1980b; Trochim, 2002c; and Trochim, 2002d), referring to an absence of "leading [or slanted] questions" (Kalsbeek, 1998, p. 10), and that avoid "unclear...wording" (p. 10; also see Developing a Survey, n.d.), enhance internal validity by helping to minimize survey effects. Also, because the order of questions may introduce bias (Ferber et al., 1980b; see, also, Stinson, 1999) and because long surveys wear out respondents, confounding results (1980b; see, also, Stinson, 1999), order effects and excessive survey length must be addressed.

Another word on bias: Although French (n.d.) ranks as an undergraduate researcher, she accents her own researcher-credibility by discussing one threat to the internal validity of her study: intersubject bias. She uses a commercially available survey--Cheney's 25-item Organizational scale (1983). She defers reference to the survey's effects to its maker, which is a little disappointing, but her addressing threats to the internal validity with reference to the physical circumstance of respondents completing surveys "near one another" (French, n.d., p. 22) shows good sense on her part.

For my survey, respondents will need to access long-term memory of events in school. Although the work of such recall could discourage some respondents from bothering to complete the survey (More About Mail Surveys, 1997), my appeal to their diligence in the name of pedagogy and my study's encouraging a new generation of creative writers may provide incentive. The personal relevance of the survey, in terms of what school-related events encouraged respondents to become creative writers, should also provide incentive (Stinson, 1999). Hence, positive subject effects may assert themselves in respondents who find walking down memory lane to answer the questions exciting and highly motivating. This novelty effect (McMillan & Schumacher, 1997) may actually increase internal validity as respondents diligently search their memories for relevant experiences.

Problematic subject, or respondent, effects (McMillan & Schumacher, 1997), however, may arise through comments in the semi-structured section of the survey if respondents create, rather than simply remember, events. To keep the study, then, out of a folklore domain (Borgatti, 1996b), I will ask respondents to make sure that responses truly reflect their experience(s), encouraging them to contemplate response-accuracy through their reporting facts/details. This request focuses not on "generalization or opinion" or imagination, but on "respondents' accounts of actual events" (Miller et al., 1999, Methods, para. 3). Just as the interview guide for my phenomenological studies helped participants remember "actual events," a thoughtfully constructed semi-structured section could likewise help respondents.

Problematic researcher effects--which, in the following case, could also be called survey effects--should be absent if the survey does not reveal any of my biases. Also, if I use e-mail surveys, which certainly would fit into my parsimonious budget, researcher effects such as an interviewer/researcher could inadvertently create through gestures, voice inflections, and facial mannerisms would not exist. McFadden and Winter (2001) speak about "social interactions [which can create researcher effects] lead[ing] to biases in responses to survey questions" (p. 1). Additionally, e-mail, in terms of the researcher removed from the respondent, should reduce the possibility of subject effects such as social desirability and the Hawthorne effect (McMillan & Schumacher, 1997), even reducing the effects of the Heisenberg Uncertainty Principle (Wassermann, 2002). E-mail surveys, then, have some advantages over person-to-person surveys.

Researchers who do not explain their efforts to minimize survey, or other relevant, effects (see, e.g., Bowman et al., 2002; Chairs et al., 2002; Comer, 2002; Haugen, 2001; Mark et al., 2002; and Rohs et al., 2002) invite criticism for their studies (see, e.g., Center for Child Care Workforce, 2000). In a graduate course in research methods that I took at the University of Northern British Columbia, Paul Madak, then chair for the education department, commented on how, surprisingly, even established researchers sometimes publish articles that do not address validity issues (see, e.g., Sšderfeldt, Sšderfeldt, & Warg, 1995). I want to address validity issues to ensure the survey "holds water"--to ensure I am researching what I say I am researching (McMillan & Schumacher, 1997), namely the truth of the covering theory for a sample of creative writers. Naturally I plan to sufficiently discuss internal validity in Study VIII.

Point 5) Standardize the Survey Form Within a Particular Mode of Delivery

I will need to consider the appearance of the survey, along with the appearance of the letter of introduction--of transmittal (McMillan & Schumacher, 1997)--and follow up letters to respondents. A professional appearance (Trochim, 2002f) invites respondents to take surveys seriously. Of course, the survey, letter of introduction, and follow up letters must be standardized.

As for the mode of delivery: I have already referred to e-mail (Ferber et al., 1980c) and its advantages in terms of economy, minimizing researcher and subject effects, and ease of contact. As an independent researcher, I lack university or corporate sponsorship (Ferber et al., 1980b). Although I could possibly afford small financial incentives for individuals who actually respond to the survey (Trochim, 2002d), a customary practice amongst researchers who use surveys, there exists an awkwardness in my using e-mail for contact and survey transmittal and using the post for mailing a financial incentive. Also, if I appeal to the respondents in the "elevated" name of pedagogy, introspection, and my study's encouraging a new generation of creative writers, a nominal "incentive" (for example, a toonie: $2.00 CANADIAN), may insult more than motivate.

Other modes of delivery: telephone (see, e.g., Scarpellini, n.d.; and Annie E. Casey Foundation, 2003), post (More About mail Surveys, 1997), and face-to-face interviews (Krosnick, 1999; also Stinson, 1999). Unlike other researchers, however, who can afford several interviewers (see, e.g., McElrath & McEvoy, 2001), I will not be able to afford any.

Pros and cons exist for any mode of delivery, just as pros and cons exist for any research methodology (McMillan & Shumacher, 1997). Although I have not yet chosen conclusively what mode of delivery I want to use, I must admit that e-mail looks inviting for several reasons I have already mentioned. E-mail looks inviting for other reasons: it allows for lengthy responses, allows respondents time for thoughtful responses, and it lends itself to a quick turnaround time (Trochim, 2002g).

Point 6) Reliability Concerns

The reliability of my survey's structured section could be established by my administering and re-administering it to (testing and re-testing it on) a sample of creative writers. I could compare the second set of responses for each respondent to the first set (Palmquist, n.d.a; also Reliability and Validity, 2003). Granted, test/re-test data draw into question whether the respondent has changed (diachronic reliability) over the time between tests (Reliability and Validity, 2003). Some program evaluation surveys deliberately study skill or knowledge development through pre/post assessment (see, e.g., Curriculum: Program Evaluation, n.d.). Skills or knowledge can change over time (Palmquist, n.d.f), with regard to history or maturation effects (McMillan & Schumacher, 1997); however, my survey will not test such things. My survey will explore memories, which, I predict, due to their relevance to what encouraged respondents to become writers, likely won't change.

My test/re-test process could generate a reliability stability coefficient (McMillan & Schumacher, 1997). McMillan and Schumacher (1997) recommend establishing reliability "before the research is undertaken" (p. 243) through "individuals who are similar to the subjects in the research" (p. 244). I could contact established creative writers, administering the survey on two different occasions, arriving, through quantifiable measures, at a reliability stability coefficient.

My peer debriefer (McMillan & Schumacher, 1997) would not also need to make a comparison between the test and re-test responses for each person, searching for discrepancies between his tallies and mine, in terms of interrater reliability (Palmquist, n.d.b). Tallies of answers for the structured section would not reflect subjective totals, but, rather, objective ones; therefore, an interrater reliability quotient would not be necessary.

For different reasons, a reliability quotient would not make sense for a semi-unstructured section. For example, a semi-structured question could ask, Are there any examples of experiences in school that encouraged you to become a creative writer? If yes, could you describe them in detail? A respondent might present different events on different occasions; however, such a difference does not imply unreliability. In my phenomenological interviews (Studies I, II, and III), I probed and probed until the participants ran out of experiences to talk about. If I administer the survey through e-mail, I won't be present to relentlessly probe respondents to think about their experiences. If I were able to probe and probe, no doubt a partial list of experiences would become a complete list. A partial list would translate into a reduced number of themes rather than unreliable themes.

I wish to note that I doubt that any two respondent's expressions would contradict each other. I have not to date encountered contradictory statements in any of my study's phenomenological interviews. To err on the side of caution, however, I could nevertheless look at reliability as a function of contradictory statements. Contradictory statements in the semi-structured section could define a level of unreliability; however, a coefficient of reliability may not make mathematical sense. A discussion of the semi-structured section--certainly it would be a challenge "to summarize and interpret" (Interviews, n.d., Disadvantages, bullet 3)--could reveal new insights, even an extension or modification of the theory (Glaser, 1992; Glaser & Strauss, 1967; and McMillan & Schumacher, 1997), given that follow up questions give respondents an opportunity to explain the nature of contradictory statements.

One aspect of reliability remains that I would like to mention: quixotic reliability (Palmquist, n.d.f). A structured section for the survey could follow the following logic: Each respondent could state for each part of the covering theory (assuming I will be able to divide the theory into parts according to an inherent logic) whether it 1) agrees with any of his or her "encouraging" experiences (convergence), 2) contradicts any of his or her "encouraging" experiences (divergence), 3) leaves him or her undecided, or 4) represents experiences he or she never had in school. That logic should eliminate quixotic issues.

For example, 1) Would a respondent who finds a statement agrees with his or her encouraging experiences label it 2, 3, or 4? I don't think so. 2) Would a respondent who finds a statement contradicts his or her encouraging experiences label it 1, 3, or 4? I don't think so, unless a respondent answered 4 because he or she confused divergence with experiences he or she never had. The survey, then, would have to help respondents "double-think": Divergence means the respondent had encouraging experiences that related to the part of the theory in question, but that contradicted the theory; therefore, divergence means contradiction (2), not no-relevant-experiences (4). Quixotic issues, then, should not arise from 1 or 2.

Next: 3) Would a respondent quixotically record "undecided"? Perhaps a respondent could impatiently or otherwise erroneously conclude that because he or she did not have related experiences in school (4), the answer would be undecided (3).A lucid focus on what "undecided" means should eliminate such a problem. Again the survey would need to help respondents double-think: Undecided means the respondent cannot decide if relevant encouraging experiences were convergent (1) or divergent (2) to the part of the theory in question, and does not mean the respondent had no related experiences (4). If the respondent had no related experiences, then the respondent had no related experiences (4). If, on the other hand, the respondent had related experiences, then the question would be, what kind? Convergent? Divergent? Or would the question leave the respondent undecided? Such double-think should help route out quixotic responses.

Finally: 4) Would a respondent quixotically find the different parts of the theory non-representative of his or her experiences? Each of my three participants (Studies I, II, and II) were not at any loss to recall specific events that had encouraged him or her. Any respondent who was encouraged by experiences in school should readily recall what they were and whether they represent convergence (1) with the part of the theory in question. I see nothing quixotic therein. Also, if a respondent determines that a part of the theory contradicts, or diverges (2) from, his or her encouraging experiences, then likely that determination is valid. I see nothing quixotic therein either.

Quixotic issues should not exist for the open-ended nature of the semi-structured section either. Respondents would simply detail any relevant experiences. I see no similarity here to the classic example of the researcher asking, "How are you doing today?" and the respondents typically saying the same clichŽ: "Fine" (Palmquist, n.d.f). That's quixotic. Experiences relevant to my research question, however, as shown in Studies I, II, and III, are unique and specific, and easily recalled without quixotic overtones. Really, quixotic issues should be eliminated through my addressing survey effects and my conducting pre-field- and field-testing (the pretesting stages).

Point 7) Methodology for Analysis and Interpretation.

A semi-structured section of the survey could ask each respondent to Describe, if any exist, school experiences that encouraged him or her to take up creative writing. This section of the survey would allow for respondents to comment in their own words.

A qualitative analysis of comments for the semi-structured section could draw on hermeneutic phenomenology's search for themes (see, e.g., Lukiv, 2002c, 2004a, Methodology)--assuming, of course, that themes arise from the data. Bracketing out bias, as in my phenomenological studies, would protect the integrity of my analysis. A peer debriefer would evaluate whether any of my bias appeared to drive the analysis. Interpretation (which refers to the process of free imaginative variation), in terms of establishing essential and tossing out incidental themes, too, would likewise benefit from peer debriefing. Traditional, person-to-person participant review, which would confirm that themes from any respondent's comments truly reflect his or her experience, however, could prove to be impractical, especially if the sample were large in number and respondents lived a distance from my home.

If I conduct the survey through e-mail, perhaps a modification of the face-to-face/researcher-to-participant sort of participant review could be replaced by back-and-forth e-mail communication between myself and respondents. Such a process, however, would limit the size of the sample, in view of the considerable amount of time participant review and free imaginative variation generally require. I could work with, say, a sub-sample of five respondents at a time, repeating the survey over and over with new sub-samples of five, until I have a total sample of, say, 50. In view of the intensive work of my analyzing and interpreting the open-ended responses of even just five respondents, in the context of Bogdan and Bilkan's (1992) advice that single-participant studies are more manageable than those with multiple-participants, I would require a lot of energy, time, determination, and, I think, patience. Study VIII may take a long time for me to complete, perhaps years.

 Final interpretation could describe validation, with regard to themes describing "encouraging" events that are convergent to part or all of the theory, or contradiction, with regard to themes describing "encouraging" events that are divergent to part or all of the theory. Contradiction would not imply the theory is wrong, but rather that it requires modification or extension (Glaser, 1992; and Glaser & Strauss, 1967). On the other hand, themes could express neither convergence nor divergence, but may nevertheless shed new light on what events in school encouraged some students to become creative writers. These "new" themes could stand alongside the list from the six studies, as new direction for teachers, in the sense that if theme x describes events that encouraged respondent y, then class activities based on theme x may likewise encourage someone else. That last sentence essentially describes the extrapolating power of the first six studies. Of course, new themes could also extend or modify the theory (Glaser, 1992; and Glaser & Strauss, 1967).

The structured section could look at convergence and divergence. Tallies of convergence versus divergence versus undecided versus not-relevant-to-my-experience for each part of the covering theory would provide quantitative data about the degree of relevance or applicability the covering theory has for each respondent as well as for all respondents. I am not sure whether or not I will include a structured section. If a phenomenological point of view is good for Studies I to VI, and a good foundation for Study VII, then such a point a view may suffice for an analysis and interpretation of a semi-structured survey that simply asks respondents to concretely describe, if any exist, lived school experiences that encouraged them to become creative writers. I will think about this last sentence in the days ahead.

Academic Exchange Extra invites reader response to any writings in this issue--especially articles advancing the scholarly debate of issues raised.

Copyright © Academic Exchange - EXTRA
- Web Editor

------------------------------  Page Citation Reference: AE-Extra. (2005).  AE-Extra. November. Available Online. [URL: < >. Created: 28 October 2005. Updated: --. Accessed: ]