Data Analysis

This section explores each of the data sets acquired and outlines how the rubric was used on the student data to obtain quantitative information of the quality of the writing.

Quantitative data obtained using the rubric are available upon request, or by accessing the wiki (prior permission required).

To demonstrate how the rubric was used in scoring each paper, the following excerpts from student writing exemplify meeting the requirements of a particular rubric criteria to varying degrees of success, with a short justification of how it was scored.

Sample Writing:

The problem presented to our team involved a set up of two charges of equal magnitude +Q. These charges were arranged a distance of D in the opposite directions from the origin along the x-axis. The objective was for our team to calculate the electric potential at any point on the x-axis so long that |x|»D. We were then meant to use the approximations to predict the behavior of a test charge placed anywhere along the x-axis.

Evaluation:

This received a score of 3 (Very Good). The author clearly described the physical set up of the problem (value of charges and positions of the charges), followed by a brief explanation of what he/she would be solving. The author could have defined exactly what “calculating the electric potential” meant (students were specifically asked to approximate V(x) using a fourth-order series expansion). If there were a higher score available, this would have distinguished between the two. As it stands this is a succinct, descriptive explanation of the problem being solved.

Sample Writing:

The general problem assigned was to determine, to the fourth order, a series expansion of the equation for the electric potential (in two dimensions, specifically the xy-plane) of essentially a dipole system with varying parameters. The conditions that remained the same from case to case were that the two charges were situated a distance apart of 2D, lying along the x-axis, with the axial point of the dipole lying at x=0. The conditions that differed from one situation to another were: both charges being either positive, or of equal yet opposite charge value; and that the electric potential was being determined for V(x) (sic), where |x|«D or |x|»D, or for V(x,y) (sic),

where x=0 and |y|«D or |y|»D. The specific conditions of the situation which was asked to be solved were those where both charges were positive and the electric potential was determined for V(x,y), where x=0 and |y|«D.

Evaluation:

This received a score of 2 (Fair). Though the author has set up the physical system, and defined the questions he/she will be answering in the rest of the paper, there is a large amount of extraneous information and the wording is often unclear. When the author writes, ”…of essentially a dipole system with varying parameters,” it seems like the system is not actually a dipole (the author probably intended to distinguish this system from a point dipole), and that the system itself has varying parameters (the author meant to describe that there were various different problems being asked of different groups, which he/she goes on to explain). The author went on to identify all of the varying similar physical problems that other groups would solve, only identifying his/her own in the final sentence. This makes identifying the problem being addressed specifically by the author a more difficult task than it should be.

Sample Writing:

1) Also important, will be to find a way in which to expand the final equation for potential as a series expansion, or in other words, to recognize a similarity between our equation for potential and a previously known power series expansion formula.

2) The importance of the resultant series expansion can be appreciated and better understood when compared to the resultant expansions for each other scenario.

3) Symmetry and anti-symmetry, in brief terms, are defined by the relations f(-x)=f(x), and f(-x)=-f(x), respectively. For a physical example, one can think of a sphere, which is symmetrical about all three axes of Cartesian space; unlike a pyramid, which is symmetrical only about the z-axis of Cartesian space, and anti-symmetrical about the xy-plane.

Evaluation:

The three excerpts above came from the same paper, receiving a score of 1 (Poor) in this criteria. The paper had many components similar to the excerpts above that demonstrate poor professional judgment to detail. In the first excerpt, the writer should have omitted everything after “as a series expansion,” because the second part of the sentence is redundant, if not somewhat confusing. The second excerpt is an example of a statement which is not necessarily true, as a series expansion could be appreciated and fully understood on its own without comparison to other scenarios. The author probably meant to write that it would be interesting to compare the results to other scenarios. The final excerpt is a example of the author of too much information which is already obvious to the reader. The author could have stopped after defining the mathematical relationships Furthermore, a pyramid is not symmetric about the z-axis. The author probably had a misconception about what axial-symmetry meant, or perhaps meant to say cone and not pyramid.

Sample Writing:

V(x) = kQ/(x-D) + kQ/(x+D)

Obviously it is necessary to use the equation V(x)=kQ/x to find the approximation for the potential. In this problem it is necessary to divide the formula into two parts using the super position principle: V(x) = kQ/(x-D) + kQ/(x+D) …where V(X) is the potential with respect to position x, k represents Boltzmann constant, Q represents charge and D represents distance from the origin on either side of the y-axis. The reason it is necessary for a two part equation is so that each charge is represented. The symmetry of the situation allows one formula to represent both sides of the axis because the charges are identical in value and in distance from the origin.

This paper earned a score of 1 (Poor) for criteria 3. The author did explain the terms within the equation he/she has referenced in this excerpt, however, the equation used was itself not the original equation for finding electric potential, and k is not the Boltzmann constant. Furthermore, the distances in the denominators should have absolute values. Finally, the last sentence is erroneous; the superposition principle does not require that the magnitudes of the distances between an axis and a charge be identical. This sentence seems to imply, incorrectly, that given charges need to be of equal magnitude and be the same distance from an axis to allow for one equation to describe electrostatic potential.

Sample Writing:

One of the physical concepts necessary to solving the problem is that of the superposition of potential, due in part to the fact that electric potential is not a vector quantity and can simply be summed together considering the relation V=SUM(Q_i/(4*Pi*Epsilon_0*abs(r-r_i))… [on a separate page of calculations the author begins with] V=SUM(Q_i/4*Pi*epsilon_0*abs(r-r_i)) V = Q_1/4*Pi*epsilon_0*r_1 + Q_2/4*Pi*epsilon_0*r_2 [the author included a diagram explicitly labeling r_1 and r_2, as well as Q_1 and Q_2]

Evaluation:

This paper received a 1 (Poor) for this criteria. The author correctly identified the equation necessary to solve the problem, but did not describe what physical quantity each variable or constant represented. Furthermore, the magnitudes of r_1 and r_2 were presented in the diagram, but the author did not describe how he or she had explicitly introduced these magnitudes in the equation. Ultimately, the author made no attempt to describe the equation in words.

Sample Writing:

The next step involved using the familiar power series:

(1+z)^p for |z|<1 to find the fourth order approximation for V(x). In order to accomplish this, we needed to put the denominators in the form (1+z)^p:

[here the author factors out an x from each term]

Now considering the |D/x|< 1 because we already know that |x|»D we now have the power series in the correct form. The series' can then be expanded as follows:

[here the author does the series expansion for (1+D/x)^-1 and (1-D/x)^-1]

Combining these two fourth order approximations yields: [here the author sums the two expansions together] Now by adding in the constants that were set aside we completed the approximation:

[the author writes the final solution to fourth order accuracy]

Evaluation:

This paper received a 3 (Very Good) for this criteria. Each mathematical step was explained and, when necessary, justified. The important things for students to note in this activity was what he/she needed to do in order to expand the function in a series. The author correctly identified and explained factoring out an x, since the set-up of the problem had |x| » D.

Writing Sample:

By observing the approximation it can be concluded that a positive test charge would be accelerated towards +infinity if located at +x or -infinity if at -x. Noting that there is an x in the denominator is [sic]

can be concluded that as x approaches infinity, the electric potential approaches zero, thus the power series converges for |x|»D. This equation will only work for |x|>D and breaks down otherwise.

Evaluation:

This excerpt represents the entire conclusion section of a student's paper. Because there isn't any mention of what was learned or which insights were gained in solving the problem, this paper received a score of 1 for this criteria. To receive a score of 3 in this section, the author should have reflected on using a power series to make an approximation, and what he/she needed to do to apply a power series to the equation.

Writing Sample:

By observing the approximation it can be concluded that a positive test charge would be accelerated towards +infinity if located at +x or -infinity if at -x. Noting that there is an x in the denominator is can be concluded that as x approaches infinity, the electric potential approaches zero, thus the power series converges for |x|»D. This equation will only work for |x|>D and breaks down otherwise.

Evaluation:

This conclusion section did an excellent job at discussing what the final results told about the physics of the problem, receiving a score of 3 in this criteria. The author clearly described what would happen if a test charge were introduced at various locations, taking the limit as x grew infinitely large, and discussing the situations under which their solution is valid.

In stage 2 of the project, students submitted scores for three scientific writing examples based on an in-class activity they had completed. The project team (including the instructor, doctoral students, and the collaborating undergraduate senior) also scored the examples using the same rubric and evaluation form. This provided some data as to how well students could adhere to the rubric when evaluating scientific writing. Root-mean-square calculations were carried out using the data obtained. Using these calculations, tt was found that student evaluations of the project examples were not in very good agreement with project collaborator evaluations. Calculating the root-mean-square of the difference between the average of the students' scores and project collaborator scores, it was found that for each example paper there was an RMS value of 0.72-0.9 (a score of 0 would indicate perfect agreement with collaborator evaluations). Averaging RMS differences between each student's evaluation and the collaborator evaluation scores across all criteria for all papers yielded a median RMS of 0.64 with a standard deviation of 0.23. Essentially, student scores were usually in disagreement with collaborator evaluations, and varied widely in how they disagreed.

In stage 4 of the process, students uploaded their evaluations of their peers' papers. Ideally, students were to score three other papers so that every student in the class would have three unique peer reviews on which to reflect. There were two problems that occurred during this stage of data collection, however. The first problem was that of student participation. Not all students submitted three peer reviews. Furthermore, there was an error with the online submission form, and an unknown amount of peer reviews were lost before the error was detected and corrected. Most students in the class still received at least one peer review from another student.

There was a consensus amongst students that content organization, amount of detail to include, and mathematical explanations improved from the first writing sample to the final writing sample. Of the 17 reflections received, 12 students identified improvement in their attention to detail, 6 noted improvement in their papers' organization, and 6 noted that explaining their mathematical steps improved.

There were 3 students who reported improvement in understanding of the concepts, 2 students who believed they improved in their ability to analyze their solution, and one student who reported an improvement in the ability to express their ideas. One student reported that having to explain things in words had solidified their understanding of the physics of the problems being addressed.

When identifying the factors that had lead to changes in their writing, 7 students reported that peer evaluations had helped improve their writing. The next most reported cause of change (4 students) was the repetition, or practice of writing again and again.

There were two students who reported a better understanding of the physics as being a reason for why their writing had improved. Two students also reported that the in class discussions were important to explaining how their writing had changed. There were also two students who identified the rubric as being an agent of change in the quality of their writing. One student felt that the guiding questions had helped improve their writing, and one other student cited working on the in-class activities had improved their writing.

There were two students who issued criticisms of the process, each identifying a lack of instructor feedback on their papers as a hindrance to improvement in the quality of their writing. This is an important criticism because an important aspect of the writing unit developed in this project is that it should require minimal load on the instructor.