Jamie Hale

Jamie Hale

Thursday, January 11, 2018

Peer Review is not the Antidote



 In the Peer Review Process a paper is submitted to a journal and evaluated by several reviewers (often reviewers are individuals with an impressive history of work in the area of interest-that is, the specific area that the article addresses).  After critiquing the paper the reviewers submit their thoughts to the editor.  Then, based on the commentaries from the reviewers, the editor decides
whether to publish the paper, make suggestions for additional changes that could lead to publication, or to reject the paper. 

Single Blind and Double Blind Reviews

In Single Blind Reviews authors do not know who the reviewers are.  In Double Blind Reviews authors do not know who the reviewers are, nor do reviewers know the identity of the authors.  In many fields Single Blind Reviews are the norm, while in others Double Blind Reviews are preferred.
“Peer review is one way (replication is another) science institutionalizes the attitudes of objectivity and public criticism.  Ideas and experimentation undergo a honing process in which they are submitted to other critical minds for evaluation.  Ideas that survive this critical process have begun to meet the criterion of public verifiability” (Stanovich, 2007, p. 12).

Criticisms Peer Review Process

Reviewers find it hard to remain Purely Objective due to their own education, experience and biases

The process is slow Critics point out, and this may deter submission of quality papers  

There are many examples of poor research published in peer-reviewed journals, which indicates the peer review process is often unsuccessful in preventing the publication of  bad science

Sometimes good research is not published, especially when findings are not statistically significant.  Publication bias is problematic and demonstrates confirmation bias

Reviewers are not always knowledgeable regarding contents they are reviewing (I have been asked to review papers that consist of contents that was outside my area of knowledge)

Lack of agreement and communication among reviewers

Reviewers tend to be highly critical of articles that contradict their own views, while being less critical of articles that support their personal views (example of myside bias). Well-known, established scientists are more likely to be recruited as reviewers

Lacks standardized criteria, and criteria for publication demonstrates huge variability among scholarly publications.

Final word-Peer Review Process
 
The Peer Review Process is not perfect, but some researchers suggest  it is one of the best safeguards we have against junk science (Stanovich, 2007). When evaluating the worth of scientific data, in addition to whether it is published in a peer reviewed journal, it is important to take into consideration:  funding sources, study replication, study design, sample size, conflicting interest, sampling error, different measures of reliability and validity, reporting limitations and other possible criticisms of study (special concern with *statistical validity- often not acknowledged or understood). 

There are good studies that never get published in peer review publications, and low quality studies are published by peer review publishers.  It is erroneous to label a study, review, commentary, meta-analysis or any other scholarly papers as high quality based solely on peer review status. This over glorification of peer review pervades academia and pop science.      

* When researchers question a study’s statistical validity they are questioning issues relevant to how well the conclusions coincide with the results, represented as statistics. Interrogating statistical validity may include some of the following questions: If the study found a difference what is the probability that the conclusion was a false alarm?  If the study’s finding found no difference what is the probability that a real relationship went unnoticed?  What is the effect size?  Is the difference between groups statistically significant? Are the finding practically significant? What type of inferential stats were used to assess predictions? Could different statistical procedures have been used? How would different samples influence statistical findings?  Stats make use of samples; inferences about the population are derived from data collected in the study.  It is important to avoid an exaggeration of the findings, consider sampling error, consider external validity and consider the need for converging evidence, and what the finding indicate regarding a representation of the population.      

References available upon request  

Wednesday, December 13, 2017

The Sugar Brain is not Unique



In the context of this article the Sugar-Brain refers to neural correlates regarding sugar consumption. Firstly, information is given on the classification of sugar, a brief mention of different categories of sugar, and the properties underpinning different sugars.  Secondly, a concise overview of brain reward mechanisms is appropriate.

The term sugar, in everyday discussion, generally refers to sweets or highly processed sweet tasting food. The term is excessively ambiguous, so when discussing in everyday conversation it is not always clear what one is referring to.  I usually assume the  referent is highly processed, calorie dense, sweet food, in most cases.
  
Sugar, in the context of science, is defined precisely and is dependent on various properties. Sugar (of all types ) is a carbohydrate.  The term carbohydrate was originally used to describe compounds that were hydrates of carbon; they have an empirical formula of CH2O. In my book- The Carbohydrate Files 2nd editon (2007)- I presented  two classification systems: Basic
Carbohydrate Classification System (BCCS) and the Advanced Carbohydrate Classification System (ACCS). A concise overview of the ACCS is presented here.
 
Carbohydrates are polyhydroxy aldehydes, ketones, alcohols, acids, their simple derivatives and
their polymers having linkages of the acetal (any organic compound formed by adding alcohol
molecules to aldehyde molecules) type. They may be classified according to their degree of
polymerization ((Polymerization is a chemical process that combines several monomers to form a polymer or polymeric compound.) and may be divided into three principal groups: sugars, oligosaccharides and polysaccharides. Sugars are the focus in the current article.

Monosaccharides, the basic units of a carbohydrate, are white crystalline solids that may be divided into aldoses, which have an aldehyde group on the first carbon atom, and ketoses, which typically have a ketone group on the second carbon atom. They may also be divided according to the number of carbon atoms that they contain. The suffix “-ose” is used to identify these molecules as sugars.

Triose: monosaccharide with three carbons
Tetrose: monosaccharide with four carbons
Pentose: monosaccharide with five carbons
Hexose: monosaccharide with six carbons
Heptose: monosaccharide with seven carbons

Sugars include monosaccharides, disaccharides, and polyols (sugar alcohols). The
monosaccharides are categorized as glucose, fructose, and galactose (formula C6H12O6). The disaccharides are composed of two monosaccharide units. The binding between the two
sugars results in the loss of a hydrogen atom (H) from one molecule and a hydroxyl group (OH)
from the other. The disaccharides are categorized as sucrose, lactose, trehalose, and maltose
(formula C12H22O11). Sugar alcohols, technically called polyols, are are found in numerous sports drinks, bars, and various other types of sweets. Polyols are generally known as sugar-free
sweeteners. They are carbohydrates, but they aren’t sugars. Chemically, polyols are considered polyhydric alcohols or sugar alcohols because part of their structure resembles sugar and part is similar to alcohols. However, these sugar-free sweeteners are neither sugars nor alcohols. The most widely used polyols are sorbitol, mannitol, and maltitol.

Proponents of the Sugar-Brain claim that consumption of sugar can activate the same brain reward mechanisms (dopamine pathway referred to as mesolimbic dopamine system) as the consumption of addictive drugs. This is true; so it is not a myth?  You have probably seen photos comparing the sugar-brain with the drug addiction-brain. Hold on before jumping to conclusions. The problem with the claim is the way it is presented (implications) and inferences made regarding the claim. The claim is a rhetorical device used to convince people that sugar consumption is bad, like drug consumption.
What is the mesolimbic dopamine system? Drug researchers have traditionally identified the mesolimbic dopamine system as the brain system mostly involved with drug addiction. This system may be extended to include cortical areas (in PFC)- mesolimbic cortical dopamine system (Kandel, 2012). Some sources refer to these systems as being the same; however, the mesolimbic dopamine system can be more accurately described as projecting from the ventral tegmental area to the nucleus accumbens (NA, often referred to as the major pleasure center in the brain), while the mesolimbic cortical dopamine system projects from the ventral tegmental area and extends to areas in the PFC. Their distinction is not important in the context of this article (either are involved with drug consumption and brain reward mechanisms- reward/pleasure circuitry). These circuits are rich in dopaminergic neurons. Dopamine cell bodies are located in the ventral tegmental area and substantia nigra. The focus here, is on the projection from the ventral tegmental area.

Objects, stimuli, activities or internal physical states can serve as rewards for humans and non-human animals. Rewards have positive value and facilitate feelings of pleasure and positive emotion; they act as positive reinforcers. Not only do rewards lead to the activation of dopaminergic activity, but so does expectations or anticipation of rewards. "[T]he flow of dopamine is set off by the simplest expectation of pleasure, even though the pleasure may not materialize" (Kandel, 2012, p.428).
The brain's reward mechanisms are activated when we enjoy art, experience beautiful scenery, are exposed to attractive faces, listen to pleasant music, are exposed to humor or novelty, drive a sports car and experience romantic love. The Sugar-Brain could easily be called the Love-Brain. 
 
Evaluating brain imaging is complicated, and there is often disagreement among those highly qualified regarding implications of imaging. If sugar consumption is not a rewarding act what can we expect regarding activity of brain reward circuitry? It is a drastic over-simplification to suggest that - because, consuming sugar may lead to activation (reiterating- the variability  in activation is large) of brain reward mechanisms- it should be held in the same regards as drug use. "Dopamineric neurons in the striatum respond to all kinds of pleasure." Eric Kandel- Nobel Laureate

Science sounding information is often inserted to enhance persuasive value.

References available upon request

Tuesday, October 3, 2017

Association Between Scientific Cognition and Scientific Literacy (Brief Review)



Association Between Scientific Cognition and Scientific Literacy: Implications for Learning  Science (Hale, Sloss, & Lawson, 2017)
Abstract
In the current research scientific literacy is synonymous with general scientific knowledge. This form of literacy is sometimes referred to as a form of derived scientific literacy. Scientific cognition is not the same thing as scientific literacy; scientific cognition involves multiple components and sub-components. At the very least scientific cognition involves philosophy of science, scientific methodology, quantitative reasoning  and logic. The primary interests in the study were whether or not scientific cognition and scientific literacy scores were associated, and whether or not there would be gender differences for total scores for each scale. The scientific literacy and scientific cognition assessment consisted of mostly questions  derived from measuring devices used in the past. The assessments were administered as part of an online survey. The participants were 202 university students. The study was approved by the university's Institutional Review Board. The results indicate a positive association between scientific literacy and scientific cognition, and no gender differences for total scores from the scales. Additional analyses indicate there was gender differences for some of the items. There was gender differences for one item from the scientific literacy assessment and for two items from the scientific cognition assessment. The research report includes a discussion regarding future directions for relevant research,  implications of learning science and limitations of the study.    

Full Research Report Available Upon Request (PDF File)


Discussion
            The results show a positive association between scientific cognition and scientific literacy. The association was moderate in strength. The differences between men and women for total scores on scientific cognition and scientific literacy were not significant. The results indicate an association between gender (men vs. women) and responses (correct vs. incorrect) for three items from the online survey; one of the items from the scientific literacy assessment and two of the items from the scientific cognition assessment.
            Scientific cognition and scientific literacy are measured on a continuum. Results from a study conducted by Drummond and Fischhoff (2015) show a positive association between the Scientific Reasoning Scale (SRS) and two widely used measures of scientific literacy, the Trend Factual Knowledge of Science Scale (TFKSS) and the Understanding of Scientific Inquiry Scale (USIS). The strength of the association was moderate,  similar to our findings regarding the association between scientific cognition and scientific literacy. The SRS assesses skills needed to evaluate scientific evidence; the scale consists of items related to research methodology. The TFKSS assesses knowledge of scientific concepts. The USIS assesses knowledge or research methodology and probability.   The scientific literacy assessment used in this study is similar to the TFKSS, as it is an assessment of knowledge of general scientific concepts. These scientific literacy scales have been used often in the field of public understanding of science (Alum, Sturgis, Tabourazi, & Brunton-Smith, 2008). The scientific cognition assessment is similar to the SRS and the USIS, as it involves questions regarding research methodology.  Similar to the USIS it also involves questions regarding probability (quantitative reasoning). In addition, the scientific cognition assessment involves items that require knowledge in the philosophy of science.
            In contrast to the finding that total scores, for men, on a general scientific knowledge test were better than for women (Sloss & Hale, Paper Forthcoming) we found no significant differences. Also, there were no differences between men and women for total scores on the scientific cognition assessment. There were significantly different scores between men and women for one item from the scientific literacy assessment and two items from the scientific cognition assessment. Men scored better on one item from the scientific literacy assessment and one item from the scientific cognition assessment. The item, question no.9, for which men scored better from the scientific literacy assessment involved a chemistry question. Question no. 9 was "[w]hich of the following are smaller than atoms a) proteins b) electrons c) amino acids." The correct answer is b. Past research indicates a difference in scores for some chemistry related items.  A study comparing the performance of boys and girls in the Australian National Chemistry Quiz found no differences on some of the questions, but on some of the questions boys performed better than girls (Walding, Fogliani, Over, & Bain, 1994). There were other chemistry items on the scientific literacy assessment, but there were no gender differences for chemistry items other than question 9.   Men scored better on an item (question no.9), involving quantitative reasoning,  from the scientific cognition assessment.  Question no. 9 was "[i]n the universal lottery, the chances of winning a prize are 1%. How many people do you think would win a prize if 1000 people buy a single ticket?" The correct is answer is 10. A gender difference on a task involving quantitative reasoning is in agreement with the scientific literature that demonstrates better performance of males regarding quantitative reasoning (Friedman, 1989; Leahey, & Guo, 2001). The only gender difference on quantitative reasoning occurred for question no. 9; there were no differences for other items involving quantitative reasoning. Women scored better on an item (question no.3) involving the philosophy of science. Question no. 3 was "[t]he falsification criteria in the context of science suggests a) If a scientific claim is proven then it is not false b) False claims are not accepted c) In order for a claim to be scientific it must be testable." The correct answer is c. The concept of falsification is one of the most discussed concepts in the philosophy of science. The concept is taught in low level research methods courses and philosophy of science courses. We weren't able to locate studies that investigated differences between genders regarding philosophy of science. It is unclear why the gender difference occurred on this task. There were other philosophy of science questions on the scale, but there were no gender differences on those tasks.  Gender differences are often found when comparing scoring for individual items. A study investigating gender differences, for Hong Kong students, didn't find significant differences for total score in scientific literacy, but differences were found for components of scientific literacy (Yan Yip, D., Ming Chiu, M., & Chu Ho, E., 2004). Scientific literacy as conceptualized in that study was different than the conceptualization used in the current study. Scientific literacy ,in the study of Hong Kong students, consisted of five components: "understanding concepts, recognizing questions, identifying evidence, drawing conclusions, communicating conclusions." Females scored significantly higher in "recognizing questions" and "identifying evidence" while boys scored higher in "understanding concepts." These components demonstrate various elements involved with scientific thinking.  To reiterate, our conceptualization of scientific literacy, is that scientific literacy demonstrates general scientific knowledge. Scientific literacy has a much broader definition in the Hong-Kong study than the definition we used.
            Another important finding in the current study was that students confused science with pseudo- science. The overwhelming majority of students (79%) in the current study report that astrology is scientific, or is at least partly scientific. Only twenty one percent of participants in the study answered the following question correctly: "Which of the following statements are true? A) Astrology is not at all scientific B) Astrology is partly scientific C) Astrology is a legitimate field of scientific study."  The correct answer is A. The astrology question is an item from the scientific literacy assessment. The results from a study conducted by Sugarman and colleagues (2011) found that majority of students (78%) considered astrology at least sort or scientific. Only 52% of science majors indicated that astrology was “not at all” scientific. Those finding are similar to what we found. Astrology has no scientific validity, although at one time it was considered a science by some. Newspapers and magazines dedicate sections to horoscopes, and belief in astrology is prevalent in western society. This exposure to astrology as a legit domain probably has a strong influence regarding belief in the scientific validity of astrology. Cognitive priming is often powerful, and may modulate beliefs, even when  priming is used to promote pseudo-science.   Some people may confuse astrology with astronomy; astrology has origins associated with positional astronomy. This confusion may lead to an incorrect response regarding the scientific validity of astrology. A high level of scientific literacy and scientific cognition may serve as safeguards against these sort of pseudo-scientific beliefs.
             The question most often answered incorrectly, from the scientific cognition assessment,   was a question involving a covariation task. The question was presented as "A new medical treatment was designed to treat a serious health problem. Using the information provided below decide whether the treatment was effective: 200 people were given the treatment and improved 75 people were given the treatment and did not improve 50 people were not given the treatment and improved 15 people were not given the treatment and did not improve A) Treatment was effective B) Treatment was not effective." The probability that the treatment is effective is (200/275) .727. The probability that the treatment is not effective is (50/65) .769.   The answer is B.  Approximately 53% of the students answered the question incorrectly. The incorrect response given to this question stems from at least two key cognitive errors: too much focus on the large number of people for which improvement occurred following treatment and a focus on the fact that more people who received treatment showed improvement than showed no improvement (Stanovich, 2009).     
            The survey used in this study consisted of items derived from other assessment tools, as well as questions designed by the researchers, similar to those from past studies. Some of the questions on the survey were  designed by researchers involved with this study. Thus, it follows that the constructs of scientific literacy and scientific cognition were validly measured. A more comprehensive measure may require assessments consisting of more items. It is also possible that measures of these concepts may yield different results inside and outside the laboratory.  Different conceptualizations of scientific literacy and scientific cognition  require different measuring devices. 
            The study involved non-probability sampling.  The participants in the study were college students, who have a great variability in scientific knowledge and variability in the number of science courses completed. Some of the students had taken higher level courses in research methods and stats, and it is reasonable to suggest that some of the students had probably taken philosophy of science courses and other courses that may have had an impact on performance.  The external validity of this study is limited.  Non-student samples and samples of other students may provide different results.           
            Future research could include using the scientific literacy assessment and scientific cognition assessment in a variety of contexts. The assessments could be revised and expanded in an effort to increase sensitivity and make them more comprehensive. Further investigation of gender differences as related to specific items from the assessment may be beneficial. A key focus for extensive investigation is the development of a model that allows, at least a basic framework, that can be used in teaching students and the general public. This type of investigation requires a multidisciplinary approach and a line of studies involving different science related areas.  The cognitive processes underpinning scientific cognition are important and can be extended to various situations. To reiterate, scientific cognition is about much more that remembering scientific theories, laws and principles.  Scientific cognition is essentially analytical thinking that can be used, and should be used in a wide range of conditions. At the very least in an effort to develop better scientific cognition students should be educated in the areas of the philosophy of science, research methodology, quantitative reasoning (probabilistic reasoning) and logic. These components are involved with scientific thinking. Science educators and the media do a disservice when they promote science and its wide range of relevant concepts as "just" being able to remember scientifically derived information, or promoting science as if it is all about a just having a sense of "wonder."  Being able to recollect scientific facts and having a sense of wonder is important regarding science, but those qualities alone do not ensure high levels of scientific thinking. Assessment tools may help predict scientific eminence and be used as screening tools when hiring or considering admissions to college programs.  More research needs to be done regarding scientific literacy and scientific cognition. Both of these concepts involve related cognitive mechanisms, and being knowledgeable in these areas will have positive consequences. Society is heavily dependent on science and technology, and these complex endeavors require complex thinking. We would like to see future research indicating a high positive association between scientific cognition and scientific literacy. A moderate association is not satisfactory.