Science

…and how to know it when you see it
Every once in a while it is good to briefly review the general concept of what science is and how to separate it from other non-science based concepts.

Science involves a set of cognitive and behavioral methods designed to describe and interpret observed or inferred phenomenon- past or present- that are aimed at building a testable body of knowledge open to rejection or confirmation(1), The crucial part of this is that science- the best system of gathering knowledge we have- grows over time as useful features are retained while non-useful items are eventually discarded based on the rejection or confirmation of the gathered testable knowledge.

Put another way, testable knowledge -a theory for example- needs to satisfy two fundamental requirements. First, it needs to explain the observed phenomenon better (more comprehensively) than other competing theories. Second, it has to be able to make testable predictions that are correct.

Non-science/pseudoscience
Michael Schermer put together ten questions(2) worth asking when trying to define the boundaries between science and non/pseudo-science that serve the inquiring mind well in the search for reason among, for example, the plethora of medical claims and therapies out there. They are as follows:

1)How reliable is the source of the claim?

2)Does this source often make similar claims?

3)Have the claims been verified by another source?

4)How does the claim fit with what we know about how the world works?

5)Has anyone gone out of the way to disprove the claim, or has only supportive evidence been sought?

6)Does the preponderance of evidence point to the claimant's conclusion or to a different one?

7)Is the claimant employing the accepted rules of reason and tools of research, or have these been abandoned in favor of others that lead to the desired conclusion?

8)Is the claimant providing an explanation for the observed phenomena or merely denying the existing explanation?

9)If the claimant proffers a new explanation, does it account for as many phenomena as the old explanation did?

10)Do the claimant's personal beliefs and biases drive the conclusions, or vice versa?

Here is another short list of considerations to have handy that help identify non-science based claims that was put together by Langmuir(3,4). It's another set of tools that helps cut through much of the confusion when perusing the literature:

1) Claimed effect being studied is often at the limits of detectability.
Subjective visual observations replace objective instrumental measurements.
The maximum observed effect is produced by an agent of barely detectable intensity.

2) Investigators readily discard prevailing ideas and theories and disregard criticism of their new ideas and theories.
Investigators concoct new ad hoc theories to account for the phenomenon.

3) Investigators do not attempt critical experiments that could refute their new theory by determining whether or not the effect is real.
Experiments done by others that refute the new theory are disregarded.


Ref:
1) Shermer M. The triumph of the scientific method: The most precious thing we have. Skeptic 1(1):34-49. 1992
2) Shermer M , Scientific American, 285:5, November, 285:6, December2001
3) Dr. Irving Langmuir, 1932 Nobel Laureate, as condensed by DL Rousseau in: "Case Studies in Pathological Science: How the Loss of Objectivity Led to False Conclusions in Studies of Polywater, Infinite Dilution and Cold Fusion," American Scientist 80:54-63 (1992))
4) Ramey, DW (1998). Pathological Science. World Eq Vet Rev 3(2):25-27.