| 1 | Credibility, Relevance;
the need to generalize;
many articles do not generalize properly |
| 2 | Finer-grained notions of quality;
Quantitative studies: validity;
Excursion: Scientific method (theories, scientific theories,
falsificationism, alternative: qualitative exploration,
hard science, soft science, physics envy) |
| 3 | Quantitative studies validity (internal, external, construct, conclusion);
construct validity (bad operationalization);
conclusion validity (correlation vs. causation, spurious correlation) |
| 4 | Qualitative studies validity: worthy topic, rich rigor, sincerity, credibility,
resonance, significant contribution, ethical, meaningful coherence |
| 5 | Patterns for low quality: violating an ordinal scale; failing to visualize;
broken assumption "developers to the Right Thing" |
| 6 | Patterns for even lower quality:
broken operationalization;
broken assumption "ROI will easily be achieved";
overgeneralizing;
not explaining usefulness |