Scientists properly referencing things is great^{[citation needed]}, but sometimes proper referencing leads to improper science.

With the apparent need for scientists to produce more papers, it is increasingly common to see three or four separate short papers on the same subject – the same experiment, even – rather than one big paper which rounds them all up. This isn’t necessarily a bad thing. Reading a long paper with several experiments which are variations on a theme can take all morning, and sometimes you’re just looking for one specific bit of information anyway. Papers written like this (or articlettes, as I think of them) generally cite their sister articlettes to avoid repeating things every time. It often looks something like this:

Methods

The methods are the same as in Me et al. (20XX), but this time the stimuli were presented visually instead of auditorily. See Me et al. (20XX) for a detailed description.

This isn’t too much of a problem. It cuts down on the length of the articlette by removing material which has already been written and published. If you want to see the detailed methods, it tells you exactly where to find them (and if you’re not interested in the detailed methods, then maybe you should read papers more thoroughly). The author also benefits by sneakily increasing their citations by citing themself.

The problem is when the articlettes read something like this:

Methods

We did blah blah blah with stimuli that were designed based on Me et al. (submitted).

…or even worse,

Methods

We did blah blah blah with stimuli that were designed based on Me et al. (in prep).

In this case, the author is citing their own work which has been submitted to (but not yet accepted and published by) a journal, or isn’t even ready to be submitted. Either way, it’s impossible for the enthusiastic reader to follow up and have a look at their experimental manipulations more closely, because the sister articlette is unavailable (I think of this sort of thing as ghost literature). This is really frustrating – it’s very difficult to know what to think of an articlette’s conclusion when all kinds of things could depend on the manipulations. It could be just as the author describes; but there could also be various things in the experimental set-up which could easily determine the results, possibly more so than the main manipulation which the author thinks is responsible, and it’s just not possible to have a look. Moreover, the articlette which has been published will always look like that – somebody could be reading it years later, and not know where to find the sister articlette with all the interesting information in it, even if it has since been published.

There are mitigating factors, of course. Given the appeal of articlettes to both reader and author, the author can’t necessarily be blamed for putting something in one articlette and then citing it in another. It’s easily possible that the two articlettes were submitted for review at exactly the same time, and that the second one was reviewed and published more quickly than the first one. This would lead to the second one, which depends on the first, citing a paper which is not yet available. It’s unfortunate, but understandable, considering the casserole of nonsense that is the scientific journal system. A rather more cynical interpretation would be that the author knows that their work wouldn’t pass peer-review if submitted completely, and therefore cites ghost literature to obscure the deficiencies of the articlette in question.

I’m surprised that there aren’t measures or rules against these things. Some, but not that many, journals have an editorial policy which states that papers citing as yet unavailable manuscripts will be rejected (and hey, journals will find all kinds of excuses to reject papers). It shouldn’t be too hard to fix. Either the journals prevent the citation of ghost literature, or the authors go full Open Science and publish their stimuli on open repositories online somewhere.

As it stands, ghost literature is haunting science, making it hard to evaluate a paper for what it is. Thing is, I don’t know who to call.