I discussed with other participants of the WSKS the situation in science regarding scientific publishing. Scientists have to prove high-impact publication (see science index), but publication processes are often very long. Consequently, results are often out-dated at the date of appearance.
A tendency that one may recognize is that more and more submissions focus on convincing that they are capable of doing something (realizing an idea) rather than displaying fully implemented and evaluated research systems. As I was told, this is also referred to a shift from “Publish or Die” towards “Demo or Die“.
If we look at the mass of scientific publications today, it gets harder and harder for scientists (in particular novice) to distinguish papers on fully evaluated algorithms and systems from vision-papers or work-in-progress as well as paper on still ongoing work from out-of-date publications. Would it be possible to classify papers more explicitly, i.e. to help (young) scientists to immediately know what to expect from a paper (ideally before having to purchase it)?
I agree that (to a certain degree) it is our task to learn how to make this distinction ourselves. For example, we can take the impact factors and reputation of the publication sources into account. However, although in these highly-recognized sources, the actual state of the presented work remains unclear (sometimes). This particularly refers to the presented systems, i.e. to decided whether a mock-up, a research prototype, or “almost-product” has been presented.
And overall, when is a good time to publish, i.e. what state should the research be in before it is “ready” for publication?