I have published in peer reviewed journals and done a few reviews myself. It is not a perfect system. There are generally only a few reviewers. Typically others that have published in the journal in that area. The goal is not to check the work deeply and the tilt is to allowing and also trusting the authors. The other thing that shocked me, the authors generally pay quite a lot of money by the page to publish articles. Also not every journal is the same. Some hard to publish in and others easy. Some nonprofit and others profit making entities. The rush to publication and the publish or perish situation in science creates its own issues too.
This is not that much a failure of science in that it was discovered pretty quickly and presumably a retraction has been made. It is comical.
I would add that the patent system has similar issues. It is far from perfect too. Invalidating a patent is a lot more time consuming and costly and a lot less funny.
Jounal articles are the start of discussion not the end. They get an idea out there for others to consider test and extend or dispute.
Keep in mind reviewers are generally unpaid as well.
Code reviews do not test correctness of code either of if the code is bug free. They also tend to assume good will of the participants. They have similar issues.
Further, most of the times, it’s simply infeasible to test the data in-depth. We’re all humans with busy schedules and it is, unfortunately, not trivial to replicate experiments. If a reviewer feels more data is needed to support a claim, they can ask for a follow-up test or experiment, but it has to be within reason
I have published in peer reviewed journals and done a few reviews myself. It is not a perfect system. There are generally only a few reviewers. Typically others that have published in the journal in that area. The goal is not to check the work deeply and the tilt is to allowing and also trusting the authors. The other thing that shocked me, the authors generally pay quite a lot of money by the page to publish articles. Also not every journal is the same. Some hard to publish in and others easy. Some nonprofit and others profit making entities. The rush to publication and the publish or perish situation in science creates its own issues too.
This is not that much a failure of science in that it was discovered pretty quickly and presumably a retraction has been made. It is comical.
I would add that the patent system has similar issues. It is far from perfect too. Invalidating a patent is a lot more time consuming and costly and a lot less funny.
That sounds like the wrong fucking goals then. I push for detailed code reviews all the time, and encourage my peers to ask questions.
Jounal articles are the start of discussion not the end. They get an idea out there for others to consider test and extend or dispute.
Keep in mind reviewers are generally unpaid as well.
Code reviews do not test correctness of code either of if the code is bug free. They also tend to assume good will of the participants. They have similar issues.
Further, most of the times, it’s simply infeasible to test the data in-depth. We’re all humans with busy schedules and it is, unfortunately, not trivial to replicate experiments. If a reviewer feels more data is needed to support a claim, they can ask for a follow-up test or experiment, but it has to be within reason