The Quantum Moves Case

On this page you will find a short overview of the Quantum Moves Case, as well as links in the boxes below to FAQs about the Quantum Moves case (they will be regularly updated), a blog about the Nature paper retraction, and a timeline.

Short summary

Both scientific and more personal critics, the press and the AU Practice Committe have systematically oversimplified and misrepresented the case by combining a half sentence from the Nature-abstract with the public presentation of the project instead of reading and understanding the Nature-article in its entirety. My scientific arguments have been confirmed by a wide range of independent scientific quantum experts whose statements have, however, consistently been ignored. 

• The main points of the Nature article are not wrong even after the correction of the error.
• I have not refused to hand over codes and ignored valid scientific criticism since 2017.
• The right to scientific criticism has in no way been under attack in this case. My three ardent personal critics at the AU-physics department have never engaged in a scientific discourse but consistently put forth wild personal allegations in contradiction with all independent expert judgements.

Timeline of the Quantum Moves case

• April ‘16: Nature paper is published under global media attention drawing headlines such as “take that AI” by LA Times and in general “players beat algorithms”.
• ‘16 and onwards: group of Jacob Sherson develop improved algorithms, which does not invalidate the Nature paper but clarifies the scope of generality of the results
• Summer ‘17: First and only scientific correspondences with Dries Sels
• Based on a part of a sentence in our abstract “players succeed where purely numerical optimization fails” and combined with the popular press presentation of our results, Sels presents a purely numerical algorithm that “beats the players” and thereby calls our conclusions “untenable”. In a civil correspondence of the summer months I point out that he has removed the purely algorithmic result from our Nature paper that also outperforms all players and that he is therefore misrepresenting our results.
• In this correspondence Sels requested once an antiquated game code (unrelated to the algorithmic codes in question) to help his investigations. I promptly supplied him with more useful information for his particular unclarity and never received any further code requests from Sels. (Q&A for details)
• Sels submits his results to Nature but is unanimously rejected with comments like “not that surprising” and that it contained “several strong claims, regarding comparisons to prior work, that appear unfair”
• Sels acknowledges having a “confrontational tone” and promises to correct misrepresentations in future submissions. 
• Spring ‘18: Sels publishes paper in PRA without correcting the misrepresentations. In this paper, Sels clearly states that the poor performance of our algorithm compared to players is no surprise and can be understood by its lack of numerical tricks compared to his algorithm (i.e. Sels has no suspicion that our algorithm contains an error). We completely agreed with this assessment at the time.
• Fall ‘18: anonymous dissatisfaction with our research gives rise to an investigation of Sels’ criticism by faculty representative for responsible conduct, who finds no wrongdoing on our side but heavily criticizes the scientific communication of Sels.
• Dec. ‘18: the main author of the Nature defends his PhD. The two world leading quantum control experts noted a strong criticism of Sels’ work: “it contains a very mature discussion of criticism brought forward by someone else [Sels], whose communication standards seem questionable at best”.
• April ‘19: Allan Grønlund publishes a public “verification” of Sels work but really continues misrepresenting our original work, so whereas we do not disagree with his calculations we disagree strongly with his conclusions.
• May ‘19: The first and only meeting with our three ardent critics at the AU Physics Department. They all three expressed bewilderment at why precisely they had been invited to the meeting and they made it clear that they “do not want to engage in the discussion” but insist that we should issue a public peer reviewed response to the criticism.
• July ‘19: Allan Grønlund had performed more thorough verification calculations and asked for the original Nature codes.
• Sep-Dec ‘19: the codes were located, verified and documented and handed over to Grønlund. The error is discovered independently by Allan Grønlund and our post doc handling the detailed code documentation. 
• Dec ‘19- July ‘20: continuous communication with Nature to clarify the extent of consequences of the error ending in the retraction of the paper.
• Jan.-March ‘20: dramatic criticism and accusations of scientific misconduct sent by the three Physics colleagues to the Head of Department.
• March ‘20: Physics Head of Department decides to send matter to AU praxis committee rather than arrange a process of mutual communication between the parties. His accompanying letter gives us the first knowledge that our three critics claim that Sels had requested the erroneous algorithmic codes. HoD presents this untrue claim as an established fact. See (Q&A for a description why this is wrong).
• Apr ‘20: we publish a thorough scientific response to the criticism with several algorithmic comparisons over multiple challenges confirming the main result of the original Nature paper that gamification is indeed a viable and interesting path for “Crowdsourcing human common sense for quantum control”. In Jan ‘21 this was published after peer-review and none of our critics have since published any new results in the scientific debate.
• May ‘20: Praxis committee finds no indication of scientific misconduct (intentional wrongdoing) but rule that the tardy delivery of the code was questionable research practice. We disagree strongly on the part of the ruling in relation to Sels but accept it because the Dean solemnly promises that the matter is hereby closed and harassments should stop. (Q&A for details)
• Summer ‘20: The three critics keep digging and believe to have found “new” incriminating evidence in the form of old Master’s theses with algorithm with no error. I explain in a written report to HoD that these are in fact identical to the algorithms developed since ‘16 and openly communicated to Sels in ‘17 and the Praxis Committee in ‘20. The HoD approves my documentation and the Three receive the report but choose never to respond. 
• Both Flemming Besenbacher from the Carlsberg Foundation and the Dean are so frustrated with the continued bullying of the Three that they write things to each other that are later made public and by the press misrepresented as attempts to quell the right to valid scientific criticism, although the Three have never engaged in any scientific discussions on the matter.
• Dec. ‘20: anonymous sources leak to the national newspaper the list of 31 strong accusations composed by the Three including now explicit allegations that we should have known about the error for years but covered it up and lied to colleagues, funders and the public in order to harvest as much fame as possible on the results. The one-sided coverage presents a picture of us for several years ignoring justified scientific criticism, and predictably creates a national stir in the scientific community. We try to explain that the presentation of the matter is counterfactual (twisted by the later knowledge of the error that no one had at the time) and that we have in the entire matter simply followed the advice of more than a handful field experts but in the media shit storm these subtleties do not get through.
• Feb ‘21: Three distinguished physics professors at the Niels Bohr Institute in Copenhagen publish a public appeal that the case is blown way out of proportion and that it is human and natural to make mistakes in research. 
• March ‘21: the Three present their “new” evidence to the Dean as documentation for the need for another formal investigation. We strongly plead with the Dean to not initiate another case on such a loose background because it is extremely personally damaging and the evidence is non-existent or at least to let the Three send in their own accusations. To our dismay instead the Dean says he will “put a letter stamp (frimærke) on their accusations” and deliver it to the Praxis Committee.
• May ‘21: AU praxis committee sends the case to the national committee for scientific misconduct (NVU) for evaluation.
• Nov ‘21: NVU confirms the ‘20 Praxis Committee ruling that there are no indications of scientific misconduct in the matter.
• Jan ‘22: the Praxis Committee decides to take up the case again.
• Since the part of the May ‘20 ruling dealing with the supposed request for codes in 2017 has turned out to be the main source of credibility for the newspaper attacks and is obviously wrong to any quantum control expert, I urged the Committee to reevaluate that part of their ruling. As documentation, I collected written statements from the censor of the relevant master’s theses and the mentioned PhD opponents in which they confirmed that there was no indication in the written record of Dries Sels (public or private) to indicate that he even had any suspicions of errors in the codes. 
• 25/1-22: The Committee acquitted me of the accusations of intentional  coverup. Frustratingly the Committee admits that the codes were never requested by Sels and they also acknowledge that no one at the time may have suspected the codes to be erroneous. Still, they maintain that we should have handed over the codes that no one requested or suspected in 2017!

• Fall ‘21 and ongoing: Robin Engelhardt for some reason starts using the case as a public shaming in his scientific responsibility class for junior researchers at Copenhagen University. His presentation of the case is as one-sided as the Berlingske description and when confronted with the numerous factual errors in his presentation he makes cosmetic changes and otherwise refers to the Practice Committee ruling. When pressed with our scientific counter arguments he, like Sels, refers to the original public press one-liners from ‘16 and states “We are led to believe in magic – in the metaphysical powers of “human intuition” and all the rest of the self-aggrandizing bullshit that has darkened human history for ages”. This again confirms that the core drive of the criticism in an outrage of the popular presentation of the project and the Committee ruling continues to legitimize these ideologically rather than scientifically driven attacks. 

Short personal summary of events: This episode occurred on my personal journey from traditional quantum physics research to broad interdisciplinary investigations into the difference between and possible synergies of human and artificial intelligence. On the way, I have ventured into cross-disciplinary work, being driven by a fascination for the (to me) new fields and the shared excitement about the potential of combining several of them. I am personally driven by understanding and attacking big societal problems and both communicating to and involving the general public to the largest extent possible. In my current capacity as Director of the Center for Hybrid Intelligence, I often speak of the concrete research and the dreams of myself and my dozens of collaborators in different disciplines. I am starting to learn that my excitement for the interdisciplinary endeavors we are pursuing together has been interpreted by some as personal arrogance and lack of respect for the research traditions of the various scientific disciplines represented in our Center. This perception of my work is only coming to my attention now, years into the harrowing experience that I will describe below. To all of those, who have been scientifically insulted by my public appearances over the past years, I offer my deepest apologies. I had no idea that this was the perception and if someone had reached out to me with their honest perception earlier perhaps much of the personal animosity that I will detail below could have been avoided. I am a wiser person today, but the toll in getting here has been and continues to be quite high both professionally and personally for me as well as many of my younger colleagues…

The topic of this account is a citizen science game, Quantum Moves, which allowed the general public to contribute to finding solutions to research relevant quantum optimization challenges. The study was published in Nature in 2016 under world wide acclaim but in 2020 retracted due to the discovery of numerical errors in the utilized optimization code. The controversy is a complex saga involving questions of who said what and who knew what and when. At the heart of the debate lies the question, “what was the main scientific result of the Nature paper” - a surprising question as all four pages of the published paper have been publicly accessible since 2016. 

In terms of the scientific importance of the sign error, our critics claim that the error invalidates the foundation of the entire study and subsequent line of research because they claim that the main result of our paper was the simple but extremely general statement “humans beat algorithms”. We do not feel that this claim makes sense, since we have in our own paper presented algorithms that clearly beat all players. Instead, we document in the paper two main results and an observation that players can outperform the particular algorithm. Our newly published Quantum Moves 2 paper shows that while the error did invalidate the observation and influence the magnitude of the effect of the two main findings (it is less pronounced than we originally thought/found), it did not alter the direction. 

In other words, the main finding of the original (retracted) article has been upheld in subsequent studies. Irrespective of our differences of opinion, I welcome any scientific discussion of interpretations with my quantum physics colleagues. Whereas it has been painful to see this debate being championed by critics who have openly proclaimed to not have read or understood our paper, the most painful part of the ordeal has come from an entirely different corner.

In terms of the handling of the error, my ardent critics claim that my team and I have known about the error for years but concealed this knowledge (and perhaps even manufactured the error ourselves) in order to harvest as much acclaim on the erroneous results as possible. In the process we have been absolved of scientific misconduct and any intentional wrongdoing (both by an AU Practice Committee in 2020 and a National Committee in Nov. 2021) but convicted of questionable research practice for not handing over codes supposedly first requested in 2017 until late 2019. This three-year delay has caused outrage in the Danish media, general public and scientific community but the truth is not quite as black and white.

Both the AU Research Practice Committee and the AU leadership acknowledge that the “source code for the game” mentioned in 2017 was in fact not the erroneous scientific, analysis code but rather an outdated version of the game code used for gathering the data. His request came in conjunction with wanting to understand the x and y limits to the player movements. Since the requested Matlab code was outdated and did not even contain the relevant game challenge and the actual game code would require Unity game development experience to understand, I instead promptly and politely provided him with the necessary information and made an open invitation to further discussions. Since then, he never repeated any requests for code and I therefore firmly believed to have answered his request to his satisfaction. Still, the research practice committee reasoned in their verdict that we should have understood that the critical researcher in fact wanted the erroneous code since he “continued to criticize the numeric optimization method”. This is documentably wrong, since that researcher in his 2018 paper clearly writes that our algorithm is bound to fail by its construction (and not its error), because it does not incorporate a fancy trick that he had devised for his algorithm. In addition, in the period 2017-2018 the criticism of Sels was investigated by three sets of independent physics experts (a Nature peer review committee, a phd committee and the Faculty Research Practice advisor) and all three expressed serious criticism of the communication methods of Sels. So, we have been convicted of questionable research practice first by the committee and then in the court of the public opinion for not handing over a code that was never requested and for not being able to foresee that years later an error would be found that neither we, the critical researcher in question or a large set of independent physics experts suspected at the time. This is extremely unfair. 

In july 2019 another researcher did in fact investigate the problem more thoroughly, requesting the erroneous analysis codes in the process. In this case, I accept full responsibility for the four month delay in getting the codes ready to be handed over. However, I stress that there is a world of a difference between the claimed three years of “arrogantly ignoring rightful criticism” as Berlingske Newspaper writes (backed by the erroneous Committee ruling) and a four month response time in an environment that the committee later stated was at this point “characterized by a scientific communication gone bad and that has created personal mistrust among the involved parties”.

In the period of controversy the size of my research center has diminished from roughly 20 full time employees to less than a handful. As a Director, it has been extremely painful to observe the shattering effect on so many young careers and personally it has led to stress-related health problems for me as well as large costs to my personal life. As the Research Practice ruling came in May ‘20, I found it extremely unfair but accepted it due to the promise of the AU leadership that the case was hereby closed. This did not happen. Instead the attacks were intensified resulting in the harsh accusations of conscious scientific misconduct put forth by my Three Colleagues at Physics. In Nov. ‘21 I was finally relieved to receive the full acquittal from the National Committee. There was indeed no indication of scientific misconduct! I drew a huge sigh of relief and happily informed my employees and close colleagues that the harrowing ordeal now finally seemed to have come to a conclusion and we could start to move on. Alas, again the relief was only short lived because short before christmas I was informed that my university would now try to see if the latest round of dirt-digging could potentially give rise to new accounts of questionable research practice on top of the old convictions. The latest investigations are supposedly set to end in January ‘22. Since there is not a shred of new evidence, I feel hopeful that the Committee will finally put an end to the case. For my latest report to the Committee Jan 10th, I have gathered statements from four independent quantum physics experts, who unanimously confirm that there is no indication in the written material that Dries Sels ever expressed concern about the validity of our numerical codes, and that Committee May ‘20 ruling therefore is obviously erroneous. In light of this new documentation, I hope that the Committee will agree to modify that part of their old verdict, so Berlingske and others no longer can use this ruling to back up their sharply pointed narrative. 

Update Feb. '22: the committee acquitted me of further accounts of  questionable research practice but frustratingly refused to heed the expert advice to ammend their old ruling. The case is thereby closed to everyone but me, because my personal nightmare just continues...