Risks from AI in medicine: present

Data

AI systems use specific algorithms that need large datasets to improve their accuracy (specificity/sensitivity). This process is at great risk, as far as the security, privacy, and confidentiality of the sensitive individual patient’s data, is concerned¹¹. Today, the danger of hacking of such datasets has increased tremendously due to the interest of the pharmaceutical or insurance companies. In addition, the hacking of medical files could be a part of a cyber-attack against a government^12,13.

Another issue is data bias. During the collection of the data, intentionally or unintentionally, certain minorities, races, ethnicities, or genders may be significantly misrepresented. Therefore, these algorithms are biased and inadequately represent the general population^14,15. This bias effect could be magnified by the reluctance of medical practitioners, hospitals, or other health organizations, to provide the medical files of their patients due to fears of security leaks. Another significant danger of medical data misuse is the data poisoning effect, which refers to the deliberate manipulation of medical data to introduce errors or biases in healthcare. This has serious consequences on the accuracy and reliability of medical recommendations. This could also affect the outcomes of clinical trials or insurance claims¹¹. Finally, when AI uses different epidemiological data models, as was seen during the COVID-19 epidemic, this could lead to different conclusions.

Development of AI algorithms in medicine

An inaccurate medical algorithm could affect a large number of patients. This may be the result of inadequate testing of such an algorithm since there are currently no solid rules to test its validity. A potential example is the case of double-blind trials, which are generally considered the most powerful tool to establish the effect of a particular treatment¹⁶. Nevertheless, it is unclear who would be responsible in case of such a mistake: the personal doctor, the hospital, the provider of the machine, or the company developing the algorithm. Therefore, there are significant legal issues of accountability for medical errors due to malfunction of the machines¹⁵.

Implementation

Health providers are not adequately trained to use AI technologies, thus, the implementation of these modes of treatment in practice, raises additional security and error issues, until a widely accepted training system for healthcare providers can be established¹⁷. Furthermore, AI may harm the doctor–patient relationship¹⁸. In-depth knowledge of AI evaluation and performance is required by doctors, to communicate the role of AI meaningfully to patients, and avoid patients’ fear and confusion¹⁹.

Fake news-medical information

Algorithms of fake information have been used to affect human behavior altering the socio-economic patterns of individuals or societies. In that case, AI generates articles and social media posts that are designed to deceive the public since they appear legitimate but are completely fabricated. It was reported that fake news could affect even the results of elections^20,21. Similarly, AI has been used to generate, amplify, and manipulate the perception of medical issues spreading confusion. An example is the use of fake news in the global movement against vaccinations. People without the appropriate background of knowledge and training can easily misunderstand and misinterpret information about medicine and science. Unfortunately, fake information may affect even health practitioners²².

Risks in medical education

Although AI made it extremely easy to access medical information, the crucial concern is that this information is not always properly validated. The recently developed Chatboxes, which can provide written articles, answers to university exams, or the contents of a requested lecture, should be also considered as potential threats of AI¹⁹. As their accuracy and validity in various medical aspects have not yet been vigilantly tested, they may represent significant threats.

Moreover, there is the so-called ‘lazy doctor’ effect^18,23. If the doctor exclusively uses AI algorithms, for the diagnosis as well as the treatment options, this may lead progressively to catastrophic forgetting of practical skills and loss of intellectual creativity and capacity to solve medical problems, taking into consideration all the medical and ethics parameters of the patients.

Risks of AI in medicine: near future

During the development phase (approximately up to 2050), in terms of social gaps, it is expected that AI in medicine, instead of promoting equality in health services around the globe, may increase the gap between technologically advanced countries and those lacking digital facilities.

Furthermore, the race of the major AI companies to develop the best, fastest, cheapest, and most profitable medical algorithms, raises extreme pressure on the developing scientists and that may cause fatal errors and a lack of transparency^14,24. Therefore, even AI is still weak during this period, all the previously discussed risks are expected to be intensified. Deeper applications of AI in biology and genetics may lead to dangerous pathways for humanity since AI is going to manipulate human biological functions and genetic codes^25-27. Currently, scientists are trying to connect machines with human brains or implant digital chips into humans. This field is known as brain-computer interface (BCI) technology, which aims to create a direct communication pathway between the brain and external devices, such as computers or prosthetic body parts, primarily used for medical purposes^28,29. Although this technology seems fascinating in assisting individuals with disabilities, it is important to note the ethical considerations that are raised^30,31.

In addition, as it is predicted that AI will cause huge global unemployment, this would affect also the healthcare professions²⁴. The warning about the replacement of tests and examinations, as well as particular medical specialties by AI, is evident. It has been claimed that specialties based on pattern recognition such as Radiology, Pathology, Cytology, Microbiology, and Dermatology could be the first to be substituted by AI^25,32. AI algorithms have already been studied in several medical fields to provide an easier way for diagnosis and evaluation of treatment response³².

Risks of AI in medicine: distant future

Currently, AI is weaker than human intelligence, but as AI is very rapidly advancing, it is foreseen that it will reach the general level of human intelligence by the year 2050 or even earlier^33,34. Thereafter, AI easily would become Super AI, an AI stronger than the human brain. Super AI would have general intelligence and might possess the capability to improve its algorithms, resulting in rapid advances. When (and if) this happens, even if this could be the most significant scientific event, it raises unpredictable issues of how the machines will use medical science³⁴. If, finally, AI algorithms replace doctors’ evaluations, there is a rationale to wonder whether all medical doctors are going to be replaced by robots using AI algorithms. Although this is an extremely difficult question to be answered at present, some experts argue that this may happen in the distant future.

It is even more difficult to predict accurately how Super AI will help scientists interfere with the human genome or use new drugs on humans. Genetic editing may be done using biotechnological methods, while manipulation of the human brain may be achieved via various means, including psychological manipulation, sophisticated brain-computer interfaces, or even direct neural interference^29,35,36. Even if it sounds like scientific fiction, AI may produce clones of ‘human laborers’ by altering human genetics. There are scenarios describing how machines either intentionally or unintentionally may use medicine to harm humans. In the future, Super AI may try to explore the human brain to advance its scientific knowledge. Super AI may behave towards humans the way we have behaved to animals.

Finally, there is a large number of unknown risks and dangers or never thought of scenarios of how Super AI will use medicine in the distant future^1,25,27. To avoid the possible harmful effects of AI that may lead to human extinction, it is of paramount importance to develop a robust control system for safe and human-friendly Super AI^10,37.

Ethical pathway for safe AI

There are several proposals to maintain AI-friendly and beneficial for the human race, in the future. Our proposal is based on the idea that medical ethics could be the prototype for digital ethics. Using deep machine learning techniques, we can embed the machines with ethical and moral principles¹⁰.

Medical Ethics was the first attempt at human morals started 2500 years ago with the Hippocratic Oath. In modern times the Oath is complimented by internationally accepted ethics rules and the various ethics committees. Analogously, AI ethics could have similar components: A Hippocratic oath for computer scientists³⁸, globally accepted ethics rules, and ethics committees for digital science^10,37.

The Oath is expected to improve the ethics knowledge in the computer society and particularly of the AI scientists, who will be responsible for enriching the machines with ethics algorithms. In addition, the Oath may emphasize the personal responsibilities of each scientist, acquainting them with the consequences of wrongdoing³⁸. Details of this proposal have been previously presented in detail^10,38. Internationally accepted ethics and moral rules are urgently needed and if under the auspices, for example, of the United Nations it may be easier to implement globally. Complimentary Ethics Committees for AI at Universities, Research Institutes, and AI companies may add moral filters towards safe AI, enhancing the responsibility and accountability of scientists.

The history of science shows that major discoveries are followed by greater risks. Nuclear power, climatic changes, and AI are considered human inventions that may lead to the extinction of humanity^1,3,4. In this article, the currently existing risks and dangers of AI in medicine, as well as the potential ones in the near and distant future, were discussed. Currently, the major risk concerns the safety and confidentiality of the sensitive medical data of each patient. Even though computer scientists are making tremendous efforts to develop secure methods, medical organizations are reluctant to provide such data, while medical doctors are also not fully convinced. This looks like the fight of policing against crime that is not always successful, although computer scientists are trying vigorously to develop strategies to detect and prevent data manipulations and advance robust cyber security and validation techniques.

Data bias, in other words, incomplete data, or intentional poisoning of the data, are other known dangers of the use of AI in medicine. Incomplete data sets may become harmful when applied to various minorities, such as ethnic, gender, or race that have been excluded from the data. Poisoning of medical data could be made for obvious insurance-claiming reasons, or for affecting the results of clinical trials³¹. Moreover, possible errors of AI in medicine have extremely complicated legal consequences concerning responsibility. Due to the lack of transparency in the development of medical algorithms and the difficulties in the implementation by healthcare professionals, it would be very difficult to verify accountability and apply for claims or penalties^31,37,39. International organizations and governments are now trying to fill these gaps in legislation, with the European Union presenting recently the most comprehensive document^40,41. The inform consent forms, when needed, pose another significant threat because it is difficult to explain complicated AI procedures and test understandability.

Additionally, it is predicted that AI is going to affect drastically medical education and medical practice. Therefore, Medical Schools and Medical Societies should closely monitor these changes and modify their curriculum accordingly. A possible replacement of various medical specialties by AI may lead to unemployment among health practitioners. Another alarming danger is that in the future Medical Doctors may become over-reliant on AI. This could lead to a decrease in critical thinking, clinical judgment, and their ability to solve problems. Thus, the syndrome of ‘lazy student’ and ‘lazy doctor’ may increase^18,23.

The future direction is Super AI that will be able to perform any intellectual task. This remains a theoretical concept for now, but the possibility of Super AI development causes debates and concerns among scientists and experts in AI and ethics, given the idea that in such a case, Super AI may manipulate the human brain. Importantly, editing of the human genome by AI could have catastrophic consequences of unpredicted dimensions for humanity. When (and if) machines reach the level of Super AI, it will be extremely difficult to predict how they will use their medical and genetic scientific knowledge on humans^1,4,25.

While AI is at its developmental stage, it is of utmost importance to act in a preventive way and develop solid and safe mechanisms to ensure that AI remains human-friendly in the future^1,37. Public awareness is crucial, but apart from that, robustly safe mechanisms should be developed by scientists for AI to remain beneficial to humanity, well before it becomes stronger than humans. One way to achieve a friendly AI is to embed the machines with ethics algorithms. We argue that if Medical Ethics become the prototype for Digital Morals, then the dangers of AI on general human life, and on medicine in particular, could be minimized. It was proposed that an oath similar to the Hippocratic one could be the foundation for Computer Ethics³⁸. Such an oath will enhance the ethical and moral responsibilities of AI scientists and their knowledge about the consequences of wrong practicing. Finally, this oath for Computer scientists will increase global awareness of the potential misuse of AI in medicine and will enhance international collaboration for a safe AI^10,38. As the advance of science leads to the ongoing enormous development of AI, the key for humanity is to accomplish symbiosis with AI.

Conclusion

Although AI has a huge beneficial impact on medical science, it is followed by several significant risks and dangers. It is strongly suggested for medical organizations to monitor the changes which are associated with the giant steps of AI development, and modify accordingly medical education and practice. The major risks might emerge when AI becomes more powerful than the human brain, thus it is of paramount importance to develop solid and safe mechanisms to keep AI under control. The establishment of an ethical pathway could be one of the safe ways for AI to remain human-friendly in the future.