Outreach initiative


The concept of ROBOETHICS arises as a consequence of the coexistence between robotics and social and human disciplines. This section of the IRI website aims to provide a general approach to this issue and to compile the initiatives that are being carried out at our institute within this theme.

María de Maeztu Project

This initiative is in the framework of the María de Maeztu (MdM) Strategic Research Programme.
→ More about MdM project


The term Roboethics (aka Robot ethics) was coined by the roboticist Prof. Gianmarco Veruggio in 2002 to refer to Ethics applied to the development of Robotics for human and social progress. It inherits aspects of the broader concept of Technoethics in a natural way, and does also share similarities with the concept of Bioethics in the sense of an accelerated scientific and technological blooming with relevant -if not drastic- potential consequences on human society.

There are three different broad lines of research within the discipline of Roboethics. This is why the term is generally understood as entailing three distinct meanings.

The first research line is known as Roboethics in the strict sense. It focuses on the ethical issues arising from human action regarding the different stages of robotics deployment, namely: design, research, implementation and use. These issues relate to the implications and risks that such deployment may pose at the different levels of human life (individual, interpersonal, professional, sociostructural, economic, etc.), in a physical, psychological and economical sense. The ethical reflection is thus concerned with the grounds of human decision-making in all spheres of human activity coming into play in the deployment of robotics (technoscientific activity, institutional adoption of technology, usage by individuals, etc.).

The second line of research is called Machine ethics, and it is focused on the behaviour of autonomous artificial agents. The ethical reflection concentrates on the question of how to grant that robots' behaviour is aligned with human values, thereby contributing to the task of programming moral codes in these artefacts or even inserting in them the capacity for moral reasoning.

The third research strand can be named Robot's Ethics, and it consists in a more speculative line of ethical reflection, insofar as it focuses on ethical issues related to a technological development stage that is still far from reality. For instance, it tackles issues such as robots' rights or how to grant human interests and values in the face of a potential emergence of general or strong artificial intelligence.

Brief history of Roboethics

Ethic related concerns have been present long ago, but the first event gathering together a collective of roboticists was the Roboethics Congress at San Remo (Italy) in 2004, and can thus be considered as the beginning of the official history of Roboethics. Since then, the history of this young discipline has witnessed specialized workshops in the most relevant robotics congresses, public institutional declarations, the creation of technical committees on the subject, and several legislative initiatives. You can see more info in this section.

Impact on society

Robotics will have a major impact on human society during the next decades. As new technological advances enhance their capabilities, and manufacturers begin to produce them on market scale, robots will pervade more and more aspects of our everyday lives. They will transform our jobs, they will modify our healthcare , they will affect the way we displace ourselves and our goods, they will reshape our households, they will change warfare , and they will even alter our personal relationships. Quite soon legal issues will arise, related to possible malfunction, accidents caused by robots, or criminal use, which will imply the development of new laws, including the clear delimitation of responsibilities of the different involved actors. As robots evolve and their computational capabilities become more and more sophisticated, we may also question ourselves of what it actually means to be human, conscious, or sentient, and if we have to contemplate them as our offspring or as a new species. Science-fiction writers have dealt with such questions already, and their visions may be inspirational for scientists and Technological developers.


The compromise of the researcher

The way and the extent of transformation of human society by robots will not be decided by a single actor. At the end, it will be society as a whole who will set the limits, but the decision power of reduced sectors on the spread of robots is also unquestionable. Influencers and trendsetters can obviously favour one or other robotic product, but they are constrained by what is actually available, and manufacturers will naturally shape the evolution of their products in terms of the expectations generated in the market. However, at the very base of the performances of robots as market products lie the achievements and exploits of scientific and technological research. Thus, the researcher has an evident responsibility on the outcome, and should be aware of the consequences and potential implications of their research. As for their concern about ethical questions, three different attitudes can be observed in the research community:

  • No concern at all, the researcher thinks their work is purely technical, without any ethical or social responsibilities.
  • Concerned about short-term ethical questions.
  • Concerned about ethical questions that may forseeably arise in the long term.

Whereas it is generally accepted that research should proceed in an unconstrained fashion, the only limits posed by scientific rigour itself, it is also true that the researcher forms part of human society and they have the background to evaluate how far their results may reach, and to some extent also what the derived consequences may be. In some sense it is reassuring that the concept of Roboethics owns its birth to a roboticist, and that more and more scientific researchers are concerned by ethical questions. The willingness of the scientific community to participate in the ethical debate is witnessed by a large number of contributions in dedicated projects, workshops, surveys, etc., as well as by the readiness in approaching their research to the public.



2004 marks the spectacular launch, as no less than three important events took place this year:

  • The first Roboethics congress at San Remo (Italy): The program included sessions devoted to the need for a Roboethics, AI and Ethics, Humans and Humanoids, Robots helping Humans, Robots beyond Humans. problems in Human-Robot coexistence, and Robots for the Planet.
  • The creation of the IEEE-RAS Technical Comittee on Roboethics, with the following declaration: "The focus of the TC includes the unintended warfare uses of robotics research results, the preservation of human integrity in the interaction with robotic (even bionic) systems, and the study and development of the robot-ethics concept. The TC pursues its objectives by organizing focused events and publications at RAS-sponsored conferences and elsewhere."
  • Fukuoka World Robot Declaration: "Confident of the future development of robot technology and of the numerous contributions that robots will make to Humankind, this World Robot Declaration is issued on February 25, 2004 from Fukuoka, Japan." It declares the following "Expectations for next-generation robots:
    • Next-generation robots will be partners that coexist with human beings
    • Next-generation robots will assist human beings both physically and psychologically
    • Next-generation robots will contribute to the realisation of a safe and peaceful society"
  • It should be noted that already in 2003 Japan established a series of "Tokku" Special Zone for Robotics Empirical Testing and Development (RT special zone) in the Fukuoka Prefecture, Fukuoka City and Kitakyushu City. Observing the difficulties of regulating a priori the complex interaction between humans and robots, the Japanese Government opted to go the other way by empirically testing what happens in a "deregulated" zone. The results after 10 years of experience can be found here (since then there are other special zones in Osaka, Gifu, Kanagawa and Tsukuba).
  • A page "Laws of Robotics" is created in Wikipedia as alternate capitalization redirect (i.e., without specific content) for "Three Laws of Robotics", which was in turn created in 2001 and referring specifically to Asimov's well known "laws". Content was added later in 2011 (see below).


  • The IEEE International Conference on Robotics and Automation (Barcelona) includes a workshop on Roboethics. Contents and speakers are listed next, and the talks were followed by panel discussions.
    • Part I - Introduction to Roboethics
      • The Birth of Roboethics, G. Veruggio
      • Roboethics and Scientific Method, R.Cordeschi and G. Tamburrini
    • Part II - Case-studies
      • Robotics and warfare, R.C. Arkin
      • Ethical issues involved in hybrid bionic systems research, P. Dario
      • Artificial emotional creatures, T. Shibata
    • Part III - Philosophical and theological issues in Roboethics
      • Cultural Differences in the Perception of Robots in Japan and in Europe, A. Takanishi
      • Techno-ethical and anthropological aspects of the Synergy of Natural and Artificial Hardware, J.M. Galvan
  • Kick-off meeting of the European Union FP6 - Science and Society Project ETHICBOTS (Emerging Technoethics of Human Interaction with Communication, Bionic, and robOTic systems), in Naples, November 26th 2005. A multidisciplinary international consortium analyses the "techno-ethical issues concerning the integration of human beings and artificial (software/hardware) entities", including human-softbot, human-robot non-invasive, and physical invasive integration.


  • First Roboethics roadmap (EURON Roboethics Atelier): The Italian Scuola di Robotica coordinated this workshop that brought together about 50 researchers from scientific and humanistic disciplines. Previous setting up work began already in 2005. The outcome of this preparation work and one week of debates in February 2006 was the EURON Roboethics Roadmap. Questions that should be addressed, following this document, include:
    • How far can we go in embodying ethics in a robot?
    • Which kind of "ethics" is a robotics one?
    • How contradictory is, on one side, the need to implement in robots an ethics, and, on the other, the development of robot's autonomy?
    • Although far-sighting and forewarning, could Asimov's three Laws become really the Ethics of Robots?
    • Is it right that robots can exhibit a "personality"?
    • Is it right that robot can express "emotion"?
  • The roadmap should be viewed as a tool whose aim is "to provide a systematic assessment of the ethical issues involved in the Robotics R&D; to increase the understanding of the problems at stake, and to promote further studying and transdisciplinary research." Being its scope near future urgency, it clearly targets at a human-centered ethics, leaving out of consideration moral topics associated to the emergence of consciousness and free will in robots. Inspiration is sought in Computer and Information Ethics as well as in the precautionary principle of technological development, from which other rules derive like "non-instrumentalisation, non-discrimination, informed consent and equity, sense of reciprocity, data protection." The roadmap also comes up with an application-oriented roboethics taxonomy.The final document was presented the following year at ICRA 2007, see below.
  • BioRob2006 (The first IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics), Pisa, Italy. A minisymposium on Roboethics was held.
  • Supported by the EU project ETHICBOTS (see above), an international workshop was held, entitled "Ethics of Human Interaction with Robotic, Bionic, and AI Systems: Concepts and Policies", Naples, 17–18 October 2006.
  • On December, 14th, the page "Roboethics" was created in the Wikipedia.



  • International Symposium "Robotics: New Science": Accademia dei Lincei, Rome, February 20st. It is conceived "as a seminar whose aim is to see the current developments in robotics in the perspective of the general history of the machines, of Automation and Control, of Wiener's Cybernetics, and of the Artificial Intelligence." Among the speakers, G. Veruggio focuses on "Philosophical, Social and Ethical Implications of Robotics"
  • ETHICBOTS project concluded in April.



  • The UK Engineering and Physical Sciences Research Council (EPSRC) and the Arts and Humanities research Council (AHRC) organized a joint Robotics Retreat of experts "to discuss robotics, its applications in the real world and the huge amount of promise it offers to benefit society." In their page "Principles of robotics: Regulating robots in the real world" they describe the five principles for designers, builders and users of robot they came up with during the workshop, Moreover, the group also devised a set of seven high-level messages, "designed to encourage responsibility within the robotics research and industrial community, and thereby gain trust in the work it does."


  • ICRA 2011 Fourt hWorkshop on Roboethics Shanghai, China, May 13. "Among the objectives of the workshop is the opportunity of developing rules for roboethical quality insurance, aimed at preventing unethical uses of robots. Long term objectives include the increase of robotics researchers' ethical awareness, in the context of the ever growing interdisciplinarity that will characterize the new generation of robotics research."
  • Publication of ISO 10218-1:2011 in July, which "specifies requirements and guidelines for the inherent safe design, protective measures and information for use of industrial robots."
  • Publication of ISO 10218-2:2011 in July, which "specifies safety requirements for the integration of industrial robots and industrial robot systems as defined in ISO 10218-1, and industrial robot cell(s)."
  • IACAP 2011 : International Association for Computing and Philosophy (IACAP)
  • The page "Laws of Robotics" is properly created in the Wikipedia (October 3rd), with original content and incorporating different sets of "laws" (besides Asimov's original set which has its own page).


  • Start of the European Union FP7 - Science in Society Project RoboLaw (Regulating Emerging Robotic Technologies in Europe: Robotics facing Law and Ethics), on March 1st. Its principal aim is the elaboration of a document on "Guidelines on Regulating Robotics", "containing regulatory suggestions for the European Commission".
  • Foundation of ROBOLAW.ASIA - YSAiL Initiative for Robotics, Law & Policy, between the EU Project RoboLaw and the University of Beijing (China) Institute for Internet Law. The Humanoids Robotics Institute of the Waseda University (Japan) also joined this initiative.
  • We Robot. Inaugural conference on legal and policy issues relating to robotics, University of Miami School of Law, Coral Gabels, FL, April 21–22. "This conference will build on existing scholarship exploring the role of robotics to examine how the increasing sophistication of robots and their widespread deployment everywhere from the home, to hospitals, to public spaces, and even to the battlefield disrupts existing legal regimes or requires rethinking of various policy issues."
  • AISB/IACAP World Congress 2012 - Alan Turing 2012 . The symposium "The Machine Question: AI, Ethics, and Moral Responsibility" seeks to bring philosophers, AI scientists, robotics engineers, and other researchers together in order to investigate and discuss the current state of and future possibilities for machine morality."



  • Publication of ISO 13482:2014 in February, which "specifies requirements and guidelines for the inherently safe design, protective measures, and information for use of personal care robots".
  • We Robot 2014. Risks and opportunities., University of Miami School of Law, Coral Gabels, FL, April 4–5.
  • RoboLaw project concluded on May. On September 24th, the project's findings are presented at the European Parliament, at the workshop "Upcoming issues of EU law" (organised by the JURI -legal affairs- Committee) with he paper "REGULATING ROBOTICS: A CHALLENGE FOR EUROPE"
  • Robophilosophy 2014 "Sociable robots and the future of social relations". First Edition of the biennal International congress on "philosophy of, for, and by social robotics", in the words of Johanna Seibt in this article Aarhus University, August 20-23


  • The Headquarters for Japan's Economic Revitalization, from METI (the Japanese Ministry of Economy, Trade and Industry), issues in February the "New RobotStrategy. Japan's Robot Strategy. Vision, Strategy, Action Plan". Ethical concern is posted specifically at nursing robots. This report does also include the establishment of the Robot Revolution Initiative, a private-led open innovation platform.https://www.jmfrri.gr.jp/english/outline/763.html
  • We Robot 2015, University of Washington School of Law, April 10–11.
  • New Friends 2015, First International Conference on social robots in therapy and education, October 22-23, Almere (The Netherlands), adresses ethical issues in one of its tracks. Proceedings available here
  • Inaugural workshop of the research network Transdisciplinary Studies in Social Robotics (TRANSOR) "Methodological problems of social robotics". TRANSOR is "a platform for research exchange and joint Humanities research in social robotics, connecting researchers in philosophy, robotics, cognitive science, psychology, anthropology, educational science, linguistics, art and design studies, and communication and media studies."
  • Formation of the Foundation of Responsible Robotics(FRR), its motto being "Accountable Innovation for the Humans Behind the Robots", and its mission statement "to shape a future of responsible (AI based) robotics design, development, use, regulation, and implementation. We do this by organizing and hosting events, publishing consultation documents, and through creating public-private collaborations."


  • Publication of ISO/TS 15066:2016 in February, which "specifies safety requirements for collaborative industrial robot systems and the work environment, and supplements the requirements and guidance on collaborative industrial robot operation given in ISO 10218-1 and ISO 10218-2."
  • Symposium on "Ethical and Moral Considerations in Non-Human Agents", Stanford University, Stanford, CA, March 21-23. The aim of the event is "to bring together researchers from AI, law practitioners, philosophers of ethics, and neuro-cognitive scientists to shed light on the problems of design and regulation of ethically- and morally-informed autonomous systems as they become part of our everyday life."
  • We Robot 2016., University of Miami School of Law, Coral Gabels, FL, April 1–2.
  • First edition of the International Summer School on "The Regulation of Robotics in Europe: Ethical, Legal and Economic Implications", Pisa (Tuscany, Italy) July 11- 16. Within this summer school, conference on "Regulating Robotics: A Challenge for Europe"
  • The European Parliament's Legal Affairs Committee commissions Nathalie NEVEJANS to elaborate a study on European Civil Law Rules for Robotics, "to evaluate and analyse, from a legal and ethical perspective, a number of future European civil law rules in robotics." October, 12th.
  • The UK House of Commons Select Committee for Science and Technology publishes a report "Robots and Artificial Intelligence", which devotes a chapter to ethical and legal issues of these technologies, and recommends the Government the establishment of a 'Commission on Artificial Intelligence'.
  • The Committee on Technology of the National Science and Technology Council from the Executive Office of the President of USA issues the report "Preparing for the Future of Artificial Intelligence" and the Networking and Information Technology Research and Development Subcommittee of the same Council delivers the corresponding National Strategic Plan on Research and Development of Artificial Intelligence, both in October. Country-wide regulations have only been undergone previously for the specific cases of UAVs and autonomous vehicles, in June and September 2016 respectively.
  • Launch of the research project ANOTHER "Androids as Significant Others" 01/06/2016 → 30/11/2022
  • Launch of the research project INSOR:"Carlsberg Semper Ardens Project: What Social Robots Can and Should Do--Towards Integrative Social Robotics" 01/07/2016 → 30/06/2021
  • Robophilosophy 2016 "What social robots can and should do". Combined with the conference of TRANSOR 2016, Aarhus University, October 17-21
  • Creation and approval of the first of the IEEE P7000™ series of standards projects, addressing "specific issues at the intersection of technological and ethical considerations" of Autonomous and Intelligent Systems (see the current list of standardization projects here.
  • The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems releases on December the first (draft) version of Ethically Aligned Design (see the newest version below -2019) as a Request for Input, receiving over 200 pages of feedback.


  • Mady Delvaux-Stehres, Member of the European Parliament, Vice-Chair of the Committee on Legal Affairs, Chair of the Working Group on robotics, presents on January 27th a report with recommendations to the Commision on Civil Law Rules on Robotics. The debate took place on February 15th, and the final text was adopted by the European Parliament on February 16th.
  • Kick-off of the European H2020 Project REELER, Responsible Ethical Learning with Robotics. The project "aims to align roboticists' visions of a future with robots with empirically-based knowledge of human needs and societal concerns."
  • We Robot 2017. Information society project, Yale Law School, New Haven CT, March 31st-April 1st.
  • Saudi Arabian citizenship was granted on October 25th to a "female" humanoid robot called Sophia, developed by Hanson Robotics, at the Future Investment Summit in Riyadh. It is the first robot to be given citizenship from any country, and also the first non-human to receive any United Nations title (UN Development Programme's first Innovation Champion, in November).


  • Summer School on "The Regulation of Robotics in Europe: Legal, Ethical, and Economic Implications" Pisa (Italy), July, 2-7. Opening ceremony "Regulating Robotics: A Challenge for Europe"
  • We Robot 2018, Stanford Law School, Stanford, CA, April 12–14.
  • Release of version 2 of "Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems", a "document [..] created by committees of The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems", with the following purpose:
    • "Advance a public discussion about how we can establish ethical and social implementations for intelligent and autonomous systems and technologies, aligning them to defined values and ethical principles that prioritize human well-being in a given cultural context.
    • Inspire the creation of Standards (IEEE P7000TM series and beyond) and associated certification programs.
    • Facilitate the emergence of national and global policies that align with these principles."
  • Launch in July of OCEANIS (Open Community for Ethics in Autonomous and Intelligent Systems) which "is a global forum for discussion, debate and collaboration to further the development of autonomous and intelligent systems through standardization."
  • Launch of the Council of Extended Intelligence (CXI) by IEEE Standards Association and The MIT Media Lab. The mission of the council "is to proliferate the ideals of responsible participant design, data agency, and metrics of economic prosperity, prioritizing people and the planet over profit and productivity".
  • Launch of AI Commons , a global collaborative platform around A/IS, aimed at democratizing access to the benefits of autonomous and intelligent systems to everyone. It also acts as a platform to connect problem owners to solvers, to come up with collaborative solutions.
  • Robophilosophy 2018 "Envisioning robots in society. Politics, power and public space" Combined with TRANSOR 2018, University of Vienna, February 14-17


  • The Vatican in Rome displays its concern about roboethics, via the workshop "Humans, Machines and Health" Assembly of the Pontifical Academy for Life, February 25-26
  • Launch of the Ethics Certification Program for Autonomous and Intelligent Systems (ECPAIS), whose goal is "to create specifications for certification and marking processes that advance transparency, accountability, and reduction in algorithmic bias in autonomous and intelligent systems" so that organizations can certify their products and services.
  • We Robot 2019 University of Miami School of Law, Coral Gables(FL), April 11-13
  • RoboCup 2019 Talks: RoboEthics, Sydney, Australia, July 4
  • Version 1.3 of the Android educational app "Robo Ethics", by Softecks
  • Experimental and Integrative Approaches to Robo-ethics (EIAR) 2019 (Workshop within the International Conference on Social Robotics, ICSR 2019), Madrid, Spain, November 29
  • The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems. Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems, First Edition. IEEE, 2019.
  • Creation of the EAD First Edition Glossary on A/IS Ethics


  • We Robot 2020 University of Ottawa (Virtual), September 22-25
  • Robophilosophy 2020 "Culturally Sustainable Social Robotics. Challenges, methods and solutions". Online, August 18-21
  • Windesheim University and the Foundation of Responsible Robotics (FRR) team up to develop the audit framework and procedures to issue the FRR Quality Mark. This label will certify alignment of the product with the vàlues of "security, safety, privacy, fairness, sustainability, accountability and transparency".

Impact on labour market

Studies on jobs robots can take over in the next decades. New jobs created by robotics. Cobots. Social challenges, distribution of work and economical benefits among general public,etc.

Affective relationships and dependence

Presence of robots in domestic environments. Benefits. Privacy and security threats. Eldercare, assitance to disabled, personal assistant, education. Pros and cons of humanlikeness, user-friendly interaction, uncanny valley. Personalisation, development of affective dependence. Intimate partners. Intellectual, emotional, social growth: opportunities and dangers arising from HRI.

Robots and war

Mission taxonomy: humanitarian missions, exploration, defensive and offensive systems. LAWs (Lethal Autonomous Weapons): existing and prospective technologies. Degrees of autonomy: Autonomy in target setting, autonomy in reaching objectives. Regulatory efforts: International Humanitarian Law. Stop Killer Robots campaign.

Surgery, protheses and enhancements

Teleoperated vs autonomous surgery systems. Smart protheses, recovery of lost perception channels and mobility. Enhancements beyond natural capabilities. Superhumans. Transhumanism.

Ethical background of laws. Laws for guaranteeing ethical use of robots. Robolaw project (concluded) EU regulatory efforts: summary of resolution (February 16th, 2017)

Autonomous vehicles

Definition, technologies. Taxonomy on degree of autonomy. Social impact: benefits, challenges, risks. Programmed reactions to limit situations.


Existing and prospective technologies. Possible uses, benefits. Aerial space invasion: annoyance, privacy and security threats.

Conscious/sentient machines

Probably not in a near future, but at some moment robots may be endowed with or may develop computational traits that make them undistinguishable from conscious beings. Access and phenomenological consciousness. Mind-body problem. Dualist and monist views. Plausibility of an artificial consciousness. Why should we implement consciousness in a robot, if ever possible? Aspects of consciousness that should be implemented to have a conscious machine. Awareness, memory, learning, anticipation, subjective experience.

Sci-fi views

Science Fiction (sci-fi for short) often provides valuable insights into the possible development of science and technology and its impact on human society. The point is that creativity is only bounded by credibility and plot consistency and appeal. In sci-fi literature and movies, robots are treated either as just a piece of attrezzo to suggest futuristic ambience, or they are central to the plot development. In the latter case, they can further be just an embodiment of the evil and mighty foe, interchangeable with any type of monster, which is typical of pulp sci-fi, they can be treated as any other human character, with occasional puns to their robotic nature (e.g. Douglas Adams' Marvin) or their features as increasingly smarter alter ego of humankind are exploited in a more intelligent way. We are obviously more interested in the latter.

Robots as mere servants. Fictional examples of societies where most work is done by robots. Personal relationships with robots. Distopies: robotic rebellion. The quest for humanity: the Pinocchio syndrome. Robotic societies.

In this section general works on roboethics are presented and reviewed. Topic-specific papers are listed in the corresponding topic page, this space addresses rather generic publications, compendia, educative material and the like.

Compilations, proceedings

  • Lin, Jenkins, and Abney Eds., "Robot Ethics. The Ethical and Social Implications of Robotics ", The MIT Press, 2014
  • Lin, Jenkins, and Abney Eds., "Robot Ethics 2.0. From autonomous cars to artificial intelligence", Oxford University Press
Compilation Compilation


  • Wendell Wallach, Collin Allen "Moral Machines. Teaching Robots Right from Wrong", Oxford University Press, 2009
    • Synopsis: The monograph begins with providing reasons that justify the need of machine morality, i.e., endowing machines with the capacity of evaluating the moral consequences of their actions, which converts them in Artificial Moral Agents (AMAs). Such reasons are not only inherent to long-term futuristic scenarios, but relate already to present and near future situations. While not pretending full moral agency (requiring conscious understanding), which the authors situate on the top right corner of a two-dimensional graphical depiction of ethical sensivity and autonomy, AMAs can be developed trascending mere operational morality (e.g., of the kind embedded in safety mechanisms) towards functional morality, which implies decision making capabilities.
      The authors further analyse whether Humanity actually needs or wants machines to do ethical decisions and relate this to the increasing dependence on more and more sophisticated technology. When it comes to actually implementing AMAs, the authors examine first the top-down approaches, which try to derive a set of rules from some of the big ethical theories (deontology, utilitarianism) and convert them into a suitable algorithm. These rule-base approaches have to face ambiguity when confronted with the richness and nuances of real worlds situations. Bottom-up and developmental approaches, on the other hand, are flexible anough to adapt to a dynamical world, but learning can be quite slow and the resulting behaviors too loose to become hard constraints, which may be risky in some contexts. The possibility of merging both types of approaches is also considered, like in the aristotelian virtue ethics approach.
      Attempts to engineer software like decision support systems and multiagent software systems are examined as first steps towards reasoning about own decisions. Suprarational capabilities including emotions and empathy and their role on moral judgment are also considered. Franklin et al's LIDA cognitive cycle model is analyzed to some extent at the light of actual implementations of the ideas presented so far, including both bottom-up learning of propensities and top-down moral deliberation involving rules. The book concludes with some considerations on rights and responsibilities that may arise from the development of AMAs.
    • Comments: The monograph begins with providing reasons that justify the need of machine morality, i.e., endowing machines with the capacity of evaluating the moral consequences of their actions, which converts them in Artificial Moral Agents (AMAs). Such reasons are not only inherent to long-term futuristic scenarios, but relate already to present and near future situations. While not pretending full moral agency (requiring conscious understanding), which the authors situate on the top right corner of a two-dimensional graphical depiction of ethical sensivity and autonomy, AMAs can be developed trascending mere operational morality (e.g., of the kind embedded in safety mechanisms) towards functional morality, which implies decision making capabilities.
      The authors further analyse whether Humanity actually needs or wants machines to do ethical decisions and relate this to the increasing dependence on more and more sophisticated technology. When it comes to actually implementing AMAs, the authors examine first the top-down approaches, which try to derive a set of rules from some of the big ethical theories (deontology, utilitarianism) and convert them into a suitable algorithm. These rule-base approaches have to face ambiguity when confronted with the richness and nuances of real worlds situations. Bottom-up and developmental approaches, on the other hand, are flexible anough to adapt to a dynamical world, but learning can be quite slow and the resulting behaviors too loose to become hard constraints, which may be risky in some contexts. The possibility of merging both types of approaches is also considered, like in the aristotelian virtue ethics approach.
      Attempts to engineer software like decision support systems and multiagent software systems are examined as first steps towards reasoning about own decisions. Suprarational capabilities including emotions and empathy and their role on moral judgment are also considered. Franklin et al's LIDA cognitive cycle model is analyzed to some extent at the light of actual implementations of the ideas presented so far, including both bottom-up learning of propensities and top-down moral deliberation involving rules. The book concludes with some considerations on rights and responsibilities that may arise from the development of AMAs.
  • David J. Gunkel, "Robot Rights", MIT Press, 2018
    • Synopsis: After establishing a common playground for technicians, legal experts and philosophers by devoting some terminological efforts in the definition of "robot" and of "right", the author systematically analyses the topic on robot rights by combining (in a causal and in an adversative fashion) the two statements (and their negations) "robots can have rights" and "robots should have rights", and explores and discusses the arguments found in literature around the resulting assertions (e.g. "although robots cannot have rights, robots should have rights"). The author resorts to the philosopher Emmanuel Lévinas and his view on alterity as a way of definition of being (" I am because of the other, because I contemplate the face of the other"), to question the precedence of ontology over ethics in the uncertain terrain of the definition of conscience, sentience or life in the paradigmatic case of the intelligent robot, and to face the robot and the legal status we are willing to provide it.
    • Comments: The book stands out for the coherence in the systematic analysis, while at the same time being captivating thanks to the multiplicity of examples and the intellectual passion f the writing. No recipes are provided, but lots of food for thought. See a more detailed synopsis and comments about the book in the Spanish journal "Investigación y Ciencia", August 2019, "¿Deberían tener derechos los robots?" (in Spanish): https://www.investigacionyciencia.es/revistas/investigacion-y-ciencia/la-gran-explosin-de-la-vida-775/deberan-tener-derechos-los-robots-17746
  • Spyros G. Tzafestas, "Roboethics: A Navigating Overview", Springer International, 2016
    • Synopsis:
    • Comments:
  • Mark Coeckelbergh, "Robot Ethics", The MIT Press Esential Knowledge Series, 2022
    • Synopsis: The big topics on autonomous robot development and deployment, that raise ethical questions and dilemmas, are addressed in application-centered chapters, in the different spheres of human experience: industry and work, home and intimacy, health and care, mobility, war... The specific issues around android robots and environmental concerns related to robotics, as confronted to transhumanist and posthumanist views, are also dealt with in separate chapters. Beyond a mere account of ethical problems in these diverse areas, ethical tools and philosophical reflections are provided as well, or better said, are central to each chapter. When presenting industrial robots in advanced automation contexts, not just the well-known topic of job loss and human substitution is considered, but the very meaning of work itself, how this concept shapes society and how it may be changed by the disruptive potential of robotics. Social justice, human realization, and fairness are some of the touchstones that allow to test whether the resulting evolution points towards an egalitarian political order or towards an elitist dystopia. Discussing the introduction of robots in the domestic and personal space naturally raises questions about privacy, security and safety, ultimately related to human autonomy and control over our own lives. Deception, in contrast to the value of real human relationships, and the propagation of gender, racial, and other social bias, are other matters of concern related to personal robots. As for care robots, the accent is put on concerns affecting the dignity of the human user, in particular when they belong to vulnerable sectors such as elderly or disabled people. Working criteria for a “normative ideal of good care” are provided, and should be met when introducing robots in (health) care tasks. Self-driving cars lead towards the topic of autonomy in robots, which in turn relates to moral agency and to responsibility. Moral agency is thoroughly discussed, whether an entity without own motivations and emotions such as a robot can be considered as a full moral agent or not, and relaxed but effective alternatives are studied. The so-called responsibility gap” problem appears when autonomy and responsibility are not aligned, and it is specially challenging in the case where the actions of the (autonomous) robot cannot be predicted. The “many hands” that at some point or another intervene in the robot’s outcome (designer, manufacturer, programmer, user, the artificial system itself) make responsibility attribution more problematic. In the chapter devoted to androids, the ethical value of robot appearance is analyzed, in relation to the uncanny valley and deception, which is stressed by the human-like resemblance of this kind of robots. This is related to the moral standing of robots, now as moral patients, in the context of the moral standing of non-human beings such as (other) animals or the environment. The use of lethal autonomous weapons in warfare, such as the “killer drones”, is controversial, not only by facing again the problem of the responsibility gap, but also for the increased likeliness of starting a war, for the consideration of targeted killing as extrajudicial executions, for the easiness of killing at distance, or for if a machine should be allowed to kill a human at all, among other considerations. Finally, the last chapter displays robot ethics as a philosophical reflection on humans themselves (robot mirrors), and confronts the transhumanist and posthumanist views of potential evolution of humankind (which include man-machine hybridization) to an environmental perspective, and the role that robot ethics may play in the preservation of our natural surroundings.
    • Comments: The series this book belongs to aims at providing the essentials of the subject, in a concise, accessible, but also rigorous fashion. This aim is accomplished by the author: the book can be read by non-experts and experts alike, and it is neither too hermetic for the first nor too shallow for the second kind of readers. It is self-contained as for the concepts it manages, both on the technical as well as on the philosophical side, and a glossary helps to recall their precise meanings. It also provides references to further explore or deepen into the exposed topics. Although concepts such as responsibility, privacy, deception, or dignity are transversal to more than one of the application fields, stressing and elaborating on them at specific robot implementations allows to explore in more depth the related ethical concerns and their philosophical treatment in a practical, understandable way. As a philosopher, the author is trained to go to the essentials of each subject, which does not mean that the resulting text is dry or too abstract. On the contrary, a number of examples, both from fiction and reality, helps to fix the concepts and renders the reading agreeable.


  • Carme Torras, "The vestigial heart. A novel of the robot age", The MIT Press
    • Synopsis (from the backcover): "Imagine a future in which many human emotions are extinct, and "emotional masseuses" try to help people recover those lost sensations. Individuals rely on personal-assistant robots to navigate daily life. Students are taught not to think but to employ search programs. Companies protect their intellectual property by erasing the memory of their employees. And then imagine what it would feel like to be a sweet, smart thirteen-year-old girl from the twenty-first century who wakes from a cryogenically induced sleep into this strange world. This is the compelling story told by Carme Torras in this prize-winning science fiction novel. We meet Celia, brought back to life when a cure is found for her formerly terminal disease, and Lu, Celia's adoptive mother, protective but mystified by her new daughter. There is Leo, a bioengineer, who is developing a "creativity prosthesis" to augment humans' atrophied capacities, and the eccentric robotics mogul Dr. Craft. And there is Silvana, an emotional masseuse who reads old books to research the power of emotion. Silvana sees Celia as a living, breathing example of the emotions and feelings that are now out of reach for most people.
      Torras, a prominent roboticist, weaves provocative ethical issues into her story. What kind of robots do we want when robot companions become as common as personal computers are now? Is it the responsibility of researchers to design robots that make the human mind evolve in a certain way? An appendix provides readers with a list of ethics questions raised by the book."
    • Comments: Away from catastrophic dystopias, the not-so-far future society described in the novel is nonetheless disturbing, in a quiet subtle manner. Robots are not depicted as riotous revolts slaughtering Humankind, not even as paternalistic guardians smothering humans' free will by overcaring, but just as sophisticated machines able to help their PROPs and learn on a daily basis from their interactions to provide a better service. The problem, of course, lies not on the machines, but on the use we humans make of them. As a metaphor for what is actually happening already nowadays, with technological gadgets pervading our lives and intimacy, the personal robots described in the novel are so useful and efficient that they mediate in human relationships, and contribute to the generalized atrophia of emotions and of creativity. The educational system of this society is paradigmatic in the sense that it is not critical thinking what is fostered anymore, but efficient searching, and no holidays exist for autonomous configuration of time and occupations.
      A large amount of ethical questions can be rised from the novel. In fact they are, in the Appendix provided at the end, by the same author: questions about (ethically) desirable traits of robots, about their role in education, about their interaction with humans and the consequences on human dignity, social responsibility of robots and their morality. Seldom a fictional work focuses at so many ethical aspects at the same time, which come up naturally within the engaging story and in no way seem to be forced. Even more interesting is that it pinpoints at the consequences of the widespread use of personal robots on human society, the interhuman relationships, emotions and creativity, which is rarely dealt with in the literature. And it does it in a quite plausible way, the depiction of such a society is quite credible, which makes it even more perturbing.
      Trivia: The novel plays hommage to Isaac Asimov in the name of one of the secondary characters.

Aware of the role played by scientific research within the contemporary society, it is a shared feeling in our institute that our research has more or less direct incidence on our surroundings, and that ethical aspects have to be cared for. Some senior researchers are quite commited to the ethical aspects of robotics, but even more interesting is that young researchers in their formative period show concern about these issues, which is quite promising for the future.

Ethics during experimentation

Research in human-robot interaction (HRI) may require at some point to conduct experiments with human subjects. This may occur, for example, during skill acquisition, when human demonstrations are required for the robot to learn by generalizing the observed human attempts to perform the skill. This learning from demonstration scheme typically involves motion capture, force sensing, and/or video recording. Or the researcher may want to test new modalities of the robot communicating with human subjects. Moreover, it is often necessary that these subjects have no technical background, so as to favour unbiased testing. More delicate are those situations, such as those arising in rehabilitation robotics, where the human subjects have some kind of disability. In any case, the research project has to comply with the corresponding protocols regarding ethics, which in practice means to obtain a favourable evaluation of one of the mother institutions.

In the case of CSIC, the Ethics Committee requires the following documents for validating research involving experiments with human subjects:

  • A project report, with a brief description of the involved research, aims and methodology. Special attention should be payed to provide a more detailed description as for the risk lowering measures, privacy protection, personal data gathering and management, etc.
  • The application form, which has three sections. The relevant one is part A, "research with human beings, human samples and/or data". The researcher can retrieve this form from the CSIC intranet (Gestión Científica ˃ Ética en la Investigación ˃ Ética y ciencia).
  • Information sheet (i.e., information which is provided to the subjects about the experiments and the resulting data management).
  • Document of informed consent (i.e., form provided to the subjects where they can state acknowledgement of the main features of the experiments, and their willingness to participate and acceptance of the conditions of data management and storage).

These are the basic documents, but if the experiments are to be carried out in other premises than the research centre of the PI, or if there is some physical or psychological harm risk, or the subjects have some kind of disability, additional documents are needed, such as a favourable report from the committee of ethics of the collaborating institution (the one providing the premises and/or the subjects) or of a medical doctor supervising the well-being of the subjects during experimentation. Data gathering, management and storage deserve special attention, so as to guarantee privacy of the subjects, and no other uses than the ones the subjects have provided their consent for. Guidelines on data protection can be found in this EU document

As for the UPC, the newly created (2020) UPC Ethics Committee does also act as a consultative and advisory body for the university community. They provide links to request forms for ethical supervision of research, depending of the modality of ethical concern (stem cells, human subjects, human cells or tissues, personal data, AI, etc.)(in Catalan), and a guide for requesting such ethical guidance (also in Catalan). The documents required to conduct experiments involving interaction with human subjects are basically the same as for the case of CSIC, as described in this guide.

Carme Torras is currently Vice-president of the CSIC Ethics Committee. Pablo Jiménez is external expert evaluator for this committee, having performed the evaluation for two projects in 2021.

UPC compromise of no war-related research

IRI adheres to one of its mother institution's compromise of fostering a culture of peace. In particular, the UPC expresses in its statutes that the knowledge generated in the university has to serve a culture of peace, and that it has not to participate in research projects with military purposes.

Article 5.4 "La Universitat Politècnica de Catalunya ha de cultivar l'esperit crític de la ciència i de l'art i els principis de la llibertat de docència i de recerca, i ha de fomentar l'ús pacífic i respectuós amb els drets humans dels coneixements científics, tècnics, artístics i humanístics, i els ha de posar al servei de la construcció d'una cultura de pau." (Article 5.4 "The Polytechnical University of Catalonia has to cultivate the critical spirit of science and arts and the principles of freedom of instruction and research, and has to promote the peaceful and respectful use with the human rights of the scientific, technological, artistic and humanistic knowledge, and has to put it at the service of constructing a culture of peace.")

Article 134 Principis generals [...] "La Universitat ha de procurar que la recerca que s'hi desenvolupa sigui conseqüent amb els seus principis informadors i contribueixi a la creació dels coneixements i la base científica necessaris per al desenvolupament humà sostenible, fomenti la pau i el desenvolupament i contribueixi a la desaparició de les desigualtats socials i econòmiques entre les persones i entre els pobles. La Universitat no ha de participar en projectes de recerca amb finalitats bèl·liques."] ("The University has to procure that the research which is being developed be consequent with the institution's principles and that it contributes to the creation of the necessary knowledge and scientific bases for the sustainable human development, that it fosters peace and development, and that it contributes to the abolition of social and economical inequality among individuals and communities. The University must not participate in research projects with war purposes.")

Projects, theses, surveys

  • Predoc position offered at the María de Maeztu project: "Ethics in Robotics and AI: a historical and intercultural perspective", WP involved: WP7, Supervisor: Carme Torras, Cosupervisor: Begoña Roman (UB). This position is held by Júlia Pareto Boada, who is working on her doctoral dissertation "Robótica e Inteligencia Artificial: (Re)Flexiones Éticas para el Apoderamiento Humano en una Era Disruptiva".

  • P. Jiménez participates in the R-ICIP (Institut Català Internacional per la Pau) project "Sistemes d'armes autònomes letals (SAAL): els reptes de l'adopció d'una regulació internacional sobre ús, control operatiu i disponibilitat per empreses privades"(Lethal autonomous weapon systems: the challenges of adopting an international regulation on the use, operational control and availability for private enterprises). Project leader is Prof. Jordi Bonet, from the Faculty of Laws, University of Barcelona (2018).

  • Ana Puig-Pey elaborates, within the TERRINet Project, the TERRINet ELES Questionnaires, which are later applied to the project's partners.


Be at high school, at undergraduate or postgraduate level, ethics related to robotics and AI is a subject with special relevance for the generations in their school and university years. At IRI we are happy to contribute to the education of the people who most probably will have to face the ethical challenges of advanced robots in everyday life.

  • 2017

    "Joves, Ciència i Ètica" Program: Participation in the 2017-18 edition of this program organized by the Fundació Catalana per a la Recerca i la Innovació and l'Obra Social "la Caixa", devoted to Roboethics. Five high schools (Príncep de Viana, Sant Andreu, Zafra and Icària, from Barcelona, and Institut Can Puig from Sant Pere de Ribes) took part in the project, with a total of 320 students. A first session on Robotics and Roboethics was given to the involved high school teachers by Pablo Jiménez on October 4th, in the Auditori de la Fundació Catalana per a la Recerca i la Innovació, Barcelona. Afterwards, during a whole week (23-27/10/2017), Gerard Canal, Sergi Foix, Anaís Garrell, and Joan Vallvé visited the high schools and worked through these four topics, respectively:
    • Limits of Artificial Intelligence
    • Impact of Robotics on the labour market
    • Robotics and affective relationships
    • Robotics and warfare
    The students worked further on their own on these themes, and on November 29th the final conclusions, together with their IRI mentors, were drawn and exposed in public in the Conference room of Cosmocaixa.The congress was closed by a conference given by Prof. Carme Torras, "Dissenyant l'assistent perfecte: Robòtica, ètica i literatura".

  • 2019

    Carme Torras "Social Robotics and AI: Present and Future" within the "Digital Transformation" Postgraduate degree, Escola d'Administració Pública de Catalunya, 09/10/2019 Carme Torras "Assistive Robotics and AI: Research and Ethics" within the Master in Business Administration, ESADE School, Sant Cugat 12/11/2019

    Carme Torras "Assistive Robotics and AI: Research and Ethics" within the Master in Business Administration, ESADE School, Sant Cugat 12/11/2019

  • 2020

    Carme Torras and Pablo Jiménez complete the teaching material "Intel·ligència artificial i robòtica", to be used in the Màster en Ciberdelinqüència and the Postgrau en Protecció de dades, both from the Universitat Oberta de Catalunya. This material (over 60 pages) has a part devoted to ethics in AI and robotics.

    Carme Torras, "Ètica en la Intel·ligència Artificial" Grau en Ciència i Enginyeria de Dades, UPC

    Pablo Jiménez lectures "Ethics" in the Postgrado BarcelonaTech (UPC)-Accenture "Digital Innovation for Business Development" (3h, online)

    Pablo Jiménez prepares and records the MOOC course "Robótica: nociones básicas, aplicaciones y ética" for the program Professorship of the Instituto de Altos Estudios Universitarios (1.5 h)

    Pablo Jiménez gives a talk on Roboethics at the high school Tecnos in Terrassa, on February 1st, 2018, with the title "Roboètica: la dimensió ètica de la robòtica". Around 50 13-16 yo students, as well as some parents and teachers are present.
    Pablo Jiménez participates as invited robotic expert in the REELER workshop "Robot Expert Panel II. Roboticists engage in sociodrama".

Scientific talks, debates and Publications

The scientific community, as well as researchers in humanities, law, economics, psychology, sociology, etc. are increasingly concerned about the ethical impact on the individual and on society of AI and robotics. Be it within the roboticists community, be it in interdisciplinary meetings, members of IRI have been participating along the years in the debate of the ethical implications of their own research or of robotics in general. Details on these contributions can be found here. [link a la llista que es detalla a continuació]

Outreach and dissemination talks to general public

The ethical dilemmas related to advanced robotics and AI concern the whole society. Thus, it is very important that the experts do not only provide a realistic and understandable picture of their research, but also that they share their reflections about the ethical implications of robotics and AI. Different members of IRI have shown once and again their commitment in this dissemination task. Besides appearances and interviews in the media (newspapers, radio, TV), they have offered along the last years diverse talks to the public (some of them online during the pandemic), as listed in this page.

Internal talks, seminars and debates

15/12/2017 Meeting of people in IRI concerned about ethic issues related to robotics. After an introduction by V. Vílchez, the involved researchers in the activity "Joves, Ciència i Ètica" explained their experiences in the interaction with the young students, as well as the conclusions elaborated by the students groups. Next, P. Jiménez exposed the results obtained on a poll on Robotics and Roboethics filled by the students before and after the activity, and compared them with the results obtained on the same poll by experts, i.e., members of IRI. The flaws of this first poll were analyzed, with the idea of making better future versions, to uncover the expectations and opinions of non-expert population groups. An interesting debate followed on different topics related to Roboethics, mainly on affective dependence and on robots and warfare.

23/07/2019 1st Perception and Manipulation Group (P&M) Debate "Is it possible to implement/program ethics in robotics?"

15/04/2020 4th P&M Debate: "Is robotization going to raise social inequalities?"