In 2013, the state of Michigan replaced the 25-year-old computerized system of the unemployment insurance agency (equivalent to OAED in Greece) with a modern automated system called MiDAS (Michigan Integrated Data Automated System) to speed up processes and reduce the agency’s operational costs. By having MiDAS take on the bulk of the work, staff could be reduced by one-third, dismissing 400 employees, which is exactly what happened. From the very beginning, MiDAS’s algorithms proved to be remarkably capable: unemployment benefit applications flagged by robotic algorithms as fraud attempts quintupled, while the fines imposed by MiDAS skyrocketed from 3 million dollars (the average of previous years) to 69 million. MiDAS’s severity was extreme: the usual fine it imposed was 400% of the benefit amount. After two years of operation, several thousand lawsuits against MiDAS’s decisions were required (95% of which were upheld) and intervention by federal authorities, in order for the state to pull the plug on the robotic bureaucrat.
Michigan is not the only case: robo-adjudication, i.e. the robotic process of making judicial and administrative decisions, is an emerging trend. Australia has been using a system similar to MiDAS, Centrelink, since 2016, with the same results. And while the Australian government claims that the system is working as it should and within its specifications, an independent investigation concluded that Centrelink was designed primarily to reduce costs and deliberately operates in a way that rejects a large percentage of applications or even generates revenue through fines.
Corresponding applications are already being tested in criminal courts as well. Last August, IBM, in collaboration with some U.S. juvenile courts, launched a pilot program of an artificial intelligence system that is supposed to “assist” judges. The program processes all testimonies and evidence of a case and delivers a summary of its findings to the human judge, on the basis of which the verdict is reached. Put simply, under the pretext of the volume of evidence, the robot will direct—or manipulate—the judge toward the “correct” decision.
Last November, a team of researchers from University College London and the University of Sheffield also built an AI “judge” program, which they tested on cases from the European Court of Human Rights. The program was fed the case files of 584 cases and, after processing the data, issued its own decisions, which matched those of the court in 72% of instances.
Therefore, in the near future, justice will not only be blind but also binary; and perhaps we will be allowed the choice between a robo-judge and Judge Dredd.