‘Computers are already able to arrive at better judgements than people’

Sifting and evaluating information is second nature to lawyers. But what if computers are able to process information more effectively? This is the tipping point we are now reaching. Computing power has increased and software has been refined to such an extent that legal acumen is now programmable. Legal teams working in business are now genuinely able to rely on data technology to back up their own legal skills. 

Text Annemarieke Noordhoff Photos Erik van der Burgt

Interview with Jaap van den Herik, professor of Law and Computer Science at Leiden University.

Jaap van den Herik is professor of Law and Computer Science at Leiden University. In an interview with Luc van Daele from Legadex he talks about data technology’s impact on legal work - both in litigation and in the corporate environment.

Intuitive decisions
Twenty-five years ago, Jaap expressed the opinion that one day computers would be able to make better legal judgements than people. And he was by no means the first. This prediction was made as long ago as the 1960s, and over the following 20 years a select group of researchers worked to flesh this out. One of them was Jaap. At the time he was completing a doctorate on computer chess and how to program the intuitive decisions that chess players make. In 1988 he was appointed professor at Leiden University. ‘If computers can learn to play chess, then they can also learn to make legal rulings,’ was his view. In 1991 the front pages of the national newspapers carried his inaugural address with the question: ‘Can computers make legal rulings?’.

‘Look at where we are now,’ he declaims. ‘A recent large-scale trial using 600 human rights cases showed that in 79% of the cases the same ruling resulted using Artificial Intelligence as was handed down by the European Court, the ECHR. This is a major step, certainly for lawyers, although scientists are less impressed by this.’ Jaap is himself a scientist by profession. From his perspective, a trial of this kind represents a breakthrough only if it takes in 99.9% of the cases. But he has spent sufficient time in the legal world to realise that it certainly implied a breakthrough for lawyers - and for society as a whole. ‘I don’t want to crow about this success, because there is a lot that still has to happen technologically. Nevertheless, I definitely expect that we will get there, roughly in 2080. Computers will then be able to make better legal judgements than people and will be better at weighing evidence and making ethical rulings. But what does this say about us as human beings? As lawyer, judge or banker?’

If a computer can learn to play chess, you can also teach it to make legal rulings.

In the corporate environment
Data technology is increasingly a logical component of litigation. Luc also uses data technology to back up legal work, but mainly in a corporate environment. Luc: ‘Within companies, particularly in an international setting, several people are often looking for the same data from different angles. They are looking for the most recent version, would like to know whether the source is correct and whether the data are complete. This used to be done in a terribly inefficient way, and that is often still the case. In 2008 I launched Legadex along with my partner. We were the first alternative legal service provider in the Netherlands, with a sharp focus on the work process and information management. Data analysis has gradually gained an increasingly important role here.’ Other players are now directing their attention to using data technology in the legal sphere, even if the number of areas where it is being applied remains limited. These areas include E-discovery, investigation of fraud and competition law, as Luc makes clear. ‘If you employ data technology for the operational legal business process, there are an increasing number of useful applications.’

Jaap is enthusiastic about this. ‘Research is making huge progress here, but we can’t do without business. They have the data needed to test scientific findings. Players like Legadex are in companies’ nervous systems, on top of new developments. Let’s be honest: lawyers are often a bit reserved and approach new technology with caution and meticulousness.’ Jaap’s implicit message is that much greater progress can be made with a progressive commercial player prepared to go out on a limb.

Never again samples
High-performance computing has provided huge stimulus for the giant strides taken by data technology. The term itself would appear to apply rather to a frenetic ICT business than the stately university building on a quiet canal in Leiden. The huge increase in the capacity and power of data technology has made an impact, and this has been felt in scientific research as well. ‘Take for example the research into the run-up to the financial crisis at the end of the 1920s. Researchers select two or three newspapers, going through one or two days from one week over a number of years. A properly conducted representative study in accordance with a method that researchers have used for their doctoral theses for decades and that has generated a host of results. But what would happen if the researchers were able to go through all the newspapers from every day in the years leading up to 1929? That could definitely create a completely different picture.’ Data technology and high-performance computing spell the end of stratified sampling, a scientific method that has been uncontroversial for decades. A completely new and powerful toolbox is arising in its stead: ‘new statistics’. ‘This may potentially generate completely new scientific insights,’ Jaap predicts. He lists Brexit and the US presidential elections as examples where ‘new statistics’ would never have led to a surprise of this nature.

Data technology and high-performance computing spell the end of sampling, a scientific method that has been uncontroversial for decades.

Jaap is a fanatical chess player. It holds boundless fascination for him as a mathematician and IT specialist. As long ago as the 1970s he was discussing chess programs with his academic supervisor. He believed that a computer could possibly beat a grandmaster, whereas his supervisor thought this was impossible. In the supervisor’s view, a grandmaster would have something that a computer could never have: intuition. ‘And that’s where he was wrong,’ Jaap says – certainly with respect for the supervisor, but with the expression of a chess player who has just won his match. ‘Intuition can certainly be programmed. With playing chess, this is the answer to the question: what’s the best move?’ Luc is not a chess player – at least not at Jaap’s level, as he freely acknowledges. ‘But I do understand the system and its significance. You can fathom the system if you have sufficient computing power.’

Mankind and machine
Luc sees possibilities in the corporate environment. ‘With M&A processes and document review projects you often see that the thinking still takes place in existing information structures. The belief is that we’ve got these folders, dossiers, documents – we can’t get away from the fact. And we need people to plough through the material and to evaluate and classify it. But what if both the preselection and the substantive research could largely be automated? That would provide the banker, the CFO or the General Counsel (GC) with a complete picture more quickly, including all the relevant links.’

Jaap adds: ‘Computers are able to evaluate much more data than people. Moreover, they can evaluate this data better.’ Luc: ‘Sometimes GCs tell me that they are sceptical of data technology because they demand 100% certainty. But you can miss cases, even if you bring in the most expensive barrister, certainly if you have an inadequate overview of your information. Artificial Intelligence can really help here. At the moment a computer cannot as yet be used to replace the human eye and judgement 100%, although certainly to assist the human process very considerably.’ 

Data analysis platforms also offer new areas of application. Luc mentions the example of selling a business. ‘A data analysis platform can perhaps transform the traditional “passive” folder structures into a virtual data room. The different people involved on the part of the seller are already using data analytics in the preparation. The seller’s consultants gain an overview more quickly using this technology.’ Would the seller be able to allow the buyer access to information on the company in a searchable and analysable platform? Luc is certain. Legadex has already done so on one occasion. ‘It makes the provision of information extremely transparent and intelligible for a potential buyer. There is of course the question of how far the seller wants to go here – in a tactical sense,’ Luc says.

The crucial question
The precise definition of queries, the underlying search queries for instructing the technology to analyse in a relevant way, remains an intensive process. The question is how do you phrase the correct queries to get to the answers required? What are the elements that you want to combine, what does the software need to put into its evaluation? Consider for example finding change of control clauses in contracts in the course of an M&A due diligence. Luckily data technology is becoming ever more intelligent, Jaap says. ‘You’ve got supervised learning and unsupervised learning. With the latter, data can cluster itself on the basis of corresponding characteristics that the software discovers of its own accord. This means that new links become clear. For example there was the case of a trial in which 12 charges were being dealt with. The software investigated all the material and indicated that there were grounds for a 13th charge.’ Posing the correct question is a risk in any due diligence process – even with manual investigation. This problem is currently got around by having the same search conducted 10 times in India, combining the results and filtering out the margin of error in this way. This is extremely laborious and can be done much more quickly and effectively by software. As an additional advantage, data technology renders the results visual, i.e. translates them into graphs, histograms and other visual aids. Traditionally visualisation has been a step in data technology (see box, ed.). But it is currently being used throughout the entire process to help visualise fresh insights. It is more than merely an end-product.

Everything, even scent
Businesses still sometimes fail to see the kinds of information and data that can be used. ‘Lawyers think in words and see e-discovery primarily as a form of text mining,’ Jaap notes. ‘But what about audio files? Even scents have for the past 20 years been translated into ones and zeroes – and can as a result be searched. This is a consequence of the internationalisation of flower auctions. All of our sense perceptions can be coded and used in data analyses.’

The downside
Is there another side to all these possibilities? What about data protection? Luc and Jaap see a sharp rise in interest in this issue. Along with his colleague Cees de Laat, Jaap has recently published something on this topic. ‘Ideas on data protection are shifting extremely fast. For example I know of a case dating back to 2012, where a young postgraduate and his assistant discovered that surveys conducted by a bank could simply be downloaded. For all participants, to be sure, only by altering the ID number in the web address. The interesting thing in this case is that you can see all the details going past (see box, ed.). The researchers took the ethically questionable decision to continue downloading, first a single copy until they had thousands of surveys. A decision that would now be taken differently, and with the aid of an ethical commission. A single download may possibly still be seen as scientific curiosity, but thousands - that is a clear violation of data protection rules. Are there researchers who would still do this? In this example the security was also not up to the mark, because the developer – a solo operator – was not given this as a task to carry out. Another relevant question is whether a bank would take this risk today, just a few years later? Precisely. This makes very clear how rapidly ideas on data protection have changed for the better in a short time.’ Just as have ideas on allowing computers to take ethical and moral decisions. As surreal as this now seems, the more real and logical it will perhaps seem in the decade to come. In any event, data technology is for the General Counsel and CFO a tool that opens up a wide range of options.