Popular in Law

Law

Diversity and Social Mobility in the UK legal profession

Law Society I. Stephanie Boyce

Internet Law

The Law of AI for Good

University of San Diego Orly Lobel

Human Rights Law

Transforming Restorative Justice for Transitional Settings

University of Nottingham Kerry Clamp

Law

Surrey’s Silence: Subpart F and the Swiss Subsidiary Tax that Never Was

Boston University Steven Dean

Criminal Law

Prosecuting Misdemeanors

American University Jenny Roberts

Human Rights Law

Siege Starvation: A War Crime of Societal Torture

Tufts University Tom Dannenbaum

Corporate Law

Data Valuation and Law

University of Southern California Jordan Barry

The corporate board in an age of collaborative intelligence and complex risk

View more insights in Professional - Law - Corporate Law

View more figures from Swinburne University of Technology

View the page of Helen Bird

View the page of Natania Locke

Read the Study

Helen Bird and Natania Locke explains how AI advocates define ‘administrative’ and ‘subjective’ governance roles and argue AI would at least replace ‘administrative’ operations to avoid micro governance difficulties.

Image courtesy of interviewee. January 11, 2024

Generating Workspace

Export as PDF

This video video discusses the integration of artificial intelligence (AI) into corporate governance, particularly in the context of risk management and organizational structure, as highlighted in a case study involving Westpac, an Australian bank. The research was part of the edited book "Technology and Corporate Law" published in 2021, emphasizing how AI can aid in reducing information asymmetry between management and boards of directors.

Key points include:

1. **Corporate Governance and AI:** The work underscores a shift in corporate governance, where AI could assist in oversight functions, facilitating better communication and decision-making between levels of management and boards.

2. **Westpac Case Study:** Westpac faced significant issues regarding compliance with anti-money laundering laws, resulting in a historic fine of $1.3 billion. This video video scenario illustrated the failures in risk management and technology oversight, revealing the inadequacies of both management and board actions.

3. **Human Involvement:** The authors stress that while AI may enhance efficiency, it cannot completely replace human directors. Human judgment is essential, particularly for making value-driven decisions that AI cannot handle effectively.

4. **Operational Challenges:** The case showed that technological tools require skilled personnel for their design, implementation, and oversight. In Westpac's case, inconsistent corporate knowledge and management indifference contributed to its failures.

5. **Future of Governance:** The article predicts a potential flattening of organizational structures as AI reshapes the traditional hierarchies in companies, allowing for improved flow of information. However, it emphasizes that AI's effectiveness hinges on clearly established objectives, ongoing monitoring, and the continued presence of human oversight.

6. **Conclusion:** The authors argue against the concept of a completely AI-driven corporation, positing that human directors are necessary for effective governance. They contend that AI should augment human capabilities rather than replace them, enabling better decision-making and oversight processes.

In summary, the text argues for a balanced approach to implementing AI in corporate governance, suggesting that while AI is beneficial for risk management, human oversight remains critical for making nuanced, ethical, and strategic decisions.

Language:

Export as PDF

X-TIMESTAMP-MAP:LOCAL=00:00:00.000,MPEGTS=900000

00:00.035 --> 00:01.565
This work was published as part

00:01.665 --> 00:04.725
of an edited book entitled Technology

00:04.785 --> 00:08.565
and Corporate Law, how Innovation Shapes Corporate Activity,

00:09.465 --> 00:12.565
and was published in 2021 by Edward Edward Algar.

00:13.105 --> 00:15.965
And our focus was specifically on the role

00:15.965 --> 00:19.205
and functions of boards in this time,

00:20.105 --> 00:23.565
and specifically in the way that risk governance,

00:24.105 --> 00:25.965
is being influenced and might be influenced

00:26.225 --> 00:29.805
by the acceptance of artificial intelligence systems.

00:30.825 --> 00:33.405
Uh, another important thing is there were two articles

00:33.405 --> 00:35.445
published, more or less this, you know, just

00:35.445 --> 00:40.005
before, we started this research one by Martin Petrin,

00:40.545 --> 00:45.045
and published in the Colombian business, review,

00:45.065 --> 00:49.165
law review, where he postulated that he sees,

00:49.565 --> 00:51.925
a fusion between management,

00:52.225 --> 00:54.685
and board over the long run so

00:54.685 --> 00:57.925
that you don't have the two tier system anymore, right?

00:58.625 --> 01:03.365
Uh, another, article this time by John Almer

01:03.505 --> 01:07.565
and host Mueller postulated the possibility

01:07.745 --> 01:11.165
of fully AI driven corporation, so sort

01:11.165 --> 01:13.045
of the self-driving corporation.

01:13.745 --> 01:17.085
So our intention was really to put a little bit

01:17.085 --> 01:20.605
of reality back in the minds of people in this area,

01:20.945 --> 01:25.885
and just to emphasize that it's much easier said than done

01:26.745 --> 01:30.325
to incorporate these systems in already established running

01:31.325 --> 01:33.005
hierarchical organizations,

01:33.265 --> 01:34.925
and that there are some practical

01:34.925 --> 01:36.405
conservations to keep in mind.

01:37.505 --> 01:40.845
Um, and another thing that happened sort

01:40.845 --> 01:41.965
of while we were busy

01:41.965 --> 01:45.405
with this is there was a very important case in Australia,

01:45.505 --> 01:49.165
the Austrac and Westpac decision, which we use

01:49.305 --> 01:51.525
as an example to illustrate some

01:51.525 --> 01:54.725
of these practical difficulties,

01:54.875 --> 01:56.645
that we wanted to highlight.

01:57.075 --> 01:58.965
Picking up from where Natan left off,

01:59.165 --> 02:01.845
I thought I just explained essentially the basics

02:01.845 --> 02:03.045
of the Austrac litigation.

02:03.675 --> 02:05.285
Westpac in Australia is one

02:05.285 --> 02:06.885
of the large four banks that we have.

02:07.165 --> 02:09.445
Probably at that time it was the second largest spy

02:09.445 --> 02:10.845
market capitalization.

02:11.545 --> 02:14.445
But what in, in essence this is about is money laundering,

02:14.445 --> 02:18.085
which is an issue that's, transatlantic Atlantic,

02:18.185 --> 02:19.245
and, and, and global.

02:19.385 --> 02:23.005
And as a problem, AUSTRAC was the regulator in Australia

02:23.105 --> 02:24.245
of anti-money laundering

02:24.245 --> 02:26.005
and counter-terrorism financing laws.

02:26.345 --> 02:30.445
And it sued Westpac for 23 million alleged breaches

02:30.465 --> 02:32.365
of Australia's money laundering laws.

02:33.265 --> 02:35.845
And in its particulars of claim, it asserted

02:35.845 --> 02:36.885
that the failures

02:36.985 --> 02:39.725
or there were failures by simply not making proper reports.

02:39.785 --> 02:42.605
But those failures in turn were actually the result

02:42.605 --> 02:45.045
of some systemic failures inside Westpac,

02:45.275 --> 02:48.805
that they didn't properly control their risk environment.

02:49.055 --> 02:51.205
There seemed to be indifference by management as

02:51.205 --> 02:53.245
to the problems, and the board had inadequate

02:53.245 --> 02:54.605
oversight of what was going on.

02:55.305 --> 02:56.605
And in effect, this was the case.

02:56.605 --> 02:58.605
Therefore, that went really very much to the nature

02:58.605 --> 03:01.365
of the relationship between the technology that was used

03:01.365 --> 03:05.005
to operate the money laundering arrangements inside Westpac.

03:05.425 --> 03:07.325
Um, that is the way they dealt with that.

03:07.445 --> 03:08.605
Secondly, the governance

03:08.625 --> 03:10.925
and the risk inside a major corporation

03:11.385 --> 03:14.325
and these wallets, Australian-based case, is an issue

03:14.325 --> 03:16.565
for every bank across the world at this time.

03:17.305 --> 03:18.685
Now, Westpac, it has to be said,

03:18.685 --> 03:20.805
filed a defense in which they largely denied all

03:20.805 --> 03:23.205
of these allegations, but they did end up settling it

03:23.205 --> 03:25.405
after this, article was published

03:25.745 --> 03:29.485
and paying a fine of $1.3 billion towards austrac,

03:30.255 --> 03:32.525
which is in fact the largest fine ever paid

03:32.545 --> 03:33.965
by an Australian corporation

03:34.465 --> 03:36.045
for breaching money laundering laws.

03:36.625 --> 03:39.485
Uh, but the, at the heart of it is an argument about

03:39.625 --> 03:43.125
how the technology inside, Westpac was properly managed

03:43.505 --> 03:46.125
and itself lends a, a, a conversation

03:46.305 --> 03:50.205
to a conversation about whether AI is the solution given the

03:50.205 --> 03:52.125
problems that Westpac, experienced.

03:52.945 --> 03:56.485
So, having regard to works previously published,

03:56.865 --> 04:00.445
we agree that, artificial intelligence systems

04:01.345 --> 04:04.005
has every potential to, to

04:04.225 --> 04:07.405
as assist in breaching the information asymmetry

04:07.405 --> 04:09.765
that exists in in organizations

04:10.545 --> 04:14.685
and thereby reducing that problem where a lot

04:14.685 --> 04:16.125
of the information that ought

04:16.125 --> 04:19.005
to reach the board is really housed in different levels

04:19.065 --> 04:21.245
of management inside of the corporation,

04:21.615 --> 04:24.485
which is a typical problem for governance,

04:24.665 --> 04:26.085
of corporations generally.

04:26.785 --> 04:30.005
Um, and so when that happens, we do foresee

04:30.005 --> 04:32.845
that there might be, a, a flattening

04:32.845 --> 04:35.805
of the organizational structures inside of corporations

04:36.035 --> 04:40.725
because you'll need fewer levels of management in between

04:41.705 --> 04:45.725
to to lead to, you know, this information being sort

04:45.725 --> 04:48.765
of trickled up to, to the board of directors.

04:49.275 --> 04:52.525
However, we make, we very strongly make the point

04:52.875 --> 04:56.205
that the use of this technology is only as good

04:56.345 --> 04:58.205
as the technology actually working.

04:58.945 --> 05:02.085
So being fit for purpose as well as the receivers

05:02.085 --> 05:04.365
of the information that is the, the human actors,

05:04.465 --> 05:08.125
the directors at the top, making adequate use

05:08.425 --> 05:10.125
of the information,

05:10.125 --> 05:11.685
that they receive from their systems.

05:12.385 --> 05:17.205
Um, and again, we can illustrate this, by reference to

05:17.205 --> 05:18.965
that AUSTRAC and Westpac case.

05:19.705 --> 05:22.165
So taking up that point, really,

05:22.165 --> 05:24.765
essentially there were various different platforms that,

05:24.765 --> 05:26.925
Westpac was using in order to,

05:27.225 --> 05:29.485
get information about transfers overseas

05:29.585 --> 05:32.645
and also report what as required to austrac.

05:32.985 --> 05:35.005
Now, those systems were regularly being updated,

05:35.055 --> 05:37.165
there were constant goals for that work to be done,

05:37.705 --> 05:39.525
but the people in charge of looking

05:39.525 --> 05:42.685
after that weren't all were, shifting population

05:42.685 --> 05:45.125
of staff with the various levels of experience.

05:45.295 --> 05:47.565
There was no consistent corporate knowledge.

05:47.825 --> 05:50.245
And so what you saw happen in, in a sense,

05:50.525 --> 05:52.325
occurred on two levels at Westpac.

05:52.325 --> 05:55.765
Firstly, you had a management problem within the very large

05:55.765 --> 05:57.045
organization of Westpac.

05:57.280 --> 05:59.845
There was, the, the subsidiary in charge

05:59.865 --> 06:03.925
of money laundering, was the Westpac institutional bank.

06:04.545 --> 06:06.885
And, what was discovered was

06:06.915 --> 06:08.565
that when there were problems

06:08.565 --> 06:11.925
and reports being, given to management, management,

06:12.175 --> 06:14.005
first off didn't take them seriously.

06:14.625 --> 06:17.885
And secondly, was rather lax in responding to them,

06:18.345 --> 06:19.645
partly one suspects

06:19.845 --> 06:21.805
'cause they didn't fully understand the import

06:21.825 --> 06:24.125
or the seriousness of what was going on.

06:24.195 --> 06:27.325
They were updates, there were things wrong with the system,

06:27.325 --> 06:28.605
there were capability issues.

06:28.735 --> 06:30.325
These were very technical words that

06:30.325 --> 06:33.125
unless you are very deeply steeped in this area yourself,

06:33.125 --> 06:34.845
you don't necessarily appreciate.

06:35.585 --> 06:36.965
And, you know, you might say, well,

06:36.965 --> 06:38.645
what would you have done if you became

06:38.645 --> 06:39.805
aware of some of these issues?

06:40.025 --> 06:42.565
And what at the very least you would've expected

06:42.585 --> 06:44.925
to have happened was there would've been an investigation

06:45.505 --> 06:48.365
and steps taken to remediate the gaps identified in the

06:48.365 --> 06:51.085
system and to address its risk management processes.

06:51.745 --> 06:54.645
But it wasn't, as I said, just enough to fail to act.

06:54.645 --> 06:55.925
It was also a laxness.

06:56.025 --> 06:57.685
And it took them 18 months even

06:57.685 --> 07:00.805
after the first of these reports started to be received

07:01.065 --> 07:04.165
before anyone decided to undertake some kind of review.

07:04.825 --> 07:06.605
And you could say, I guess

07:06.605 --> 07:09.205
therefore, that the technology worked in the sense

07:09.205 --> 07:11.445
that somebody within the organization thought

07:11.445 --> 07:12.525
to say it's not working.

07:13.105 --> 07:14.885
But it didn't, it failed in the sense

07:14.885 --> 07:16.485
that the management didn't appreciate

07:16.545 --> 07:18.365
or understand its critical nature.

07:18.985 --> 07:20.845
And that would be the same with ai,

07:20.915 --> 07:23.765
even if this is perhaps not necessarily strictly speaking

07:24.105 --> 07:25.805
ai, it was more a machine learning

07:25.815 --> 07:27.125
based tool that they were using.

07:27.865 --> 07:29.285
Um, secondly,

07:29.865 --> 07:31.805
the management problem caused a second role,

07:31.805 --> 07:34.565
which was the failure of the board to do proper oversight.

07:35.105 --> 07:38.085
Um, they didn't act urgently, they didn't inform the board.

07:38.185 --> 07:40.445
It took another year after all of this happened,

07:40.445 --> 07:42.085
before the board was even made aware of it,

07:42.305 --> 07:46.085
and they in turn, sat on their response, to that.

07:46.665 --> 07:49.965
So we see that information came in some forms,

07:50.335 --> 07:53.205
inadequate forms, uninformed forms to those

07:53.205 --> 07:54.605
who were making critical decisions,

07:55.265 --> 07:57.645
but it failed to actually result in action.

07:58.185 --> 08:00.165
And that is gonna be the case.

08:00.475 --> 08:03.525
This is just one small component of what Westpac does

08:03.625 --> 08:06.205
and one small set of technology that it uses

08:06.585 --> 08:08.285
to manage its money laundering program.

08:09.025 --> 08:11.885
Um, and how we could see this extrapolate

08:11.905 --> 08:14.885
to all AI makes you realize the problems are

08:14.885 --> 08:15.925
ahead for all companies.

08:16.265 --> 08:20.285
And Westpac in particular is a, a very digitalized business,

08:20.465 --> 08:23.485
as are all banks, and it is not immune from these problems.

08:23.985 --> 08:26.885
But when we talk about the collective duties of company

08:27.405 --> 08:30.885
directors, it can be broadly divided between monitoring

08:30.885 --> 08:34.085
and oversight duties and the sort of leadership functions,

08:34.085 --> 08:35.885
those strategic functions that a board

08:35.885 --> 08:37.045
of directors undertake.

08:37.665 --> 08:40.205
And as lawyers, we tend to focus very much on,

08:40.225 --> 08:44.325
on those oversight, the oversight aspects of the role

08:44.325 --> 08:45.885
of functions of directors.

08:46.625 --> 08:48.565
And the reason for this is

08:48.565 --> 08:52.485
because of well established, agency problems

08:52.485 --> 08:55.205
that you have where there's a separation between ownership

08:55.370 --> 08:58.085
and control in large organizations, large corporations.

08:58.745 --> 09:00.845
And the theory behind it is that the board

09:00.845 --> 09:04.605
of directors must be a check, against,

09:04.745 --> 09:07.685
the incentives that arise on the side of management

09:07.745 --> 09:10.605
to basically use the resources available to them

09:10.605 --> 09:15.245
for self-interest rather than to, you know, see to the,

09:15.245 --> 09:17.325
the most returns to shareholders.

09:18.385 --> 09:21.485
Um, and you know, at the heart of this, again,

09:21.505 --> 09:23.005
is information as symmetry.

09:23.185 --> 09:26.525
The fact that the, the investors shareholders does not have

09:26.525 --> 09:30.885
the same access to information as the, the management

09:30.945 --> 09:33.405
of the corporation, has.

09:34.225 --> 09:37.965
Um, so it seems that all authors

09:37.965 --> 09:41.005
that work in this areas is largely agreed

09:41.155 --> 09:45.165
that the oversight functions over time will be very much

09:45.485 --> 09:47.565
assisted, if not replaced

09:47.945 --> 09:51.445
by appropriate artificial intelligence systems,

09:51.595 --> 09:55.725
because it's the type of functions that really lends itself

09:55.785 --> 09:57.965
to, to this type of technology.

09:58.905 --> 10:03.445
Um, and so theoretically this should open up the time

10:03.715 --> 10:05.165
that boards of directors have,

10:05.195 --> 10:07.445
that they now spend on oversight

10:08.305 --> 10:11.485
to spend more time on these strategic things

10:11.755 --> 10:15.085
that they are also tasked with fulfilling, right?

10:15.665 --> 10:20.245
So, strategic planning, communications,

10:20.825 --> 10:24.045
decision making, all of all of the sort

10:24.045 --> 10:28.285
of softer things perhaps, that is really crucial

10:28.945 --> 10:31.565
for, boards of directors to fulfill.

10:32.705 --> 10:37.625
Um, we, we do agree with patron

10:38.255 --> 10:42.065
that in time you might see a merger between management

10:42.245 --> 10:44.505
and boards of directors, right?

10:44.505 --> 10:47.825
Because of the reduced need really

10:48.005 --> 10:50.745
to have this oversight function of, that,

10:50.745 --> 10:52.265
that we have at, at present.

10:53.085 --> 10:56.465
But where we differ from Petrin very strongly is,

10:56.885 --> 11:00.145
he makes out that the board would be completely replaced

11:00.285 --> 11:01.705
by its artificial intelligence systems.

11:02.005 --> 11:06.265
And basically you'll have, as Orman

11:06.365 --> 11:09.185
and I Mueller refers to as self-driving corporation.

11:09.245 --> 11:11.305
So everything would basically be done

11:11.325 --> 11:12.545
by artificial intelligence.

11:13.125 --> 11:14.345
We don't agree with that.

11:14.605 --> 11:17.705
Um, we feel very strongly that you do need

11:17.705 --> 11:21.145
to retain humans in this loop of governance.

11:21.805 --> 11:26.025
Um, and the other point that Petrin makes is that oversight,

11:26.205 --> 11:27.865
in time could be fulfilled

11:27.865 --> 11:29.545
by shareholders of the company.

11:30.165 --> 11:31.825
And we disagree with this as well.

11:32.045 --> 11:33.505
And there are two reasons for this.

11:33.575 --> 11:37.145
Firstly, retail shareholders will not have the expertise to,

11:37.605 --> 11:39.785
exert oversight, you know, over the money,

11:39.975 --> 11:41.945
over the operations of a corporation.

11:42.525 --> 11:45.305
And institutional shareholders, even though, you know,

11:45.335 --> 11:48.105
theoretically speaking, they could acquire the resources

11:48.125 --> 11:50.625
to do so that would be very costly.

11:50.625 --> 11:54.005
And in the end, those costs will be, born

11:54.065 --> 11:55.285
by the end beneficiaries,

11:55.285 --> 11:56.925
the investors in those institutions,

11:56.925 --> 11:59.285
which then has a societal influence as well.

11:59.985 --> 12:01.365
Um, so it's not ideal

12:01.465 --> 12:04.245
and we can't really see that practically making sense.

12:05.065 --> 12:08.725
So we would, we would retain directors at the helm,

12:09.095 --> 12:11.765
human directors at the helm of an organization.

12:12.425 --> 12:14.845
But we do realize, we do agree

12:15.315 --> 12:19.485
that artificial intelligence systems will augment, and,

12:19.665 --> 12:21.245
and assist in the duties

12:21.435 --> 12:23.325
that directors will fulfill every day.

12:24.345 --> 12:26.765
Um, we also strongly suggest that are,

12:26.765 --> 12:30.725
there are certain realities surrounding the adoption of,

12:30.745 --> 12:32.405
of artificial intelligence systems

12:32.705 --> 12:36.965
inside established corporations where they especially need

12:36.965 --> 12:40.525
to replace existing systems that needs to be brought

12:40.525 --> 12:44.165
to the forefront when we consider the possibility

12:44.225 --> 12:47.525
of artificial intelligence replacing some operations

12:47.585 --> 12:48.845
inside of a corporation.

12:49.625 --> 12:52.685
Um, the first very important thing is to the adoption.

12:52.705 --> 12:56.325
The, the planning, the, development

12:56.345 --> 13:00.445
of these systems take very, technical expertise.

13:01.305 --> 13:05.245
Um, and in most instances, corporations will need to

13:06.265 --> 13:08.645
buy in this expertise from outside, either

13:09.035 --> 13:12.325
through outsourcing or consultancy services.

13:13.105 --> 13:17.285
Um, and because it's so costly, they'll need

13:17.285 --> 13:20.045
to make the strategic case for it.

13:20.665 --> 13:24.045
Um, and they'll need to be really convincing that it's going

13:24.045 --> 13:26.925
to bring a benefit to the corporation,

13:26.925 --> 13:28.845
that's not already present, right?

13:29.305 --> 13:32.045
So that is, that's an obstacle to make out the case.

13:32.675 --> 13:36.645
Then if the proposal is accepted to adopt these systems for,

13:36.865 --> 13:40.125
you know, to replace particular operations,

13:40.315 --> 13:43.725
that it'll have to go hand in hand with detailed planning,

13:44.065 --> 13:46.645
and decision making to support the process.

13:46.945 --> 13:51.605
And that includes being very explicit about the objectives

13:51.905 --> 13:53.045
behind certain systems.

13:53.585 --> 13:57.245
And we postulate that there are many objectives currently

13:57.245 --> 14:00.365
inside corporations that management might not want

14:00.365 --> 14:01.405
to make explicit.

14:01.625 --> 14:03.805
You know, it's based left onset,

14:03.805 --> 14:05.885
but there's sort of a general understanding of

14:05.885 --> 14:08.445
what they actually want to achieve that won't work.

14:08.545 --> 14:10.925
In the case of implementing artificial intelligence,

14:10.925 --> 14:12.805
you have to be, you'll have to be very explicit

14:12.865 --> 14:16.325
and you'll have to reach consensus about these objectives

14:16.325 --> 14:17.725
beforehand in the development.

14:18.505 --> 14:22.005
Um, and that would be a hard task in our views.

14:22.785 --> 14:24.005
Um, then

14:24.095 --> 14:27.245
after implementation, you'll have to consistently monitor

14:27.435 --> 14:30.765
that system to make sure that it works as intended,

14:31.665 --> 14:33.285
and that any human error

14:33.385 --> 14:36.805
or bias that might have been present in development does not

14:36.815 --> 14:39.245
creep into the proper functioning of that system

14:39.955 --> 14:42.885
that will also take money, right and time.

14:43.985 --> 14:48.605
Um, and you also need high quality data for these systems

14:48.785 --> 14:49.805
to develop these systems.

14:49.985 --> 14:51.925
And it might be that in many organizations,

14:52.545 --> 14:55.525
the historical data needed to develop appropriate systems

14:56.465 --> 14:57.565
is just not available.

14:58.935 --> 15:01.285
After all of this, the final accountability

15:01.705 --> 15:05.245
for the proper functioning of the systems will still,

15:05.705 --> 15:07.605
it still needs to vest with someone.

15:07.945 --> 15:12.285
And we say that that needs to be humans, needs to be human

15:12.805 --> 15:15.885
directors, and that basically implies that those human

15:16.405 --> 15:19.885
directors will need a, a particular level of expertise

15:20.705 --> 15:22.445
to fulfill this oversight duty.

15:22.745 --> 15:26.405
So basically, where current oversight is over human actors,

15:27.505 --> 15:29.725
the new oversight will be over

15:30.165 --> 15:31.245
artificial intelligence systems.

15:31.555 --> 15:35.285
They will need to be able to monitor those systems offered

15:35.285 --> 15:38.685
for purpose and actually do what is intended,

15:38.915 --> 15:40.525
what it is intended to achieve.

15:41.535 --> 15:44.315
Um, and this will go, will, will very much,

15:44.585 --> 15:46.875
very strongly tie in with their duties

15:47.095 --> 15:48.595
of care and diligence.

15:48.985 --> 15:51.675
When it comes to consultancy firms, I think it's important

15:51.675 --> 15:54.475
to make the points, some have alleged

15:54.585 --> 15:57.595
that they think artificial intelligence systems will act

15:57.595 --> 16:00.475
without any conflict of interest or self-interest,

16:01.255 --> 16:02.995
but this is of course not true

16:02.995 --> 16:06.195
because you always have human developers, human actors

16:06.575 --> 16:07.755
behind these systems.

16:08.775 --> 16:13.235
And if, as we suspect in most cases consultancy firms will

16:13.235 --> 16:15.155
be used to implement and develop

16:15.295 --> 16:18.235
and continue to, monitor these systems,

16:19.345 --> 16:20.355
they are in these,

16:20.375 --> 16:23.715
in their own rights competitive corporations that you,

16:23.715 --> 16:25.955
you know, need to be competitive

16:26.175 --> 16:30.835
and, gain more, business for themselves.

16:31.575 --> 16:34.275
And implicit in that is self-interest.

16:34.575 --> 16:36.275
So it's not necessarily true

16:36.305 --> 16:39.595
that self-interest could not creep into these systems

16:40.385 --> 16:43.595
because of the underlying motivations of the people

16:43.655 --> 16:45.555
who develop it, right?

16:46.095 --> 16:48.635
So that is also an important point to make.

16:49.375 --> 16:51.435
So what what's going on is at the moment,

16:51.655 --> 16:53.795
boards are looking at the operations of companies

16:53.795 --> 16:54.835
and watching what they're doing

16:55.015 --> 16:57.635
and making sure that that things are being done in a way

16:57.635 --> 16:58.675
that people are accountable.

16:59.135 --> 17:01.355
And now we're going to shift and make that about ai.

17:01.455 --> 17:04.075
So it's not as though really strictly speaking,

17:05.105 --> 17:07.765
the monitoring and observation role goes away entirely.

17:07.905 --> 17:10.085
It doesn't, it just changes its emphasis.

17:10.625 --> 17:14.165
And so, that isn't necessarily an issue

17:14.165 --> 17:16.965
that we've focused on a great deal in all the

17:16.965 --> 17:18.205
discussions that are going on.

17:18.405 --> 17:20.205
Everyone wants to tell you how they'll save money

17:20.305 --> 17:23.605
by streamlining, but there's just a new infrastructure about

17:23.605 --> 17:26.125
to be created that we haven't yet really discussed.

17:26.575 --> 17:28.405
Maybe just to reiterate our main point

17:28.505 --> 17:30.325
and is that, is that we, we don't think

17:30.325 --> 17:32.925
that this task will be completely taken

17:32.955 --> 17:34.365
over by artificial intelligence.

17:34.365 --> 17:37.125
And even if it could, the point is perhaps

17:37.195 --> 17:40.365
that it should not, it should not be taken over

17:40.705 --> 17:42.245
by artificial intelligence systems.

17:42.985 --> 17:44.165
Um, and that is just

17:44.165 --> 17:47.445
because artificial intelligence systems are not good at,

17:48.785 --> 17:50.765
at least at the point, they are right now,

17:51.425 --> 17:55.165
at making value-driven decisions in the way

17:55.165 --> 17:56.405
that human actors are.

17:57.025 --> 18:01.325
And so, you know, recently with the pandemic as well as,

18:01.465 --> 18:04.045
you know, continuing, challenges

18:04.045 --> 18:07.885
that the climate crisis presents, there's a lot

18:08.025 --> 18:11.165
of value driven decisions that need to come from the top

18:11.715 --> 18:15.085
that we just simply do not see artificial intelligence

18:15.085 --> 18:17.365
systems being able to, to do.

18:18.305 --> 18:21.205
Um, that being say, said, we do foresee

18:21.205 --> 18:24.845
that artificial intelligence systems will augment

18:25.625 --> 18:27.925
the capability of boards of directors,

18:28.465 --> 18:31.005
and that you'll find a sort of collaboration

18:31.075 --> 18:32.485
between those systems

18:32.745 --> 18:35.365
and the functions of the board, basically

18:35.885 --> 18:37.925
enabling a more effective governance

18:37.985 --> 18:39.845
of corporations in the long term.

18:40.985 --> 18:42.725
Um, what we don't see

18:42.725 --> 18:47.565
however, is, you know, a completely self-driven corporation.

18:47.865 --> 18:50.885
And definitely we don't advocate for that.

18:51.065 --> 18:54.565
We advocate very much for keeping humans in the loop.

18:55.545 --> 18:59.565
So our main conclusion is that, the ideas of, you know,

18:59.605 --> 19:01.085
self-driving corporation

19:01.265 --> 19:04.285
or a, a fused board where the board of directors is,

19:04.745 --> 19:07.325
you know, completely overtaken by artificial intelligence,

19:07.325 --> 19:09.285
that's very much blue sky at this point.

19:09.425 --> 19:13.005
The reality is realities of, you know,

19:13.175 --> 19:16.565
where we are currently just does not support that type of,

19:17.225 --> 19:19.805
replacement of humans.

19:20.705 --> 19:24.895
Um, and then we, you know,

19:24.895 --> 19:28.695
support this very much by the, the case study of Westpac,

19:28.955 --> 19:30.375
and I'll leave Helen

19:30.375 --> 19:32.535
to illustrate that a little bit further.

19:33.165 --> 19:35.735
Basically, what in essence the Westpac,

19:35.735 --> 19:38.655
case has shown was how poorly managed its technology was,

19:39.395 --> 19:41.015
particularly for managing

19:41.015 --> 19:43.535
and governing it's, money laundering risks.

19:43.955 --> 19:46.055
And this is just one aspect of running a bank.

19:46.115 --> 19:48.935
As I've said, it's very difficult to see how something

19:49.485 --> 19:50.655
that was so small

19:50.875 --> 19:53.655
and was so badly done could ever be considered

19:53.655 --> 19:56.335
as being something that could run a whole of an organization

19:56.335 --> 19:57.775
with all of its complexity.

19:58.315 --> 20:01.055
But I think there are really four key messages that come out

20:01.055 --> 20:02.095
of the Westpac example.

20:02.555 --> 20:04.615
The first is the technology,

20:05.095 --> 20:07.815
whatever form it takes is only ever as good at

20:07.915 --> 20:10.615
and effective as the personnel who design it,

20:11.155 --> 20:12.775
who monitor its implementation

20:13.195 --> 20:14.815
and who evaluated its effectiveness.

20:15.915 --> 20:19.695
Uh, westpac's laws, the system

20:19.695 --> 20:22.295
that they operated under required continuous review,

20:22.825 --> 20:24.975
evaluation and updates the whole time.

20:25.475 --> 20:28.815
So human involvement and expertise was absolutely key,

20:29.115 --> 20:30.655
and it was very inconsistent.

20:31.275 --> 20:33.455
Uh, there was no permanent staff, there was a lack

20:33.455 --> 20:36.615
of corporate knowledge and it played havoc over the years.

20:37.465 --> 20:40.495
Banks are increasingly type technology and data driven.

20:40.515 --> 20:42.375
In fact, what corporation isn't really,

20:42.955 --> 20:45.815
and yet they remain hierarchical in the way they're

20:45.815 --> 20:46.935
appropriately run

20:47.385 --> 20:49.605
and dependent on information, which at this stage,

20:49.605 --> 20:52.245
because there is an adequate reporting goes

20:52.245 --> 20:55.445
between management and boards, I think AI will assist in

20:55.445 --> 20:57.085
that process in the fullness of time.

20:57.585 --> 20:59.965
But it is the lack of escalation of the problems

20:59.985 --> 21:02.565
to management in a way that management understood

21:02.825 --> 21:04.365
and realize to do something.

21:04.785 --> 21:07.965
And from management to the board that gave effect to

21:07.965 --> 21:10.125
what was really a perfect governance storm.

21:10.905 --> 21:13.325
Um, and really finally, the effectiveness

21:13.345 --> 21:16.605
of technology in performing risk oversight,

21:16.995 --> 21:20.445
depends upon core modernization of IT systems

21:21.185 --> 21:22.605
and banks

21:23.185 --> 21:25.885
and many organizations have been very slow

21:25.945 --> 21:28.725
to undertake the kind of work required to be able

21:28.725 --> 21:30.325
to function with such systems.

21:31.065 --> 21:32.885
It would require massive shutdowns

21:33.025 --> 21:35.445
and days out where while this work was done

21:35.785 --> 21:37.405
and no one across the globe appears

21:37.505 --> 21:38.645
to be prepared to do that.

21:39.245 --> 21:39.365
I.

00:00 - This work was published as part of an edited book entitled Technology and Corporate Law, how Innovation Shapes Corporate Activity, and was published in 2021 by Edward Edward Algar. And our focus was specifically on the role and functions of boards in this time,

00:20 - and specifically in the way that risk governance, is being influenced and might be influenced by the acceptance of artificial intelligence systems. another important thing is there were two articles published, more or less this, you know, just before, we started this research one by Martin Petrin, and published in the Colombian business, review, law review, where he postulated that he sees, a fusion between management, and board over the long run so

00:54 - that you don't have the two tier system anymore, right? another, article this time by John Almer and host Mueller postulated the possibility of fully AI driven corporation, so sort of the self-driving corporation. So our intention was really to put a little bit of reality back in the minds of people in this area, and just to emphasize that it's much easier said than done to incorporate these systems in already established running hierarchical organizations,

01:33 - and that there are some practical conservations to keep in mind. and another thing that happened sort of while we were busy with this is there was a very important case in Australia, the Austrac and Westpac decision, which we use as an example to illustrate some of these practical difficulties, that we wanted to highlight. Picking up from where Natan left off,

01:59 - I thought I just explained essentially the basics of the Austrac litigation. Westpac in Australia is one of the large four banks that we have. Probably at that time it was the second largest spy market capitalization. But what in, in essence this is about is money laundering, which is an issue that's, transatlantic Atlantic, and, and, and global. And as a problem, AUSTRAC was the regulator in Australia

02:23 - of anti-money laundering and counter-terrorism financing laws. And it sued Westpac for 23 million alleged breaches of Australia's money laundering laws. And in its particulars of claim, it asserted that the failures or there were failures by simply not making proper reports. But those failures in turn were actually the result of some systemic failures inside Westpac, that they didn't properly control their risk environment.

02:49 - There seemed to be indifference by management as to the problems, and the board had inadequate oversight of what was going on. And in effect, this was the case. Therefore, that went really very much to the nature of the relationship between the technology that was used to operate the money laundering arrangements inside Westpac. that is the way they dealt with that. Secondly, the governance and the risk inside a major corporation

03:11 - and these wallets, Australian-based case, is an issue for every bank across the world at this time. Now, Westpac, it has to be said, filed a defense in which they largely denied all of these allegations, but they did end up settling it after this, article was published and paying a fine of $1.3 billion towards austrac, which is in fact the largest fine ever paid by an Australian corporation for breaching money laundering laws.

03:36 - but the, at the heart of it is an argument about how the technology inside, Westpac was properly managed and itself lends a, a, a conversation to a conversation about whether AI is the solution given the problems that Westpac, experienced. So, having regard to works previously published, we agree that, artificial intelligence systems has every potential to, to as assist in breaching the information asymmetry that exists in in organizations

04:10 - and thereby reducing that problem where a lot of the information that ought to reach the board is really housed in different levels of management inside of the corporation, which is a typical problem for governance, of corporations generally. and so when that happens, we do foresee that there might be, a, a flattening of the organizational structures inside of corporations because you'll need fewer levels of management in between

04:41 - to to lead to, you know, this information being sort of trickled up to, to the board of directors. However, we make, we very strongly make the point that the use of this technology is only as good as the technology actually working. So being fit for purpose as well as the receivers of the information that is the, the human actors, the directors at the top, making adequate use of the information, that they receive from their systems.

05:12 - and again, we can illustrate this, by reference to that AUSTRAC and Westpac case. So taking up that point, really, essentially there were various different platforms that, Westpac was using in order to, get information about transfers overseas and also report what as required to austrac. Now, those systems were regularly being updated, there were constant goals for that work to be done, but the people in charge of looking

05:39 - after that weren't all were, shifting population of staff with the various levels of experience. There was no consistent corporate knowledge. And so what you saw happen in, in a sense, occurred on two levels at Westpac. Firstly, you had a management problem within the very large organization of Westpac. There was, the, the subsidiary in charge of money laundering, was the Westpac institutional bank. And, what was discovered was

06:06 - that when there were problems and reports being, given to management, management, first off didn't take them seriously. And secondly, was rather lax in responding to them, partly one suspects 'cause they didn't fully understand the import or the seriousness of what was going on. They were updates, there were things wrong with the system, there were capability issues. These were very technical words that

06:30 - unless you are very deeply steeped in this area yourself, you don't necessarily appreciate. And, you know, you might say, well, what would you have done if you became aware of some of these issues? And what at the very least you would've expected to have happened was there would've been an investigation and steps taken to remediate the gaps identified in the system and to address its risk management processes. But it wasn't, as I said, just enough to fail to act.

06:54 - It was also a laxness. And it took them 18 months even after the first of these reports started to be received before anyone decided to undertake some kind of review. And you could say, I guess therefore, that the technology worked in the sense that somebody within the organization thought to say it's not working. But it didn't, it failed in the sense that the management didn't appreciate

07:16 - or understand its critical nature. And that would be the same with ai, even if this is perhaps not necessarily strictly speaking ai, it was more a machine learning based tool that they were using. secondly, the management problem caused a second role, which was the failure of the board to do proper oversight. they didn't act urgently, they didn't inform the board. It took another year after all of this happened,

07:40 - before the board was even made aware of it, and they in turn, sat on their response, to that. So we see that information came in some forms, inadequate forms, uninformed forms to those who were making critical decisions, but it failed to actually result in action. And that is gonna be the case. This is just one small component of what Westpac does and one small set of technology that it uses to manage its money laundering program.

08:09 - and how we could see this extrapolate to all AI makes you realize the problems are ahead for all companies. And Westpac in particular is a, a very digitalized business, as are all banks, and it is not immune from these problems. But when we talk about the collective duties of company directors, it can be broadly divided between monitoring and oversight duties and the sort of leadership functions, those strategic functions that a board of directors undertake.

08:37 - And as lawyers, we tend to focus very much on, on those oversight, the oversight aspects of the role of functions of directors. And the reason for this is because of well established, agency problems that you have where there's a separation between ownership and control in large organizations, large corporations. And the theory behind it is that the board of directors must be a check, against, the incentives that arise on the side of management

09:07 - to basically use the resources available to them for self-interest rather than to, you know, see to the, the most returns to shareholders. and you know, at the heart of this, again, is information as symmetry. The fact that the, the investors shareholders does not have the same access to information as the, the management of the corporation, has. so it seems that all authors that work in this areas is largely agreed

09:41 - that the oversight functions over time will be very much assisted, if not replaced by appropriate artificial intelligence systems, because it's the type of functions that really lends itself to, to this type of technology. and so theoretically this should open up the time that boards of directors have, that they now spend on oversight to spend more time on these strategic things that they are also tasked with fulfilling, right?

10:15 - So, strategic planning, communications, decision making, all of all of the sort of softer things perhaps, that is really crucial for, boards of directors to fulfill. we, we do agree with patron that in time you might see a merger between management and boards of directors, right? Because of the reduced need really to have this oversight function of, that, that we have at, at present.

10:53 - But where we differ from Petrin very strongly is, he makes out that the board would be completely replaced by its artificial intelligence systems. And basically you'll have, as Orman and I Mueller refers to as self-driving corporation. So everything would basically be done by artificial intelligence. We don't agree with that. we feel very strongly that you do need to retain humans in this loop of governance.

11:21 - and the other point that Petrin makes is that oversight, in time could be fulfilled by shareholders of the company. And we disagree with this as well. And there are two reasons for this. Firstly, retail shareholders will not have the expertise to, exert oversight, you know, over the money, over the operations of a corporation. And institutional shareholders, even though, you know, theoretically speaking, they could acquire the resources

11:48 - to do so that would be very costly. And in the end, those costs will be, born by the end beneficiaries, the investors in those institutions, which then has a societal influence as well. so it's not ideal and we can't really see that practically making sense. So we would, we would retain directors at the helm, human directors at the helm of an organization. But we do realize, we do agree

12:15 - that artificial intelligence systems will augment, and, and assist in the duties that directors will fulfill every day. we also strongly suggest that are, there are certain realities surrounding the adoption of, of artificial intelligence systems inside established corporations where they especially need to replace existing systems that needs to be brought to the forefront when we consider the possibility of artificial intelligence replacing some operations

12:47 - inside of a corporation. the first very important thing is to the adoption. The, the planning, the, development of these systems take very, technical expertise. and in most instances, corporations will need to buy in this expertise from outside, either through outsourcing or consultancy services. and because it's so costly, they'll need to make the strategic case for it. and they'll need to be really convincing that it's going

13:24 - to bring a benefit to the corporation, that's not already present, right? So that is, that's an obstacle to make out the case. Then if the proposal is accepted to adopt these systems for, you know, to replace particular operations, that it'll have to go hand in hand with detailed planning, and decision making to support the process. And that includes being very explicit about the objectives behind certain systems. And we postulate that there are many objectives currently

13:57 - inside corporations that management might not want to make explicit. You know, it's based left onset, but there's sort of a general understanding of what they actually want to achieve that won't work. In the case of implementing artificial intelligence, you have to be, you'll have to be very explicit and you'll have to reach consensus about these objectives beforehand in the development. and that would be a hard task in our views.

14:22 - then after implementation, you'll have to consistently monitor that system to make sure that it works as intended, and that any human error or bias that might have been present in development does not creep into the proper functioning of that system that will also take money, right and time. and you also need high quality data for these systems to develop these systems. And it might be that in many organizations,

14:52 - the historical data needed to develop appropriate systems is just not available. After all of this, the final accountability for the proper functioning of the systems will still, it still needs to vest with someone. And we say that that needs to be humans, needs to be human directors, and that basically implies that those human directors will need a, a particular level of expertise to fulfill this oversight duty. So basically, where current oversight is over human actors,

15:27 - the new oversight will be over artificial intelligence systems. They will need to be able to monitor those systems offered for purpose and actually do what is intended, what it is intended to achieve. and this will go, will, will very much, very strongly tie in with their duties of care and diligence. When it comes to consultancy firms, I think it's important to make the points, some have alleged

15:54 - that they think artificial intelligence systems will act without any conflict of interest or self-interest, but this is of course not true because you always have human developers, human actors behind these systems. And if, as we suspect in most cases consultancy firms will be used to implement and develop and continue to, monitor these systems, they are in these, in their own rights competitive corporations that you,

16:23 - you know, need to be competitive and, gain more, business for themselves. And implicit in that is self-interest. So it's not necessarily true that self-interest could not creep into these systems because of the underlying motivations of the people who develop it, right? So that is also an important point to make. So what what's going on is at the moment, boards are looking at the operations of companies

16:53 - and watching what they're doing and making sure that that things are being done in a way that people are accountable. And now we're going to shift and make that about ai. So it's not as though really strictly speaking, the monitoring and observation role goes away entirely. It doesn't, it just changes its emphasis. And so, that isn't necessarily an issue that we've focused on a great deal in all the discussions that are going on.

17:18 - Everyone wants to tell you how they'll save money by streamlining, but there's just a new infrastructure about to be created that we haven't yet really discussed. Maybe just to reiterate our main point and is that, is that we, we don't think that this task will be completely taken over by artificial intelligence. And even if it could, the point is perhaps that it should not, it should not be taken over by artificial intelligence systems.

17:42 - and that is just because artificial intelligence systems are not good at, at least at the point, they are right now, at making value-driven decisions in the way that human actors are. And so, you know, recently with the pandemic as well as, you know, continuing, challenges that the climate crisis presents, there's a lot of value driven decisions that need to come from the top that we just simply do not see artificial intelligence

18:15 - systems being able to, to do. that being say, said, we do foresee that artificial intelligence systems will augment the capability of boards of directors, and that you'll find a sort of collaboration between those systems and the functions of the board, basically enabling a more effective governance of corporations in the long term. what we don't see

18:42 - however, is, you know, a completely self-driven corporation. And definitely we don't advocate for that. We advocate very much for keeping humans in the loop. So our main conclusion is that, the ideas of, you know, self-driving corporation or a, a fused board where the board of directors is, you know, completely overtaken by artificial intelligence, that's very much blue sky at this point. The reality is realities of, you know, where we are currently just does not support that type of,

19:17 - replacement of humans. and then we, you know, support this very much by the, the case study of Westpac, and I'll leave Helen to illustrate that a little bit further. Basically, what in essence the Westpac, case has shown was how poorly managed its technology was, particularly for managing and governing it's, money laundering risks. And this is just one aspect of running a bank.

19:46 - As I've said, it's very difficult to see how something that was so small and was so badly done could ever be considered as being something that could run a whole of an organization with all of its complexity. But I think there are really four key messages that come out of the Westpac example. The first is the technology, whatever form it takes is only ever as good at and effective as the personnel who design it,

20:11 - who monitor its implementation and who evaluated its effectiveness. westpac's laws, the system that they operated under required continuous review, evaluation and updates the whole time. So human involvement and expertise was absolutely key, and it was very inconsistent. there was no permanent staff, there was a lack of corporate knowledge and it played havoc over the years. Banks are increasingly type technology and data driven.

20:40 - In fact, what corporation isn't really, and yet they remain hierarchical in the way they're appropriately run and dependent on information, which at this stage, because there is an adequate reporting goes between management and boards, I think AI will assist in that process in the fullness of time. But it is the lack of escalation of the problems to management in a way that management understood and realize to do something.

21:04 - And from management to the board that gave effect to what was really a perfect governance storm. and really finally, the effectiveness of technology in performing risk oversight, depends upon core modernization of IT systems and banks and many organizations have been very slow to undertake the kind of work required to be able to function with such systems. It would require massive shutdowns

21:33 - and days out where while this work was done and no one across the globe appears to be prepared to do that. I.

Faculti AI can make mistakes