The following are the outputs of the captioning taken during an IGF virtual intervention. Although it is largely accurate, in some cases it may be incomplete or inaccurate due to inaudible passages or transcription errors. It is posted as an aid, but should not be treated as an authoritative record.
***
>> We all want to trust.
>> And be trusted.
>> We all despise control.
>> And desire freedom.
>> We are all united.
>> Hello, everyone. I would like to welcome my panelists and audience to this event of Internet governance forum. Our panel was kindly put together by the Polish office of competition and consumer protection and I want to congratulate them for inviting such a brilliant line of speakers. Today representative of Polish and public administration, academia and business will present their views on strengths and challenges of using AI in public institution says. I want to welcome everyone who are joining us here offline in Katowice and everyone who joins us online including two of our speakers. Before we start on a technical note, this is a so‑called silent room, so if you are offline with us, you need to get small ear buds which you will get at the door because otherwise you won't be able to hear what we are talking about here.
You can also ask questions via dedicated chat if you're joining us online. Offline we'll try to make time for some questions from the audience.
Without further ado, I would like to present our special line of panelists today. The president of the office of consumer protection in Poland.
>> Tomasz Chrostny: Good afternoon. A client partner executive at one of the largest IT outsourcing companies in the world.
>> Antoni Rytel: Good afternoon.
>> Daniel Rzasa: Kate Brand. Hello, Kate. I hope you hear us.
>> Kate Brand: Hi, everyone.
>> Daniel Rzasa: And Dr. Gasser. Hello.
>> Dr. Urs Gasser: Delighted to be here.
>> The next level digitalization is producing AI to public institutions. Please feel free to challenge this assessment. I want to start off with asking the head of the office of competition and consumer protection in Poland about the project he and his institution have been preparing and it's an AI powered app to better protect consumers. If you tell us why you decided to use AI in your space.
>> Tomasz Chrostny: In the office of competition and consumer protection we have ‑‑ our main branch and our responsibility is our focus on the competition and customer protection, so we have to bear in mind that we are focusing on the companions as usual, but also on consumer protection which is a huge challenge concerning the fast developing commerce, big tech skills, and the whole economy and the economy that is right now growing in rapid scale and consumer behavior and consumer in daily life. We are absolutely focused on those activities that the consumer has and also the commerce, but we also are aware that as consumers we are not always safe in the net and also infringements that are attached and harm customers and that's why we are trying to develop our skills, our confident and also new tools to act in order to eliminate such infringements.
Right now we are aware probably as all of us that this process that we cannot serve concerning intelligence, concerning the development of big techs. There are possibilities to enter new markets, but also to use the big data concerning customers that give them huge opportunities to develop even faster.
As public administration, we are less effective in such an arena of new technology because traditional, both NCA but also a spokesperson from consumer protection to focus a lot instead of focus on analyzing algorithms and the whole internal space. That's why we decided to enter into new tools that can help us to eliminate infringement. It's also the first step as we also have a clause that's more than 7.5 different closers that are ‑‑ we want to use an AI tool to develop the knowledge about the different algorithms and contract that are in the banking sector, in the different areas of people's day‑to‑day life in order to help us to eliminate infringements also that we can find in the whole commerce not only in agreements that the consumer can come to us and show us the papers and ask us for the help, but also those agreements we can find in the whole net.
We also observe that such tools is the way for us to develop new tools. First, we want to know the whole process. Second, we want to build the knowledge of how AI can operate to eliminate infringement in a consumer protection and competition. Those competences should be in the public administration. So this new tool is probably the way we want to enter into AI and we want to use this tool in the ‑‑ of course, and other tools to, for instance, to also build such tools in public procurement and in analyzing different signals that we have from both consumers and companies in order to be more and more efficient.
>> Daniel Rzasa: Correct me if I'm wrong, this is the 50 time your office uses AI, but I think it might be one of the first times that a public office in Poland will use AI in its work.
>> Tomasz Chrostny: For us it's absolutely the first step. We are connected to law, so it's highly difficult to build such tools and understand such complicated processes that we have. We are right now developing new skills concerning our cybersecurity and concerning our analytical skills to understand those processes, to develop those tools and also to use this knowledge concerning AI in different other processes.
Of course, we also cooperate with our colleagues from different states and we understand that such tools and such knowledge is so complex that right now usually companions that use the big tech ‑‑ from the big tech that use big data and use artificial intelligence can use it in order to influence our behavior. Sometimes it can be against us. That's why we also have to keep up to help customers and to help the competition to work in the right way.
>> Daniel Rzasa: In order to develop the tool you are using on external construct or, I would like to ask you, because your institution organized the whole selection process of choosing an external constructor for this project and I know the whole process was somewhat novel when it comes to Polish landscape. Can you tell us a little bit more about why was it special?
>> Antoni Rytel: 50 of all, of course, the vast majority of the work was done by the colleagues from the office of competition and consumer protection. We're just helping out in our very limited capacity, but this is one of perhaps the things to draw around utilization as a process. We keep talking about solutions. They're obviously very important and policies and regulations, but then in the end we keep on spending the equivalent of half of our national budget every year on various forms of procurement. At the same time, if this was a panel titled redesigning methodologies from the directive, probably, A, we would not be having this panel. B, if we were having this panel, we wouldn't have any of you joining us today. And ‑‑ but in reality, I think this is something that does carry a lot of weight. It's not just about what we need. That's something that can be answered of how special it is. If we do want to engage the private sector, we cannot keep considering them partners which work exactly like we do, we as the public sector.
In fact, this was one of the cornerstones of overall golf tech in Poland was a charge of how to engage those companies in the startups in a manner which is both competitive, and B, friendly for them. And together with hundreds really of different entities, ended up designing a methodology and then ended up being backed up by a digital tool, a platform, which happened to increase the participation rate of startups or in general participation rates from three companies being an average to about 22. The contest you were referring to exactly hit the average of 22 companies applying so we're very glad it ended up working out. The winner ended up being a research institute with loads of experience in AI development. But something to encourage and maybe to discuss a bit more in another forum is to really sort of not forget about the sport of how do we end up choosing our partners, how do we end up choosing who we collaborate with? Do we need to go every step of the way or how much can be done within the framework we have? I'll be ending it here, but one common myth which we kept fighting for ages really is that in order to do anything to change the law, the thing is you don't. There is a lot of leeway being given to tenders, to bidders. All that is needed is for someone to get together, streamline the process, and make it friendly for both parties. And hopefully achieve it and consumers from across the country will end up benefiting from this. There again, I couldn't thank enough our partners at the office for giving it a go. Hopefully it will end up successfully.
>> Daniel Rzasa: We've touched on a very fascinating and very important topic which is how private business engages with public institutions. So I would like to ask you next what's your perspective from that in general and how ‑‑ what do you think about this special approach that the office took when it comes to this project?
>> Grzegorz Lubkowski: Thank you for the voice. I am simply impressed. It was the first time we're attending such a different approach to the procurement process in the public sector. We all get used to the tenders when the majority of the criteria are based on the prize itself. So very often when we compete for such a project, we are technically the tradeoff between the prize and the quality of the solution. In this particular case, the budget was pretty much defined. This is very important in such a project which gets used to such modern and new, brand new technologies like artificial intelligence. When we have a well‑defined I'll say out of the box solution, we can actually put it to the vendors to compete for the prize. But in this particular case, I am impressed that the office of competition and the consumer protection together with golf tech have decided to put the quality of the primary criteria of selecting the vendor for implementing the solution. This is really a change of the game. I wish to have such competition always. It's not always possible. I understand. But in this case where we have artificial intelligence, which is very modern and new stuff, we are talking about the research and development program and it's not something which we buy from the shelf. We have to develop the solution. We have to test it. The result finally is very much dependent on the involvement of the ‑‑ of the end user who have to teach the artificial intelligence how to breathe, analyze, and the result is very much dependent on the quality of training so this is a process itself. The project, we have experience with implementing artificial intelligence for the business where we are continuously developing, improving, and training the artificial intelligence system to be effective as we wish to.
The whole process here is greatly designed. We were from the very beginning understanding what is the goal the office wants to achieve, how they want to achieve it, what is the goal ‑‑ I was reading the requirements. They have done a huge effort to understand what they want to achieve from the artificial intelligence, because artificial intelligence is not a tool. It's a system where the end users are working together with the machines and learn from each other. This is great also from the consumer perspective. Thank you very much.
>> Daniel Rzasa: Now let's move to London. Kate, I would be really keen to hear how ‑‑ what is your institution's approach? Because I know that you use external experts like the Polish office, but you also hire your own data scientists. Could you tell us a little bit more about the benefits of that kind of approach.
>> Kate Brand: Sure. Let me just start off with a little bit more of an introduction of my team and stuff. I'm a director of science within the UK competition and market authority data technology and analytics unit. It's a bit of a mouthful. Basically, this new what we call the data unit is a team of about 35 or so data scientists, data engineers of what we call technology insights, officers, behavioral science and forensic specialists as well. We put the specialists all together in one central team. It's about 2 1/2 years old now. I think it's the largest team of these sort of data specialists in any consumer competition agency in the world.
One of the things we're doing is helping the CMA to be more efficient and effective. We gather and analyzing large data sets and we're starting to do more with them and also providing technology insight which is something that increasingly we're finding the team wants to understand how AI works and how companies use it, that sort of thing. It was set up because the recognition that we're going to be needing these skills and functions to work in these complex digital markets that we're getting a lot more cases in that area.
So I oversee data scientists who are building tool says for the CMA but also a program of work as well sort of on the other side I suppose where we're trying to understand how businesses use algorithms and how that might harm consumers and lessen competition. And a very little bit of background to me, I am a data scientist by training, but before joining the CMA a couple years ago I also built and led science teams in central government in the UK and that concluded of deployment machine models and that sort of thing.
So coming back to your question, our approach is actually to do most things in house. I found, and this is not just my experience from the CMA. This is sort of a wider thing. I find that if you're bringing in external expertise, it's actually preferrable to do it as bringing it in as a resource I guess where you've still got more control of it rather than commissioning out a piece of work. I think the main reason for that is a lot of what you need to do in AI machine learning and what you need to understand is about the context actually and understanding the business and sending where it's needed and why it's important. And then the other big thing that always seems to come up with about the date that that's knowing the organization's data and particularly actually the issues with the data that are not always that apparent, so people internally can understand that better. And also, of course, if you build the tools internally, you've got the people who can understand that. They can integrate with the existing tools we've got and provide ongoing advice. And then can also kind of tweak things if you need them internally to change those tools as they kind of ‑‑ as they're required, I suppose. So our general approach has been to do things more in house. I know that's not something that's available to every agency, but it's something that we've put a lot of effort in, a lot of resource to putting in.
>> Daniel Rzasa: Thank you very much, Kate. There was some really interesting remarks. Urs, you are one of the world renowned when it comes to artificial intelligence. You advise governments. I would like you to share your thoughts on early lessons learned by early users of AI by governments around the world.
>> Dr. Urs Gasser: Thank you very much. Your introduction is way too kind. I'm learning about this new technology with others as well. I've been really part of a larger network of researchers tracking over the past couple of years what are some of these early users of AI‑based technologies in the public sector. So most of the evidence so far is anecdotal and case study base the. I'm happy to share three quick thoughts or observations from that work. And maybe taking two steps back and fits quite nicely into the more concrete conversations we had about the limitation in the consumer protection space. The first observation is really about the value of strategy. I think even three years ago the AI used in governments was mostly focused on pilots. We heard in Poland this is kind of the first in some ways using this specific application in the government context.
In parallel, however, we've seen that several national governments but also increasingly at the city level have come up with comprehensive and systematic AI strategies and that's something that's valuable for a number of reasons. First and foremost, I think we've seen that such systematic approaches to the question of AI deployment for some sort of a conversation and reflection why AI‑based technology should be incorporated in the first place. I think the panel made the case why in the consumer space there might be a lot of value to such a tool.
On the other side, I think there have been also early pilots where we've seen AI being adopted as something new and flashy where it was not entirely clear what the actual value is, especially from a citizen or demand side, and I think as strategies are a good way to have this conversation. Strategies as opposed to pilot are important because they invite for a systematic approach to mapping the different opportunities to track such experiences over time and those are built towards a common repository of best practices and lessons learned. And I would say they also provide an important opportunity to include citizens in all of these treaty and interesting questions and create really inclusive processes whether it's the procurement part you talked about or later the accountability and transparency piece when technology is adopted. So first observation is really we see a shift from pilot to strategy and I think there is a lot of value to learn from the pilots and build towards strategy.
The second point is I feel and we've heard it already the biggest challenge is probably at the implementation level, so even if you get the strategy part right, I think governments are struggling with a set of implementation challenges. Some of them are structural. At the technical level, Kate and others have pointed out already we may have a challenge to find high quality data sets. They are at times unequally distributed. Interoperability is a challenge. It's one thing for a government to launch a new AI tool but then to maintain it over time may be much harder in terms of financial resources and the like.
It's not only at the technical level. I think more importantly at the human level there are massive implementation challenges. Human level, we've already touched on it when talking about doing things in house or externally. I'm worried about how can we build skills and competency among civil servants and make sure the people working in government have the knowledge that was already mentioned during the panel. How to attract talent in a very competitive market I think is very important questions.
Just this last sort of example of implementation challenges, it's awfully hard to translate some of the ethics best practices into a concrete application given that things are so contextual as I think Kate and others have pointed out. What does transparency really mean in a meaningful way in a given sector? What does accountability mean? Lots of hard implementation questions. Yes strategy matters, but even more implementation.
The third point is about the law of unintended consequences. I think AI in the public sector in these early stages is already rich of examples where governments have done a good job. Government officials have worked carefully to implement AI but surprise, surprise, many things have happened that were not intentional or at times were even harmful. I think one challenge there is at times we may be overly focused on technology and not understand that an intervention that ‑‑ AI intervention may actually upset at the organization or human level the ecosystem in which an organization operates. And we've seen examples where technology worked good enough, but good enough was not good enough because you had downstream consequences. For instance, additional burden was put on citizens or beneficiaries of technology.
The very final footnote to this problem I think is it becomes increasingly challenging to continuously evaluate AI‑based interventions. I think contrary to earlier manifestations of government, the whole point about AI is it leads to more automation. One problem it seems is how can the real time monitor what the technology that we introduce is doing in the wild when it's no longer a cultural environment? How can we intervene at the right moment? If I sea we, I mean from a public interest perspective, from a government perspective. So the question of design, continuous recalibration, corrections becomes a really important layer of infrastructure I think governments have to build in addition to all the things that have been mentioned. So I stop here. But thanks for giving me the opportunity to share a few thoughts from that work that's rather collaborative and global. Thank you.
>> Daniel Rzasa: Thank you very much. There were some really brilliant remarks here. I'd like to focus now on one common approach. AI tools are only as good as our databases. I would like to ask Tomasz, how and where do you get your data? On one hand I can imagine public institutions have much more data to analyze than private ones. On the other hand, I'm guessing that the law around it makes it much harder to actually use and practice or am I wrong?
>> Tomasz Chrostny: It is highly difficult and we also learn how to use this knowledge that we have in different registers. In public registers, of course. And the issue connected to pilots and the strategy, I was thinking about the banking sector which we also focus on. 20 years ago the building was the thing was the thing that we were thinking about banking when someone would ask billion. 10 years ago it was ATM and now it is smartphones. The business is developing highly fast. And still it has access to best resources, human resources that also give us possibilities to understand what's on the algorithm and how can we use it to analyze the situation, how can we use it. Of course, understanding the processes and using the data is absolutely crucial. But still, as we can compare the situation, for instance, to the public sector is absolutely behind the business. And right now I think that we, of course, have to build and we have to get the strategy. But we have to firstly learn how to use the tools that business has. The case of Kate and the CMA is a great example. To have access to human resources that understand the practices, know the data, understand the algorithm and can be inside rather than outside and make it work for the whole society is something crucial. But still we are thinking ‑‑ we are talking about human resources that right now are providing the main results that also businesses are trying to get. So trying to engage in its own processes and usually the goal of public administration. For instance, the CMA and the goals of the business are the same goals. Usually, we are thinking about how to eliminate business practices and they're thinking about how to get more money from businesses and it can touch their interest of the consumers and the society and we should take actions to eliminate such possibilities. Of course, we have to learn how to use the data we already have access for.
Secondly, we have to get access to some new data that the business has. We have also this knowledge and we see that right now it is Europe. People have more ‑‑ are more open to leave their data to the business rather than to a particular administration. They are afraid of particular administrations somehow.
>> Daniel Rzasa: It's an issue of trust.
>> Tomasz Chrostny: An issue of trust, of course, and showing how this knowledge and this data for public administration can work for the society. So we have different ways that we have to make a trial on, so we have to build trust. We have to build competence. We have to learn how to use these tools that the business from the last decade is operating from, to monetize it in order to secure the society from illegal practices. It's something that right now we are even more than a decade behind. So we have to get extra for it to get to the business and understand what it's operating on.
>> Daniel Rzasa: Thank you. So Tomasz, you made some excellent points, but I would like to go to Kate and he said that you're in an excellent situation that you are able to have this amazing team of data scientists, developers and so on, but I know from our chats from before the panel that you don't think that public institutions should be cutting edge when it comes to AI. They should know what the private sector is doing but not compete with them. Is that correct? Could you elaborate.
>> Kate Brand: It would be great to be cutting edge. We don't have thousands of data scientists that we can pay lots of money. I don't think that it's all about being cutting edge actually. A lot of this is not always, you know, machine learning, algorithms, things like that. They don't need to be that complex to do really exciting things. Actually, I go back to this idea that it's about understanding a problem. It's about knowing where you would apply this sort of stuff and being able to do it. And also I suppose going to the point of we're in competition with them, how are we ever going to get people. Actually, at least in the UK, it feels to me like it's changing a bit. We are not finding it as difficult to find people at least that are new to data scientists, new graduates. There's a lot more courses now, a lot more people doing that. What I think we have trouble with is getting people with more experience that kind of understand more about our subject matter competition and agencies and what they need. And it is difficult to get people that are ‑‑ we actually tend to get people from central government because in the UK we have more data scientists moving around.
I have certainly found there are a lot of people who do want ‑‑ they want to work for good, so they want to work more to kind of, you know, not just to make people a lot of money but to actually try and actually make things better. So the combination of those things means we can actually get more people in to do these things. You do often have to train them up a little bit more in the specific areas that you need them.
Probably one of the areas we're finding more difficult is engineers and I think that's because they're not as close to the subject matter, so I think actually with data engineers and platform engineers it's actually harder to compete.
>> Daniel Rzasa: Thank you. So since your company is one of the biggest IT companies in the world and you hoarded all the data scientists ‑‑
>> Grzegorz Lubkowski: Just a few of them.
>> Could you tell us a little bit more about what do you find challenging in cooperation with public institutions beyond the procurement level? Or is it not challenging at all? Is it like working for any other private company?
>> Grzegorz Lubkowski: You're reading my thoughts. Actually, I wanted to say it's not that much challenging. The solution which we are promoting on the market, artificial intelligence, we have originally looked for the legal companies to be able to analyze the lease contracts. It was just a matter of let's say a particular need. We then are promoting to the banking sector. The banking sector sees artificial intelligence as a tool to reduce the cost, to increase efficiency and to provide a better service to the customers. More or less, it's about the money. Here what I see actually and I find it really perfect that the consumer ‑‑ the office of consumer protection is actually seeing this as a kind of admission to provide the service to the consumers, to the office itself to be more efficient in their job, to identify the closers in there, in the contract agreements for the consumers, and we can benefit for sure as consumers.
But second, the next set might be that the consumers can have access to such a solution. For example, I would be more than happy to see the possibility to upload my contract which I have been given by the service provider and check with the artificial intelligence for the statements which are either abusive or not in favor to me. Kind of using the artificial intelligence which is already possible to help me to identify let's say if this is beneficial for me or if it's beneficial for the other party.
To answer your question, I don't find actually any challenge in the public sector as long as the process is such clear like it was in this case.
>> Daniel Rzasa: This was kind of your institution's goal to make the process more streamlined, but after what Tomasz and Cate said about the issue of want being able to get the best of the best from the market, I guess golf tech Poland and similar institutions elsewhere, one of their goals is also to attract startup companies. Not like the largest tech Giants in the world for the process that which are more competitive and the price is not the only factor. Correct me if I'm wrong, but you've existed for two years I think already in Poland?
>> Antoni Rytel: In various forms we have been around since late '18.
>> You're been modeled on a similar institution from elsewhere.
>> Antoni Rytel: Yes, some of the other solutions. Actually, one of the first tools, sometimes they're really very simple. One of the first tools which we ever launched is ‑‑ was a presence at ‑‑ he mentioned this is something which we are looking ‑‑ we are lagging behind in terms of maybe not competing, but measuring up to business. And one of the things we should do, therefore, is to see which path and think about using these same tools which they find effective for themselves. And in fact, this will now be the fourth edition coming up this Saturday, so I would like to invite anyone who wishes to join us over here in Katowice. Moving on from this, there's no really single tool. In the end, we can talk about procurement, legal solutions, in house or contracting or tenders or auctions, but in the end the actual bit of our mission which we value the most is to answer the actual need of the institution which approaches us. We're in a way sort of an in‑house consulting when it comes to technology and we found different cases being ‑‑ requiring different solutions. So if we need to hire people, plain and simple, there are hack a thons. If you need something for your needs, it's something which cannot be done by any state institution, then we have design contests which we've been referring to quite some time. Or legislative changes which allow for the entirety of the public sector to work together as one team, so one institution having the skills. The other one having the need. And we remove the legal barriers for them to collaborate with themselves and dedicate to this. Some institutions push you in the right direction. When it comes to consulting or those sort of soft tactics which help them get on the right path and then they carry on. One thing we've learned from expertise, it's not about tools or processes. It's more about public institutions a partner which they can trust which we'll not be charging them in a second. Which will not be even more difficult for them to collaborate with than the actual creator of the solution, sometimes help them a bit more, sometimes deliver the actual solution itself. We've done this before as well. But in the end what matters is that there's someone they can talk to. There's someone that can take a look at their needs. And build this relationship of trust while being conscious of what's done in the market and not suggesting things that would go outside the scope of what would be considered good market practice.
>> Thank you very much. You've made a great introduction to the next subject I would like to talk upon. We've touched on public institutions, working with private sector, different models of institutions such as Poland, having a team of data scientists in house. I was wondering if we could talk about where do ‑‑ I was wondering if you could tell us more about it and what's the goal.
>> Dr. Urs Gasser: Thank you so much. Of course, academia has multiple roles to play starting from research to education, but importantly I will argue also translation of findings from research into practice. I think the approach that we're about to launch, the global tech policy practice, is situated to address a specific challenge that has come up in this discussion already multiple times, and that is how can we build global capacity where governments and government agencies and the humans working there are trying to do the right thing and trying to embrace AI‑based technologies and live up to the best practices and, you know, get it right but may not have the necessary skills or knowledge or only limited capacity particularly if we think about the world countries, sometimes called the global south, where it's not the UK or it's not Germany and the government officials may face very different constraints when going through this process of identifying problems and using AI to solve these problems. And so the hope is through a network approach bringing together different research institutions around the world, building up on the backbone of something that's called the network of Internet and society centers that we can offer a capacity from academia working on the real world implementation issues together with government officials and government agencies at different levels with some sort of two key distinctions from a consulting firm. The first one is that universities unlike companies have a dedication to the public interest in the DNA. Universities are one of the few institutions left where the public interest is some sort of entity DNA of the institution itself. And that makes it very different when working on implementation issues as opposed to a traditional consulting firm that's hired for a fee.
The second key of this global tech policy practice is that the problem of solving the implementation issue, the use case from practice would be addressed in an educational setting.
We have piloted it already in several use cases that specific implementation issue, we bring together both domain experts. Of course, also from practice and working with our friends and companies, but most importantly from the academic network and pair those experts with actual students, whether it's in the context of a regular course, whether it's as part of an experiment education. We have different models we can activate to facilitate this sort of problem‑based learning. You can see if something like that scales that we can actually contribute to building global capacity by educating the future civil servants and leaders hands on in academia, but then prepare them and equip them with the skills that they'll use later on. We are excited about this perspective. We've piloted it. It's a big task, but I think it's worth the challenge to add capacity again with a deep commitment to the public interest. Thank you.
>> Daniel Rzasa: Just if I understand correctly, you will invite other academics to work in that project or public institutions as well? Or will it be a sort of academic network that will then share their knowledge with public institutions around the world?
>> Dr. Urs Gasser: It is an academic network. Think of it if you're looking for an analogy in medical schools, you have teaching hospitals with real patients, right? You have students enrolled in learning and problem solving. And so it's roughly an analogy where teaching institutions, research institutions are teaming.
To create the capacity to do real world problem solving but at the same time do so to share their experiences, to translate knowledge from theory into practice and above all also to create the living repository of such experiences that will then make available as open education resources to other governments, to other entities that may be confronted with similar questions when adopting an AI tool in education, in health, in a specific context. And working on transparency or accountability issues or how a participatory process looks like. You see that these experiences working on cases will accumulate over time and create a robust ‑‑ some sort of set of materials and case studies and blueprints and templates to build up on.
>> Daniel Rzasa: Thank you very much.
That sounds like a brilliant idea. I want to get back to Tomasz and your project, because on Friday I think, three days ago we've learned that the final external contractor you are going to use when it comes to developing your project is a part of the Poland state‑owned national research institute. They are academics really.
>> Tomasz Chrostny: I believe they can support our actions in a long‑term perspective.
>> Daniel Rzasa: Do they have sort of a commercial goal as well or are they nonprofit organization?
>> Tomasz Chrostny: It's probably the most important they have ‑‑ that's why also the way that they are operating is that we are ‑‑ we can learn and we can check how it will look like during the whole process and probably the biggest value we can get, it's not waiting on the shelf and we have to prepare something that will be customized and will help us to be more and more effective as we observe how this processing of the E‑commerce, and how it's a different type of tools in the smartphones. We observe that right now those practices that we should eliminate are absolutely different area. And E‑commerce is developing absolutely totally fast. And the region in the area ‑‑ in the pandemic, there were 12 million new users of E‑commerce. So we are more and more likely to use these new tools and also we as the public sector and at the office, we have to get new tools to eliminate those legal practices. To eliminate those closes in the agreement in different areas that can touch and harm consumers interests. To be efficient, we cannot enter new tools. Without them, we won't be efficient. We won't play their role in this public sector that people trust. We will have to secure their interests.
>> Daniel Rzasa: Thank you very much. Kate, I would like to ask you now as a director of an organization which has its own big team of data scientists and have already used AI for quite some time, I would like to ask you to give some advice to some of the other institutions such as the Polish regulator. What should they avoid and how should they approach the whole process of, you know, developing their first AI‑based tool and working with external contractors and how to kind of avoid some mistakes?
>> Kate Brand: That's a big question. My advice would be not to think ‑‑ I think a lot of people when they talk about AI, it's a magic separate thing. Actually, I always tell people to go back to where the problem is. AI can answer particular types of things but not everything and it's better for some than others. It's good when you've got large amounts of data that you can't get humans to process all of it, for example. It's good for certain things. I suppose there is also a question of the build versus buy as well that always comes up which I think, again, I would think about how generic the problem is versus how specific it is and whether you can kind of verbal take a hybrid approach. I've seen quite a lot of examples where organizations will try and get data scientists to reinvent the wheel for something that's been done very well elsewhere and you've these tools can be used, but at the same time there's some things that are very specific. I would also say that although they are quite ‑‑ internationally in the similar types of organizations, we've got the same types of problems, sometimes the same problems as well. So that's something that I've been trying to push that we can actually share tools internationally. When we have built them, we can share them and share that knowledge.
>> Daniel Rzasa: So are you kind of in touch with any other competition market authorities worldwide and share your experience?
>> Kate Brand: Yes. We've got a few different. I spend a lot of my time actually talking to other particularly smaller competition and consumer agencies to try and help them build this type of capacity. It's something that a lot of countries are interested in that don't quite know thou get started. So yeah, just talking through how they can do that.
>> Daniel Rzasa: Thank you. You're saying smaller. I'm guessing that's UK being the financial center of the world, there's not many larger organizations than yours.
>> Kate Brand: I think it depends. We still have different sized competition agencies even if countries might be bigger or smaller, yes.
>> Great. Perfect. I think our time is almost up. I would like to thank you all for joining us offline here and online. I want to thank my guests, Kate Brand, at competition market authority. Dr. Gasser, Tomasz Chrostny, and Antoni and executive partner from IBM global technical services. Thank you all very much. Thank you, Kate and Urs.