Welcome to the United Nations | Department of Economic and Social Affairs

The following are the outputs of the real-time captioning taken during the Tenth Annual Meeting of the Internet Governance Forum (IGF) in João Pessoa, Brazil, from 10 to 13 November 2015. Although it is largely accurate, in some cases it may be incomplete or inaccurate due to inaudible passages or transcription errors. It is posted as an aid to understanding the proceedings at the event, but should not be treated as an authoritative record. 

 

***

 

>> MODERATOR:  Dear colleagues.  Hello, everyone.  Welcome to the Workshop Number 58, Open Educational Resources, which also includes the digital divide problem which we are going to discuss here in this session.

     My name is Mikhail Komarov, Deputy Head for International School of Business Informatic, University Higher School of Economics from Moscow, Russia.  We have several panelists being late.  Hopefully they will join us in the discussion.  Just wanted to show, you know, a slide about organizers and panelists whom we have here.  In terms of organizers, on site we have myself and my colleague Bonface Witaba from Center for Youth Empowerment Leadership.  In terms of participants, we have Olga Cavalli from Argentina, Chapter President.  She is also Advisory Minister of Foreign Affairs for Argentina.  We have Andrey Shcherbovich from the University House School of Economics from Russia, Moscow, and Remote Panelists from Uganda Christian University; Professor Xiaofeng Tau from Beijing University of Telecommunications and Posts, Deputy Director in China and Chief Architect for GTTD Working Group.  Patrick Ryan is here from Google.  And we have some, let's say, more panelists who are coming to join our discussion today.

     So we are going to follow the schedule with small short introduction.  There is a problem of quality of online content, online education, problem of digital divide and after that I will show questions which we are going to discuss.  We have of course more questions to discuss but we are limited in time.  I would also like to ask on site participants here to participate in our discussion as we have a round table, we have a panel which is not limited to just panelists.

     So okay.  In terms of empowerment for quality online control, today there is a fast increasing number of massive open online courses, open data, educational resources and the process is getting faster with emerging technologies.

     More and more educational programs depend on Internet resources and online educational services now are major approaches for empowering people.  We have different services already introduced for educational resources for educational plaintiffs and just in general for education.  We have access.  So we have access any way to our large unregulated body of information which exists on the Internet.  But this excess suggests that we need quality control and critical evaluation of related Internet resources because information there might be unscientific when we are talking about education, right?  There should be some basic principles of quality control, of different services, especially when we are talking about educational content. 

     The thing is, this year for the first time when UNESCO had a post conference, one of the major areas, relevant teaching and learning.  It is their priority area starting from this year.  We know different programs are running under the UNESCO umbrella, education for all, a right.  Before they were focusing on access, giving everyone to everyone, information and education.  This year they also started focusing on quality of content which is accessible to everyone here.

     I also would like to emphasize that education is a big area for different companies and organisations.  So we know examples of massive online courses, platforms, right?  From their perspective, because we had let's say discussion with them before the workshop in order to get their position about quality of online content, how they are going to ensure quality of online content, how they are going to assure that these courses they are providing on their platforms are bringing some scientific data or bringing knowledge to participants who are going to apply for these courses.  They said they rely mostly on Universities because they collaborate with Universities.

     Nowadays, right?  But we know that education is not just about Universities.  When we are talking about online access, right?  But it is also about ordinary people sharing their information, sharing their videos right, through the use tube service, through other services, sharing it for everyone.  That is why now we, they don't have a clear position about quality or control.  So they move it to the Universities level.

     And another important point there that, for instance, educational market.  Just to remind you all educational market last year was measured approximately from 60 to 70 billion U.S. dollars and it is growing.  It is growing by 20 percent per year.  Sometimes even more.

     Some statistics from Russian representatives of academia, I would say that Russian online education market is just 5 million U.S. dollars which was measured last year.  But what I would like to say is that we have also other markets and different transformation.  So we have incorporated the educational sector which is actively using online educational resources, right?  And for instance, in terms of in Russian, in -- in Russia in terms of the market, it's $2 billion in the corporate structure.  We have private teachers started using online resources.  We are talking about this market, this content generators, let's say, right, for educational use.  We should somehow ensure that the quality of content they are providing and someone is going to use either up to some standards, right?

     I think from my side in terms of introduction to point of problem with the educational online content, since I would like to give the floor to Bonface who is the co-organizer of the problem with OERs:  Can it bridge the digital divide gap?  After that we will move to questions, please.

     >> BONFACE WITABA:  Thank you, Dr. Komarov.  My name is Bonface Witaba from Kenya, Center for Youth Leadership.  To dive into the topic right away, we all know that the Internet started as a government project in the late 1960s.  By then, back then it was meant for military and research purposes.  But it opened to the world in the late '80s, brought tremendous opportunities to the globe as a whole.

     Basically, today it is difficult to point out an area that is not affected by the Internet.  In the field of education, the increased demand on education and training is that traditional methods will not suffice to take Developing Countries beyond universal primary education into the open education for all agenda.  So what do we do in order to bridge this gap?  So UNESCO told us, OERs are an avenue to foster development as well as improve learning in schools.  So to the next speaker, probably we should be able to cover how do we use OERs?  How do you develop the use of OERs to foster development?  Thank you.

     >> MODERATOR:  Thank you very much, Bonface.  As you already started touching on some questions I also would like to give the floor to Andrey Shcherbovich and other panelists on this question you mentioned which is included for all discussions but also to some of the questions for discussion itself.  So please, Andrey, could you please explain some legal aspects and legal problems we have in terms of quality of content?  Thank you.

     >> ANDREY SHCHERBOVICH:  Thank you, Mikhail.  I would like to present some issues which are most important in the United Nations aspect maybe in all social cultural aspects that is problems of nonprofit institutions in online education, especially in the field of human rights.  This is one of the most issues which is submitted by the problems raised by the General Assembly of the United Nations, which proclaimed the Universal Declaration of Human Rights and the achievement of the goals of the Universal Declaration on Education is one of the points, one of the main things to achieve, the goals of the Universal Declaration of Human Rights.

     As we know, some other international documents confirming this statements made by Universal Declaration of Human Rights, one of the most important from them are Convention of the Rights of the Child, Article 29.  Also we have a Declaration and program of Action of 1993 which confirmed rights of education not only for the children but also for other people.  About distance and Internet education, there is an issue and the key program, UNESCO, education for all and they are international intergovernmental programs which support distance education and education through the tools of Internet.

     Besides the United Nations institutions there are also very important role belongs to organisations which are nonprofit based, but those nonprofit organisations are addressing human rights.  One of the examples are courses on freedom of expression, freedom made by Amnesty International.  This is one of the first human rights open courses which is accessible for everyone on different languages.  And they position themselves as a global leader on online education and human rights.  Trying to delivery massive open online courses on different areas of human rights courses.

     This Amnesty International with this course is dealing in cooperation with some educational institutions like Harvard University and others on their Web site, positioning.

     Some other nonprofit organisations are human rights education associates which are based in the United States.  And the University of Peace, also providing courses related to the peace and human rights problems.

     The next one is based in Russia on nonprofit organization Sakharov Center, named by Andrei Sakharov which also makes online distance course on human rights in the sphere of online education, not only in the human rights field but primarily the human rights field.  There one of the basic things is authority of this kind of international or national nonprofit organization.  And this authority could bring very high quality online educational content.

     Also if there is nonprofit projects on the measure of this kind, there could be kind of international programs which could be performed by very high quality professionals who are not receiving any honors for this, but they could be very good in the sphere of maybe voluntary educational programs in cooperation with nonprofit organisations.

     In the last of my presentation, I will return to the United Nations system, one of the most important initiatives in education, the United Nations Cyber School Bus which created even in 1996 and this is one of the most important educational platforms which is based on the United Nations itself and this is kind of an explanation not only for school children but also for students which are explaining the core of the United Nations system, its values and also human rights problem.

     Finally I would like to answer on the question about the digital divide gap.  And I think that this kind of education, the education in the sphere of human rights, I think it is simply dividing, simply could overcome the digital divide gap because this kind of education is more important to developed societies which could also make contributions for our common digital divide because I think in the sphere of culture, in the sphere of education, this kind of online educational systems could be simply measuring instruments for overcoming the digital divide through the cultural educational process.

     Thank you very much.

     >> MODERATOR:  Thank you very much, Andrey.  Yes, so quality depends on authorities providing con ten.  Yes, I think it is important to fix this statement.  So what about organisations providing services?  Different services for educational resources or for generating some open content like YouTube or others?  Patrick, do you have some comments about it?

     >> PATRICK RYAN:  Sure, I'm sure I have lots of comments.  Can I make sure I understand the question?  Could you rephrase the question for me, please?

     >> MODERATOR:  So what about organisations providing services?  Right?  Internet services for either educational services, right?  Within this Google Docs or YouTube or some other let's say services available online for users and which helps them to generate content and after what is used for educational purposes, for instance.  So if these companies should ensure quality control or if they use already some mechanisms or algorithms to identify what is wrong there inside or something like that.  What do you think about that?

     >> PATRICK RYAN:  Well, first of all I want to thank you for the invitation to come here.  This is the second or third time I participated in this event and you've continued to continue on this theme and really press hard on these difficult issues about online education.  I know this is something that is very important to the citizens and students where you come from and very important to many of the participants here.  Using educational resources online is just one of the most fundamental good things that can happen on the Internet because we can all learn.  There are a lot of other tools available that are increasingly available for free to be able to do that.  The question is, is YouTube would be of those tools?  If I understand the question, how should this content be controlled, regulated, reviewed, if at all?

     And this is a subject of considerable debate around the word.  There are many cultural views on this topic.  The platform that we have at Google is really founded on a principle of view-generated content and enabling users to put just about anything online so it is there forever, freely accessible as they may wish.  They may wish to have it be deployed to the world or be available to the world as a public link.  They may be backing up their personal library.  We want to give users that control but also want to give them the opportunity to you put all of that information online and enable a whole new form of journalism, a form of communication and ways to share and retain information that has never really existed before at this scale.

     In order for that, that's a long explanation in order to say that in order for that kind of ecosystem to thrive, there needs to be a strong appreciation for the community-based controls and for just the good sense of users in many different ways.  Now, that's complemented by rules in the community that do have take-downs and can lead to removal of content.  That is not because content is not of a particular quality, right?  The types of content that we would remove would be content that would be either including nudity and other things that would vary from the principles of YouTube's platform which is to really be a family-friendly platform.  There is variation in what family friendly means and we rely on the community to rank those kinds of things.

     Whether it is good or bad, that is a different metric.  That has to do in our view with a very simple metric of clicks and views.  Are people using it?  Watching it?  In many ways that's the way the company is founded, Google itself, the search engine Google is based on the premise that the information that most people are looking at is probably going to be more like the information that you want.  There is a few options of similar choices but for the most part all Google's algorithm does is look at what people are doing on the Web.  They are doing things similar to you and serving it up and saying:  This might be interesting to you.  In the case of content, it's simple.  Whether it's an article or video you find online through YouTube or another service, you are likely to find more relevant content if other people find it relevant.  That's approval of the crowd.  It certainly has a context that makes some societies very uncomfortable because we have been used to having authorities and people charged with the responsibility for making sure that the content and information is really appropriate for the citizenry.  In many cases we are used to having other authorities do that.  Not in every case, but a lot of cases we are.  Seeing these situations really conflict with cultural values is something new.

     That is a long way of saying that we don't really believe there is a place for us to be doing content control other than putting the mechanism in place to let the community do that.

     >> MODERATOR:  Thank you very much, Patrick.  So quite interesting, I would say, approach from private sector.  But here we have Olga presenting also the governmental sector.  For the government, yes, we have different standards of educational content, for instance.  But what about online content from governmental perspective?  Probably some example from your side.

     >> OLGA CAVALLI:  Can you hear me?  Hello.  My name is Olga Cavalli.  I want to thank Mikhail for inviting me and apologies for not participating in years before.  It overlapped with other activities and honestly, I find this issue of content of extremely relevance, especially for Developing Countries.

     Especially for countries like Argentina where I live or for Latin America, which is a region with such diverse variety of socioeconomic groups, of huge geography, big countries with very highly populated areas and very scarcely populated areas.  And this is extremely challenging for developing infrastructure and for developing content to access those people that want access to education.

     What I would like to say first as a general comment -- I'm always an optimistic person -- rather than saying no to some of these initiatives I would rather go to, okay, let's analyze what is it?  And which context?  And what we are talking about.

     I have a similar feeling about generating content.  I was in a session yesterday and some of the remarks made that were kind of open to analyzing and putting into context, those are the ones that I find more constructive because sometimes, and let me think about someone living in a rural area in a country without access to other technology than a mobile phone or a tablet, that could be provided by the country, by the government, and having access to something like Wikipedia.  That could change their lives.  When I was a child at home, I had two TV channels in black and white.  Yes, I'm an old lady and I used to watch star trek.  That changed my life forever because I wanted to build a star ship.  I wanted to be an engineer and I became an engineer.

     Somehow, that TV show that of course was a fantasy, but it is not such a fantasy, it became part of my daily life and my career election.  So this content can change the life of many people.  This is why I want to stress the value for Developing Countries.

     I also would like to share with you, I am a University teacher also.  I work as Mikhail said as advisor to the government of Argentina.  I used to be a teacher in the diplomacy career because I am advisor to the Minister of Foreign Affairs.  I said what can we do to make these people to be, they are mainly lawyers and political science specialists.  Very few are technology oriented.  I used to teach technology to them, introduction to technology.

     So I said:  How can I make them get interested in this?  So what they had to do was to prepare content in Spanish in Wikipedia of those issues we reviewed in our classes.  It was extremely challenging for them because they were not technically oriented or technically skilled.  But the experience was fantastic.

     Also what we wanted to do was to enhance content in Spanish with academic quality.  So we wanted to enhance Wikipedia.  I asked the, as a University teacher had I discussions with colleagues of mine which won't allow Wikipedia to be used as a source.  I said that doesn't make sense.  I tell my students, use Wikipedia and check the content, don't stick to just one source of information.  Check with others.  I'm very happy to see that there are several open educational contents in Latin America.  The language barrier is relevant.  What I wanted to do with my students at the university, when we did the Wikipedia project for three or four years, awhile ago, was to enhance the content in Wikipedia in Spanish with content from the university level.  We succeeded very much.  The content generated was very good.  Maybe I will stop here and my message is:  Talking about quality, if you find a source of open training course with content and maybe have doubts about the quality, my suggestion to someone trying to take it is go to the sources of the information, check the organisations in both the development of that content and perhaps if you have a relevant information from there, check that with other sources.  I used to study other times that the books were in the library in a place and had to be there or you have to be a friend of the professor that had the nice book, and he should lend it to you.  That time is gone.  You have many more resources now to check the sources, to check the quality and double-check if it is okay or not.

     I would say that and I think Patrick made a very interesting comment.  The community revision, the community in relation with the content.  I have encyclopedias at home that I printed on paper and they have mistakes and they will have them forever unless I burn them.

     (Chuckles.)

     >> OLGA CAVALLI:  But if it's in Wikipedia and I have written many Articles there, people go there quickly and make changes.  That's the power of the Internet.  I will stop here and I'm happy to address questions.

     >> MODERATOR:  Thank you very much, Olga.  Interesting suggestions.

     So dear panelists, what do you think, who is responsible for content quality control?  Now after these comments, probably you know some best practices.  What do you think of these open educational resources, do they bridge the digital divide gap or not?  Do they help or not?  By the way, continuing this, what do you think:  Should we first teach media literacy in order to determine what is quality of content?  What is the matter, what is the relation?  Just to panelists itself.

     So who is going to answer first?  Okay, Olga, yes.

     >> OLGA CAVALLI:  I continue bothering you with my talk.  I think there are different types of content and all of them are useful.  Some Articles could be informative.  They don't need to be necessarily extremely complicated or extremely, I would say in English -- in Spanish it would be very much academic style or extremely complex.  But an informative Article could be extremely useful for many people as a starting point.  I think, for example, that is the focus of Wikipedia or other encyclopedias.  From there start the investigation towards other content that could be more complex.  And more checked before they are available.  So I will leave other colleagues to comment as well.

     >> XIAOFENG TAO:  Okay, thank you.  This was a very easy answer.  The first step I think would be to encourage university teachers and the public to produce open courses, as many as possible.  Whereas we have a lot amount of open courses, we will be able to fill out these local courses.  In China, for example, in order to assess the quality of an open course, the Ministry of Education used to invite experts in this area and also students to the public review of these courses.  Thank you.

     >> MODERATOR:  Tracy, please?  Thank you very much.

     >> TRACY HACKSHAW:  So good afternoon, everyone.  I would like to speak on the community point, which I think is very important from the standpoint of a small country like where I come from and the communities we do see a lot of knowledge emerging and in my work where we have built a few access Centers and asking the community to come in and to utilize the centers and to consume content.  We also on the other hand want them to create content.  I do think that there's a possibility that educational resources can emerge from the community that is contextual to what they do.  As an example, I wouldn't expect a University to be extremely proficient in basket making or fishing.  But that could come from a community where that industry or that way of life is endemic.  And building the content at the literacy levels, allowing it to be put online first and perhaps revised for the quality and for other issues by perhaps some experts in that area.  This could be a richer level of content emerging that the Internet and the tools that are available, YouTube and Cosera and court academy and emerging technologies and platforms that exist, we could start to see new forms and new ways of learning and new ways of material. 

     So I wouldn't go to the university to learn how to fish, but let's go to this location and this particular platform where fishing or basket making or bracelet making course may be available.  Again that creates a new class of entrepreneurs, a new level of empowerment.  The key is that the communities themselves become empowered to continue creating content but themselves programs seeing about business and become more literate, and become a viable economy.  It is something that is available in all series of areas and themselves in the future becoming quality and gate keepers for those coming thereafter in their communities and so on.  Thank you.

     >> MODERATOR:  Thank you very much, Tracy.  Andrey, you have some comments.  I'm wondering about our technical assistants, if we have remote participants, Remote Panelists wishing to comment on something.

     If we have remote participants, wishing to comment?  Otherwise, would you please?

     >> Thank you.  We have one comment from Danang from Africa.  He says the future much the Internet in Africa lies in building infrastructure like fiber cables.  He just left this comment.

     >> MODERATOR:  Thank you very much.  Comment from Andrey, yes.

     >> ANDREY SHCHERBOVICH:  Switching this.  Yes.  It is working.  So I would just like to add something about the quality of online content.  So this quality depends, I think, on the communities within which the course is proud.  If the content provider is the most alternative international organization, as I provided in my report, this could be understandable.  But also I think this is depending on which Internet.  The problem of the quality of the content is a problem not only of the educational resources.  It is always the problem of the whole Internet which is required user-generated content.  This is a kind of a draw bask the free flow of information.  As we can see, one could also produce educational courses on his own base, but this is a question of usage and popularity of this, if there will be, of bad quality.  Nobody would come to attend.  So there is a voting of attendance, there could be.

     So this is a really good estimation by the people who are attending the course.  Thank you very much.

     >> MODERATOR:  Thank you very much, Andrey.  Any comments?  Patrick, you wanted to comment on something?

     >> PATRICK RYAN:  I have a question if I may ask, a couple questions for you or for Andrey.  I'm curious to understand the context of the approval of classroom material.  In other contexts other than online education, let's just say books or Articles.  If you are a professor and you are responsible for putting together a course and you choose certain articles that are relevant or choose to read, have a case of law, read a case that is relevant or even assign a book, is any of that material approved separately through, or accredited in any way?

     >> MODERATOR:  Olga, probably you first.

     >> OLGA CAVALLI:  What I can comment and I didn't comment before because I was enthusiastic talking about other details, is that in Argentina we have this fantastic program that has 5 million computers donated to students.  But they have also developed specific content which is both online and it is on CDs for those that have no Internet access.  Those CDs can be delivered to distant parts of the country.  There is a fantastic TV channel which is called (speaking foreign phrase.)  Encuentro, which means getting together.  The content is educational and mainly proud in Argentina.  It has also some content produced in other countries.  It is in Spanish and it is distributed all over the country through the new digital television network which covers the whole country.  So this is an interesting example.

     That content is prepared and this goes to your question about certification.  This content, this content is revised by a special agency of the government called Educar, that prepares the content not only for students but also special content for teachers.

     The question came to my mind and I reminded my high school days who had control over those books that we had?  I have no idea.  But was there something?

     >> MODERATOR:  Thank you very much, Olga.  I will shortly answer.  In the Russia we have the same.  We have special standards for let's say for the programs, right, for the courses which should in fact we should follow.  But we also can introduce some new things under the standards, right?  Under this special rules.  The books, we have official books, yes, which are approved for schools and Universities.  Let's say we have official books, more like fantasy or whatever, depending on the author.  That is actually depends, I think, the quality of this context, of this content, I think, depends on the individual educators.  Well, you say beliefs, right?  It's a quite interesting right.

     We have Remote Panelists interested in talking to this topic.  Sarah from Christian University, so please.

     >> (Speaker away from microphone.)

     >> MODERATOR:  Please, Sarah?

     >> Greetings from Uganda.  I'm -- I'm happy to be back in this panel.  I wanted to comment about who is responsible as an academic institution, I think it is (audio feedback) ideally suggest review all the content before you put it online in a repository.  You can have it reviewed discern it is important to have a policy that covers standard content.  It is important to have good content at the end instead of having a quantity of content that has to be withdrawn later.  Those are my comments.  Thank you.

     >> MODERATOR:  Thank you.  It should be reviewed, right?  Tracy, please and probably participants on site next, please.

     >> TRACY HACKSHAW:  I wanted to comment on perhaps the hybrid model where in our Diplo Foundation where I do some work, they are linked to the University, but they offer content that is considered to be more industry-specific content.  So what I've seen happen with that online content is that it is both reviewed by the University of Malta in one scenario, but depending on the particular course content, there is expert review from experts in the field.  If it is diplomacy course or a course of something academic, you might find the university treating with it.  If it's on cybersecurity or something on Internet structure, you find industry experts reviewing the content.

     A hybrid model is important.  One of the things I want to say is in my country, the content at the University tends to have, is very static, if that is the word I would like to use.  Sometimes it is not very relevant, especially in the areas of ICT.  We have people in our Universities learning COBOL and C in the days of Java and other technologies.  The University itself, even though it has the curriculum in place and the structure and so on, hasn't yet been sensitized, if that is the word to use, as to what is truly relevant in the industry.  You find students coming out of the university who are not ready for the workplace.

     I want to encourage universities who offer open content educational resources to be sure they are reviewed by the University structures but seek out industry relevance and perhaps some industry expert panels to look at it, especially as it is going into the online world.

     >> MODERATOR:  Thank you very much, Tracy.  I'm just continuing following the line.

     So from business perspective, from private sector, it is more about user generated content, but it should be under review by the community for some time in order to put it somewhere for wider community, right?  In order to guarantee that it is relevant content and it will be useful.

     So Professor Tao, you wanted to comment about MOOCs, right?

     >> XIAOFENG TAO:  Yes.  Okay.  My comment was about open education resources in China, also some cultural control.  For open educational resources, it is an international obligation.  Something like Internet plus.  In our country from this year we sometimes will see university plus.  I think maybe OER, open educational resources is kind of Internet plus, Internet plus education.

     And it is ultimately object is to educate person and promote life long learning and personalize learning.  In the end, it is a development that will benefit all of society.  The development of OER in China is mainly over seen by the government.  The Ministry of Education and the Ministry of Finance has invested a lot in OER including online quad, we call it CEOC, like the MOOCs.  This is a Web site.  The courses is an official higher education course resources, the platform now has more than 1,000 open courses that are freely accessible to everyone.  In China MOOC meetings gained popularity since 2011.  Now, 54 Chinese universities have already opened their MOOCs I-course as a Web set.

     Like other popular applications, we chat.  Maybe some of you use Facebook.  We have many users.  The number of subscribed students to the MOOC is huge.  Last year in March there were only 61,000 students registered to the I-course.  Most of them select engineering and many of them select literature.  And maybe some of them select some science maybe.  And in China most open courses and MOOCs are lectured by Professor with strong research background and teaching experience.  These courses will be submitted to the Ministry of Education for initial review.  And some may be rejected.  Therefore, their quality can be somehow guaranteed.

     Maybe we can introduce voting or recommendations system to assess the popularity of open courses and MOOCs.  For example, when we come back and we want to find a better restaurant, we can walk across the street and see one restaurant with many customers, maybe that is better.  We can look up the Web site.  We can read the comments and maybe we can select a better restaurant.

     So I think voting or recommendations systems is very important.

     Students favor the courses which have comments and the recommendation from other students.  Actually, in the university every year the students will vote us, if we give lectures to students.

     I believe we should encourage the University Professors to produce much more open courses.  I think this is very important.

     And students will have much wider career choices over open courses and MOOCs so they can work and will have an incentive to attend the courses and the patience to complete them.  I believe OER, open educational resources to educate more people, that goal will be achieved all over the world.

     >> MODERATOR:  Thank you.  I wanted to comment here, Patrick mentioned that metrics should be introduced.

     When we are talking about open resources without a connection to some particular university, we are talking about translational, trans-border access.  So it depends on media literacy, right?  Media literacy of users of this content.  That's why when we are talking about, do you think we should use these metrics in terms of media literacy or another name, right?  But indicating who is going to understand what is inside.  There are different levels of education.  So we might have also have different levels of media literacy there and different outcomes out of these courses or out of this content.  Please, Patrick, some comments, please?

     >> PATRICK RYAN:  One of the reasons why I love our conversations is because we live in a world with vastly different perceptions on how to view different things.  Being able to talk about these things and compare best practices is just unbelievable.  In terms of its value.  Thank you all.

     I wanted to share an experiment I did in this context because I know a couple of people mentioned Wikipedia and Olga talked about it.  I want to say, there's a story here that I want to share that I think outlines the philosophy that I have and that many of us have in the Silicon Valley about approvals.  And that essentially we don't need to worry about rating systems.  The only rating system we need is the crowd.  If there's an accurate way to get to the crowd, your information will be rated.

     Let me tell you a story, an experiment I went through to reach that conclusion on my own.  Right in 2004 when I moved back from Belgium to the University of Colorado, I was teaching a course on standards.  It was really a course on the policy impact of lawyers.  I'm a lawyer, not an engineer, so it was about the different roles of standards bodies.  One of the things that was just fascinating is, if you think about content, how do you generate content?  Is there a standard for the creation of content?  Of course, there isn't. 

     We are moving away from a world where publishers receive manuscripts and information and chose to publish something or not in a book.  That world is gone.  An aspect of it is still around but if you want to publish today, you put your information out on a belong and it's there.  If you want to write a book, through Amazon Publishing Services, you've got a book.  None of the traditional gatekeepers in a world where there was real scarcity of materials or to be able to produce these things, that the Internet has taken away.  We have just a different world today where that can happen.

     So in 2004 I got into a number of debates with colleagues and friends of mine about whether or not Wikipedia was as accurate as an encyclopedia.  And I have my old emails to my friends to back this up.  I said in 2004 when Britannica was still around and Encarta was still existing.  That was a Microsoft project where you take CDs, these are Articles written by professionals, take them on CDs and put them on your computer and Microsoft was getting into the business.  I said I bet you all of those companies are going to go out of business and Wikipedia will be come the most trusted source.  And everyone said they couldn't become the most trusted source because we don't know who is writing the articles.

     We have a saying in the United States, for the average person, we call them Joe Sixpack.  Joe Sixpack is the guy who comes every day with his six-pack of beer, drinks his beer and for many of us, that's the stereotypical character in the United States.  And so the idea that Joe Sixpack could be contributing to Wikipedia Article versus Dr. Ryan is just offensive to many people.  That is not the system we know.

     But as it turns out, as Olga mentioned, the systems are self healing.  What you have is a set of resources where people can constantly edit, improve and give feedback.  Joe Sixpack may be on the ground and know a fact that the Ph.D. may not be able to research, and if he's wrong, it turns out that other Joe Sixpacks in the know may be able to fix it.  We don't know or care what their educational level is, it's whether the content makes sense.  Who determines that?  It's the crowd.  If people look at things and agree things are right, they are going to surface.  By metrics that are so simple because it's a number of page views, a number of refreshes.  You can look at how long people look at sites and whether they are cited or referred to in other cases.  These are, in the old word you had a book, you printed another book and it took five years and now you can do it in a couple days.  It's just as relevant.

     In the experiment I did, I had my student -- I'm almost done here.  An experiment I did, I had my students because I wanted to establish to myself that this was true.  Multiple Joe Sixpacks would be fast and efficient so I assigned my students to do two Wikipedia entries.  One, they had to create an entry that was just absolutely false.  I wouldn't ask for this today because this is against the Credo of Wikipedia.  Mike Godwin is here.  If you see him, he is the author of Godwin's Law, look it up on Wikipedia.  It's fascinating.  He was the General Counsel of Wikipedia and I'm sure he would find some way to issue a fine to me, so promise you won't tell him.

     So I told my students to write something intently false.  The other was to go in and edit an Article, that we had a minimum number of editors, so put in a new fact, just to explain the experience in class.  And what happened was, across the board with one exception, with one outlier of the three or so -- 30 or so students, all of the Articles put in that were false, all of them were taken down almost immediately by another Wikipedian, almost immediately.

     Or if not, they were taken down within the case of maybe a week or two.  Try it.  Put up an article about something that just doesn't exist and see how long it stays up there or go into an article that is heavily edited -- I retract that.  This is not the way we want to treat Wikipedia so I retract that, but I did this and it happened and it's a fascinating experience.

     Of course, now many years later, Encarta is gone and Britannica is gone.  These are all gone, out of business because their information is irrelevant.

     This is the foundation I believe for the future of education, where you go to exactly what Mikhail alluded to.  You will still require a curator.  Somebody needs to do that, but that curator is no longer in my view the state or not even necessarily bodies that look at that.  That's gone.  You can get instant feedback from the people you are serving.  That's it.  Then you have your professor, your person that you are relying heavily on the morals and values of that person.  That's an excellent point that you made and I believe that is subjective and is maybe unsatisfying as that may be, I believe that's the future and I believe that's where we are going to head with the future of education.

     >> MODERATOR:  Thank you very much for a very interesting comments.  So I would like to ask on site participants, do you have any comments or questions to the panelists?  Otherwise I will continue to ask -- yes, please.  You are the first, yes.  Can we have a microphone here?  Oh, it is over there, yes.

     And could you please give a name?

     >> AUDIENCE:  Yes, I am representing the Digital Opportunities Foundation from Germany and we have been working there for nearly 15 years in overcoming the digital divide.  I would like to pick up on the point that Olga made.  When you, when it comes to the question of quality assurance of open educational resources to our opinion the most important thing is teaching media literacy first because if people just don't understand that they have to question any content they find on the Internet, they just take it for true.

     So firstly they need to learn how the content comes on to the Internet, how that not everything that you find on the Internet is quality checked, but they have to play a role in community assessing the content.  That is all part of the digital literacy training.  I say that comes first and then we can follow up with everything else.  Thank you.

     >> MODERATOR:  Thank you very much.  I would probably add, you know, in Russia, so we actively are discussing now media literacy starting from high schools, starting from first grade when kids already are in school because they already have digital devices, right?  And so they have to deal unfortunately, or fortunately -- anyway, they have to deal with different types of information already from their first days at school.  And that is why we are discussing actively media literacy teaching and training.  I think that's a good point.  I truly believe it is an outcome of this workshop, we should think about from one point of view, capacity building in terms of those who are going to create those courses or special training programs, media literacy, or supporting these organisations in terms of public private partnership.

     There was another question, yes, sir, from on site participants, please.

     >> AUDIENCE:  Hi.  My name is Grace from Kenya and also from the Internet Society Ambassadors program.

     My concern being a Kenyan which is a developing country, is that sometimes there is over concentration on qualification.  I think sometime back in my country there was a policy by the government that you would need a certain qualification called ICDL in order to show that you are computer literate.  That was about maybe three or four years ago.  When you think about it now, it is laughable because everybody who comes to a big city somehow becomes computer literate.  So for me, the question of open educational resources brings the other question:  Why do we need open educational resources?  For me it is to give youth life skills.  If we give them life skills we will help bridge the digital divide.

     So if they want to cite support?  I would support the side of let us have more content that will help to give people life skills.  Whether it is content that is academic or whether it is content that is about maybe traditional knowledge like Tracy talked about.  And then let the users be the ones to decide the quality because, for example, if I want the to learn yoga, which is a life skill that could give me a job back in my village, I would look for the YouTube video or the PDF document that has the most likes.  That is what is important to me.  The users have already defined the quality for me.

     So as far as we are talking about Developing Countries and bridging the digital divide, it is the user generated content that rules.

     >> MODERATOR:  Thank you very much.  Please, Tracy.

     >> TRACY HACKSHAW:  Yes, I want to respond to that quickly.  I someone once told me when we were building applications for this kind of work that did anybody teach anybody how to use Facebook or YouTube?  Is there a manual?  No.  So when you go on to those locations and you see the people using it, there is no manual.  But as grace is saying, it depends on the people who are using it.  It is almost the community is telling you now this is what we like, this is what we are interested in.  This is the value we are giving to this type of content.

     It is very important now for academia and for traditional education to understand that things are, as with other types of knowledge, the music industry has seen it, the books, everybody has seen a change, a shift that the user-generated situation is emerging and that we should really look to build and bridge those gaps now.  So we don't only focus on strict qualifications and strict ways of looking at things but also on what users and what the community is telling us is in fact the way that people are thinking and feeling.  Thank you.

     >> MODERATOR:  Patrick, you wanted to comment?

     >> PATRICK RYAN:  Yes, please.  Joseph Schumpeter was an Austrian economist that came up with the idea of creative destruction.  You have an industry that is established and it is not another established industry that takes it over, but something that eats you and you're gone.  There are examples of Schompeter in action.  Blockbuster Video, if you wanted to go get videotapes, that was the big company, and they are gone.  There are many of those because new technologies have come to replace them.

     And that context, I think that we all -- I spend a good amount of time in academia myself.  This is why it's important to me.  I spent a lot of time getting these academic qualifications.  My gosh, for those of us who want to teach, how long does it get those credentials and you take years and years.  I am of the belief that it is increasingly going to be worth less.  We can talk about whether it's right or wrong, but I believe that the Internet and the way the Internet Democrat ties everything, will make the value of degrees much less than it was in the past.  That too will lessen the need for accreditation, for review, for any body for licensing.  Just the information itself, you don't need the degrees. 

     We have a world in the United States where very senior leaders, some of the world's best leaders in our area -- Bill Gates?  Michael Dell, founder of Dell Computer?  Richard Branson of Virgin Airlines?  Anybody know what they have in common?

     They have in common that they didn't graduate college.  Right.  I think that's fascinating.  Here we are increasingly accepting that, that that's the traditional measures of degrees in education is not so relevant as well as the question of morals and how you are as a person.  These guys are successful and they have done bad things like many people do, but you don't need to have the traditional standards and metrics of the past in order to be extremely successful today.  I would ask the question whether we think that is going to change.  Is that trend going to continue to change?  Are these aberrations?  I think it's a demonstration of Schumpeterian principles in mind and as academics, we need to keep that in mind.

     >> MODERATOR:  Thank you, Patrick.  Any other comments from on site participants?  Yes, please.  And technical assistance, could you tell me if you have any questions from remote participants?  No?  Okay.

     >> AUDIENCE:  Good afternoon.  I'm Deirdre Williams from St. Lucia in the Caribbean.  It was what Patrick was just saying.  Recently we have been having arguments about governments getting laptops and tablets to children.  Maybe you spoke about this before I arrived because I was late, but I wonder if Patrick or anyone, you see -- I was just about to look up the name of the man, I've forgotten it, but the man whose idea was there and then one laptop per child, hmm?

     >> AUDIENCE:  No, not Negroponte, he was a psychologist and engineer and he did LOGO.  Do you know who I mean?

     >> (Speaker away from microphone.)

     >> AUDIENCE:  No, I don't.  Anyway, his educational theory was wonderful, fascinating.  And it is really sad that it degenerated into a commercial means of selling cheap laptops to children in Developing Countries, or whatever.  But what you are talking about is what he was saying.  That you just let the child educate itself.  By and large.  And do you think now with your creative destruction that that is what is going to happen?  Thank you.

     >> PATRICK RYAN:  Thank you.  I think it's a little bit different.  There's a certain degree of education ...

     >> (Speaker away from microphone.)

     >> PATRICK RYAN:  I believe there's a degree of self education that will be available.  For the most part you are going to get into a world where, it is not going to be individual effort.  There is going to be feedback.  There is going to be social around it.  People in your familiar pro proximity.  For example if you want to be as one colleague said, a yoga instructor, you can look online to see what they are doing, what they are reading and saying, and your particular geography the Internet may show you what types of yoga is best.  You might find that bikram is offered but another form is not.  What a great resource to use online to help you educate yourself.  Doesn't matter what the certification.  You are not doing it by yourself.  You're getting guidance and getting your hand-held by others who walked this path before.  They may be next to you or online.  They are sending indications in some way.  It is not the way we would think of it in the traditional sense.  It is more than one person.

     I still believe in the true educational context that yes, I do believe -- this is controversial, but I believe you can give a laptop with full Internet access to children.  There is crazy stuff online but you address that by educating children, this whole discussion about digital media literacy has moved away from the idea that there is a separate discussion of digital media literacy.  That's counter intuitive, but digital literacy is a thing of the past.  Now there's just education.  It is primarily, exclusively in many cases online.  Through the medium we will teach that and teach children how to differentiate between different qualities of things online.  It is not obvious.  Online, maybe the first time you look online but people learn.  You learn signals about quality and you learn as you grow older that you write footnotes and there's subjective value in footnotes.  That replaces the notion of a separate digital media literacy because we will think about it in a much broader context and that in itself will create an environment that creates an educational system that continues to grow.

     >> MODERATOR:  Thank you very much.

     As we are, have like ten, 12 minutes more, I would like within this context I would like to move to the question in terms of special domain zones.  I think the good example of Ed-U, right?  I think the United States introduced.  So in terms of policies there, it ensures that there are institutions registered under the domain zone, right?  So the courses or content or these institutions are certified by agencies, educational agencies there.

     Do you think we should focus on special separate domain zones for this let's say courses which are approved by special stakeholder like government, for instance?

     Because we are following the multistakeholder approach, right, in terms of open education as well.  So we have business introducing different services and different platforms.  We have educational institutions, right, using this platforms.  Sometimes participating in creations of their own platforms.  We have government introducing governmental platforms.  We have international organisations, but do you think under the UNESCO umbrella or other organisations, there should be a special domain zone where we give opportunities to the end users to choose which courses to follow, provided by the business or by the international organization.  And why.  Because for my opinion, there should be that choice:  Why?  Because either we have international organisations participating in creating this or different other authorities, but participating and creating this content so they can officially issue some, let's say, certificates which will be recognised.  Because legal aspect which was touched a little bit here is quite important.  You know, not everywhere in the world, not in all the countries officially open educational resources are accepted.  Even reference to open content is not accepted in some countries, right?

     So what do you think about it?  Any opinions?  Long question.  Okay, should there be he special domain zones for educational organisations providing those courses issuing some certificates under the international umbrella?  Or should it be like we have now?  Open space there, international organizations either they create their own diploma system or might use Corsair or some other platforms, what do you think?

     >> PATRICK RYAN:  Right now we have a system overlaid in different ways.  In the U.S. we have five different regional accreditations, national accreditations for different topics, topical accreditation and these are all a market for accreditations.  Some matter and some don't.  Having a stamp doesn't really mean anything except for reputation Allie, at least in the United States.  We tend to use a very open approach in that sense where anybody can say I'm going to be an accreditor and as crazy as that sounds in other places in the world, we say that's fine and we will leave it to the general public to look at that and decide whether or not that particular accreditation is sensible or not, whether or not they follow it.

     And so I think that I believe that the countries that are looking to ban open source material and open materials from the public do so at their peril because they are competing with countries that don't.  I would say that generally -- there may be exceptions to this rule, but I would say that generally the people that are organizing that, they that are making those decisions aren't going to be able to keep up.  In the case of YouTube alone, there's more than 100 hours of new content uploaded every single minute.  It's so fast, certainly no human at Google can do it.  We gave up.  We never tried because our intent was to have computers and systems do that.  Older systems are saying we are in this old world, it's analog, not even accepting that the material exists, almost.  It's unfortunate because it leaves the citizens out from the opportunity to learn a lot.

     >> MODERATOR:  Okay.

     >> OLGA CAVALLI:  Before I move on I would like to name the person that Deirdre was referring.  His name is Seymour Poppert, a citizen of Pretoria, South Africa, and I found it in Wikipedia.

     About domain names, I would like to contribute on that.  In Argentina we have dot UAR, which is part of our national, second level of our national dot AR.  You can have it dot AR if you are somehow accredited academic institution or a school or a high school.

     In Spanish we have a saying (foreign phrase).  That means that you only have, maybe have nothing behind.  So certifications are fine, but sometimes as I usually say, you have to check the sources, check who are the organisations involved, so that is important.

     If there would be a specific TLD, you mentioned something like that, TLD for that?  The problem is TLDs are global.  So I honestly don't know.  We had so much discussions in the back about the TLDs being global.  I will stop here.  We should think about it.

     >> MODERATOR:  Yes, please. 

     >> BONFACE WITABA:  Thank you.  I believe in terms of top level domain, that's a bit tricky.  The question is, when you put it under top level domain, is it still open?  If it's open, who is responsible?  Because I would like to believe, for instance, in Kenya the course has to be aligned to the curriculum in Kenya.  If you set it global, for instance, the course in America could be different from whatever is in Kenya, or Uganda or Tanzania.  If you set it regional?  Yes.  I will stick to the traditional methods, one, peer reviews whereby people are knowledgeable in the subject matter and can give approval.  Experts in specific subjects, a Professor or doctor.  Lastly, approval, writings.  For example, if phonetical is approved highly, it has five stars or five is the highest rank, then I think that would be good to go.  Yes, thank you.

     >> MODERATOR:  Thank you, Tracy.

     >> TRACY HACKSHAW:  Over the last 15 years of my life I moved from an employee to more of an employer.  And I've quickly come to realize that certifications are the least of my concerns.  As you hire employees who are certified with Master's degrees or whatever, you tend to find that the people who really do contribute and bring value to your organization tend to be beyond the certification.  So on one hand it might be experience.  The other hand it might be the personality.  It's a series of other factors.  I am very loathe to treat with certifications now, we are dealing with the issues of certifications.  The reality of certifications is that it is just one check in the box of how you move from one part of your life to another.  So I do think there is an opportunity for some sort of real world type validation of that training.  So the OER type world allows, I think, for that kind of moving to that model where you can -- I like the idea of payer validation or reference check is what we call it in the HR world, I guess.  So you are looking to see if someone else can fit your organization.  It is beyond whether they have a degree or beyond a Master's or whatever they have.  Are they a fit?  A good choice?  Do they have the passion?  The hours you need?  Those are the kinds of things you are looking for.  Maybe in this new dispensation where some sort of -- I don't know if it is a TLD, I don't know what it can be, but it can be some other method of evaluating.  When you do educate -- you do need education.  Do you need a degree or do you need some other validation?  I think that's important.  I am not sure how to address it.  That's what I'm looking for, something else beyond the certification itself.

     >> MODERATOR:  Thank you very much, Tracy.  As we have several minutes, if you have any other questions from on site participants?  So please.

     Just to sum up, I would say that we discussed most of the questions, I would say.  That's very nice.  It's great.

     Please.

     >> AUDIENCE:  My name is Mudong from Kenya.  I'm the (indiscernible) Ambassador.  We find that there is so much public knowledge that has been generated through maybe, funded through taxpayer's money like is done in most of the countries.  We find that that information, most of it is actually not public knowledge.  What can we do probably to engage the people who are hording that knowledge to release it into the masses?  Especially this is something that will really help the Developing Countries.  Thank you.

     >> MODERATOR:  Thank you.  Patrick, yes?  Please?

     >> PATRICK RYAN:  These are open government initiatives.  You are absolutely right, there's a lot of information out there that just isn't available.  Citizens that pay for it should have access to it.  Even if it is not good, why not put it online so it's there?  And the challenge in that is, one is it depends on human effort.  You need to scan it, put it up there and mechanize it.  So the answer to that is by, first of all, doing what you are doing right now which is raising attention to it and making sure that people are alerted to it.  One of those topics that you have to bring home, everybody has to bring home and they want to get their municipality or city or village online, go talk to them about what their online strategy is.  If they don't know, find tools to help.  That's really the best way to go about it.

     Because particularly in the rural context, there's other examples in much -- in broader contexts like the government of Estonia has done everything online.  That's great.  It is not going to work in a lot of rural -- you know, the next three bill are going to come online.  It is not going to work for awhile.  We have to be realize particular and be much more practical.  There are other options available.  You know, very well healed governments like Estonia.

     >> MODERATOR:  Please, a comment from Professor TAO?

     >> XIAOFENG TAO:  That was the case from our side.  Our University of Telecommunications, the government encouraged our professors to present the principle of communication.  It is very important for our students.  So one professor in our university, now he is winner of the national standards teaching team on this course.  It is a very high award and very important.  Thank you.

     >> MODERATOR:  Thank you very much.  So I guess we are almost on time.  That's great.  I would like to thank all the panelists for coming, for sharing their experience, their ideas and opinions.  I would say we will definitely circulate our report in order to correct, put some corrections, but I guess we discussed most of the questions which we wanted to discuss.

     I would like to thank the on site participants for your participation, for asking questions and I do hope that it was useful for you.  I would like to thank the Remote Panelists and participants and also technical assistants.

     So thank you very much.  I do hope that we will continue next year, but more let's say deep focus on metrics or topics connected to education and open educational resources.  Thank you very much.

     (Applause.)

     (The session concluded 1530.)