Category Archives: 21st Century Skills

Gormenghast Education

 In the Gormenghast Trilogy, Mervyn Peake paints a vivid picture of a huge and crumbling castle looming over a community. Within Gormenghast Castle the residents perform endless ritualized activities which have no relevance to the population around the castle, and even the residents of the castle can’t remember why, they just believe them to be important and not to be broken. Once a year the citizens of the village outside the walls were allowed to enter the castle and display their carvings for the castle residents. This quote describes that annual event.

Very little communication passed between the denizens of these outer quarters and those who lived within the walls, save when, on the first June morning of each year, the population of the clay swellings had sanction to enter the Grounds in order to display the wooden carvings on which they have been working during the year.

Mervyn Peake, Titus Groan

While Gormenghast is clearly a fantasy I can see certain similarities between it and our current schools. Classrooms are strange places which bear little resemblance to the world outside the school. They have their own rituals and practices which are rarely questioned.

When parents visit a classroom there is something of the atmosphere of the village people visiting Gormenghast. The atmosphere is that of people entering a space where they don’t belong. The education space is separate from their day-to-day life. Children enter this space to get an education before emerging (graduating) to enter the real adult world. Rather like butterflies emerging from the chrysalis.

Teachers find these analogies confronting. As a teacher we see schools as vibrant social spaces where children and young adults grow and develop, but this is the view from within the castle. For those looking in from the outside it appears quite different. When someone in the community thinks about a class room they often imagine something like this:

As teachers we know this is outdated. Modern secondary classrooms look like this:

The students look happier, but in my opinion, functionally the difference is small. The fact is that our K-12 schools are still based on an industrial model. They are essentially production lines where students enter in kinder and work through the curriculum in lock step, coming out the other at a graduation. It comes a surprise to many that education has not always been like this. Schools  as we know them appeared as a way of providing mass education in the 1800’s. Prior to that time education was based around community tradesmen and experts passing skills on to the young people (professional teachers were rare) and usually look more like this:

When asked to provide mass education to the community in the industrial age, the educational leaders of the time used an industrial model. They essentially developed an education factory and in industrial societies it was, and continues to be, spectacularly successful. In Australia we have emerged into a post-industrial 21st Century world. The most valued traits are now creativity, individuality, problem solving skills, communication skills, initiative and entrepreneurship. We need an educational model which encourages these 21st Century skills.

At this point you might think I am going to advocate doing away with traditional schooling. Actually no, I happen to believe that traditional/industrial schools are still a very effective way of mass educating basic skills and starting to teach 21st Century skills. They are also an excellent way for young people to develop social skills vital for surviving in our overcrowded world. In our schools students learn the limits to individuality and the communication skills and social customs which are the glue for our society. What I do want to see, however, is a change in community attitude which recognizes that learning is an integral part of life. Schools provide one type of learning important in the early years of life, but there is learning which continues outside school throughout life. This life learning allows people to adapt and enriches their life. Life learning is needed to hone and further develop 21st Century skills and learn new skills as the employment market changes. It is the life learning where people further develop their creativity, problem solving and communication skills in a real world context. In saying this I don’t want you to confuse life learning with life experience. Life experience is certainly valuable, but life learning can also be structured learning. Attending short courses, reading, practicing new skills, etc. these are the life learning experiences I am referring to.

There are a number of reasons why many in the community do not see learning as part of their life. Their experience in school, the opportunities available, cultural factors and many other things contribute to community attitudes to learning. To cover them all would require a text book rather than this short blog post. One aspect I would like to focus on is recognition of learning. Modern society places high importance on evidence and the measurement of learning. This gives institutions like schools an advantage, as they have the infrastructure to provide robust credit for the courses they offer. This in turn means that formal school-based learning is more highly valued by the community. The life learning, which is often  not supported in this way, is considered of less value and more of a hobby activity. In fact as we grow older it is the life learning that becomes more important.

To raise respect for life learning and encourage more participation, one important step, therefore, is to democratize recognition for learning. The giving of credit needs to be taken out of the exclusive control of the learning institutions and distributed to the community. One effective way of doing this is to facilitate and promote open badges in the community. Open badges are a computer image or icon which links to information about what the badge was awarded for, the issuer, the recipient and supporting evidence provided by the recipient. The badges can be shared online (social media and blogs) and through email. Using open badges it is possible to give robust credit for skills and achievements in a non-institutional context. I believe that providing an open badges infrastructure to a community so that non-institutionalized learning is recognized equally with institutional learning we will take a big step towards a learning community. A community that sees learning as a part of living rather than something associated with a period in their early life. Some communities are beginning to embrace this idea to great effect. Perhaps the best known are the Cities of Learning in the USA. Even there, however, the emphasis is on youth and I would like to see programs develop that encompass all life stages. So, in summary:

  • Our schools do their job, but they are an uncomfortable environment for most adults.
  • If we are to have a learning community, a community which views learning as part of life we need to break the association learning has with schooling.
  • In turn we need to democratize recognition and credit for learning within the community (pre-industrial model learning) so that learning outside institutions is seen on an equal footing with school learning. Open badges are an effective way of achieving that.

Teaching and assessing problem solving skills – PISA recommendations

In 2012 PISA (Programme for International Student Assessment), which operates under the OECD, released their evaluation of problem solving skills in 15 year old students. The study defined problem solving and established an assessment regime. They then compared the results from 65 participating economies. The full report is at

I don’t propose to critique, or even attempt to summarize the report, but as an educator there are some interesting points in it which I would like to refer to. I will be skipping through the report, highlighting what I see as key ideas for classroom teachers and mixing in a few personal observations and ideas. For the full context and supportive data you should refer back to the original report.

Importance of problem solving skills in the 21st Century

Across the countries it was found that a large majority of workers are expected solve a simple non-routine problem (taking less than 30 mins) once a week. One in ten workers are confronted on a daily basis with harder problems. Complex problem-solving skills are more in demand in the faster growing managerial and professional occupations.

A suggested explanation is that automated systems are increasingly dealing with routine problems, leaving the workers to deal with the unexpected or unfamiliar situations.

As a result we see the following trend in employment towards non-routine analytical and relational skills.

As educators preparing students for the world, we need to consider these trends. Amongst other things we need to be seeking to equip our students with the skills to thrive in non-routine problem-solving situations.

What are problem-solving skills?

In order to teach and assess problem-solving it is necessary to define the skills and understand them.

PISA defines problem-solving as:

…an individual’s capacity to engage in cognitive processing to understand and resolve problem situations where a method of solution is not immediately obvious. It includes the willingness to engage with such situations in order to achieve one’s potential as a constructive and reflective citizen.

( pg. 30)

The key domains in the definition are identified as:

  1. Cognitive domain: The problem solver needs to engage, understand and resolve the problem.
  2. Problem domain: The problem is non-routine, meaning that the goal cannot be achieved by merely applying a proviously developed solution.
  3. Affective domain: The problem solver needs willingness to tackle the problem. I would add that willingness also pre-supposes confidence and the ability to handle failure positively.

The problem solving framework defined by PISA involves three main elements, further divided into parameters or processes:

Using this framework a problem can be categorized and the problem solving process (not necessarily linear as shown) described and evaluated.

Levels of problem-solving competence

To evaluate  the level of competence of a problem-solver there needs to be a progressive developmental framework. In previous posts I have discussed the progressions to evaluate collaborative problem-solving skills proposed by the ATC21S (Melbourne University) group and 21CLD . I have summarized these down to 4 levels relevant to secondary education

PISA present 6 progressive levels of development for problem-solving:

Level 1

  • At Level 1, students can explore a problem scenario only in a limited way, but tend to do so only when they have encountered very similar situations before. Based on their observations of familiar scenarios, these students are able only to partially describe the behaviour of a simple, everyday device. In general, students at Level 1 can solve straightforward problems provided there is only a simple condition to be satisfied and there are only one or two steps to be performed to reach the goal. Level 1 students tend not to be able to plan ahead or set sub-goals.

Level 2

  • At Level 2, students can explore an unfamiliar problem scenario and understand a small part of it. They try, but only partially succeed, to understand and control digital devices with unfamiliar controls, such as home appliances and vending machines. Level 2 problem-solvers can test a simple hypothesis that is given to them and can solve a problem that has a single, specific constraint. They can plan and carry out one step at a time to achieve a sub-goal, and have some capacity to monitor overall progress towards a solution.

Level 3

  • At Level 3, students can handle information presented in several different formats. They can explore a problem scenario and infer simple relationships among its components. They can control simple digital devices, but have trouble with more complex devices. Problem-solvers at Level 3 can fully deal with one condition, for example, by generating several solutions and checking to see whether these satisfy the condition. When there are multiple conditions or inter-related features, they can hold one variable constant to see the effect of change on the other variables. They can devise and execute tests to confirm or refute a given hypothesis. They understand the need to plan ahead and monitor progress, and are able to try a different option if necessary.

Level 4

  • At Level 4, students can explore a moderately complex problem scenario in a focused way. They grasp the links among the components of the scenario that are required to solve the problem. They can control moderately complex digital devices, such as unfamiliar vending machines or home appliances, but they don’t always do so efficiently. These students can plan a few steps ahead and monitor the progress of their plans. They are usually able to adjust these plans or reformulate a goal in light of feedback. They can systematically try out different possibilities and check whether multiple conditions have been satisfied. They can form an hypothesis about why a system is malfunctioning, and describe how to test it.

Level 5

  • At Level 5, students can systematically explore a complex problem scenario to gain an understanding of how relevant information is structured. When faced with unfamiliar, moderately complex devices, such as vending machines or home appliances, they respond quickly to feedback in order to control the device. In order to reach a solution, Level 5 problem-solvers think ahead to find the best strategy that addresses all the given constraints. They can immediately adjust their plans or backtrack when they detect unexpected difficulties or when they make mistakes that take them off course.

Level 6

  • At Level 6, students can develop complete, coherent mental models of diverse problem scenarios, enabling them to solve complex problems efficiently. They can explore a scenario in a highly strategic manner to understand all information pertaining to the problem. The information may be presented in different formats, requiring interpretation and integration of related parts. When confronted with very complex devices, such as home appliances that work in an unusual or unexpected manner, they quickly learn how to control the devices to achieve a goal in an optimal way. Level 6 problem-solvers can set up general hypotheses about a system and thoroughly test them. They can follow a premise through to a logical conclusion or recognise when there is not enough information available to reach one. In order to reach a solution, these highly proficient problem-solvers can create complex, flexible, multi-step plans that they continually monitor during execution. Where necessary, they modify their strategies, taking all constraints into account, both explicit and implicit.

I believe that this progressive development is sufficient basis for an assessment rubric and to establish zones of proximal development for students, which is the first step towards developing an educational process for teaching and assessing problem-solving. Test questions have also been developed and they can be viewed at .


Analyzing their assessment results PISA found that some countries were doing better than others at teaching problem-solving skills. On this basis they were able to make recommendations on improving education in this area. There were suggestions for improving educational policy, but I have adapted the following 5 points from their recommendations, which can be implemented at the classroom level. I suggest you refer to the original document for a more detailed discussion.

Don’t teach solutions

In general problem solving is taught by focusing on rule-based solutions. This is most obvious in Mathematics education. This is really a two step process, the first step being formulation of the problem from a messy real-world scenario, the second step is the application of the solution. Once the solution path is established the rest of the process can be automated, so it is the first step that is the more valuable skill.

In order to assist students to develop skills in problem analysis and solution formulation they need to be exposed to numerous real-world problems.

In the language of ATC21S this means exposing students to real-world problem spaces so that they learn to develop, evaluate and select solutions. Presenting students with restricted problem spaces leading to defined solution paths is not developing effective problem-solving skills.

Teach for skill transfer by looking for connections

problem-solving skills developed in one domain do not readily transfer into another domain. Teachers can assist this transfer by using diagrams and illustrations to highlight the similarity between strategies across domains rather than the superficial differences of jargon or context.

In practice this might involve finding the similarities in the design process when designing a house or a prom dress, or calculating loads on roof trusses and optimum tacking angles for a yacht. These pairs of problems seem superficially different, but the problem spaces have things in common.

Skills are best developed in meaningful contexts

People are less likely to transfer isolated pieces of knowledge than they are to transfer parts of well-integrated hierarchical knowledge structures. The more connections a learner sees between the learning environment and the outside world, the easier the transfer will be.

( pg 121 )

Teachers need to be prepared to look at the real world, particularly the world that students live in. I have always been aware that this helps with the affective domain in problem solving (willingness and persistence), but the evidence shows that it also assists with the cognitive domain by aiding skill transfer across contexts.

Encourage metacognition

Students need to be encouraged to think about how they are thinking about a problem. Self awareness through the process is extremely powerful in developing problem-solving skills.

This can be encouraged by “thinking aloud” sequences. Solving problems in a collaborative setting is also a way of encouraging this, particularly if the communication is managed. For example if students are placed in the situation where they are collaborating through a network chat session it encourages them to communicate their thinking explicitly to one another. It also records that communication for later analysis and discussion. ATC21S made use  this strategy in their work. I have also explored this using etherpad.

Teachers also need to be courageous enough to model this behavior for the students. Because of their familiarity with the subject teachers tend to model problem-solving as a routine activity with their classes, simply because they generally go into class knowing how to solve all the problems. It is not a bad idea to sometimes attack a problem that the teacher does not know how to solve.

Utilize the visual arts

The visual arts are often devalued by teachers, particularly teachers of the core disciplines like Maths and Literacy, as a place where real problem-solving does not happen. The visual arts can be a powerful vehicle for developing problem-solving skills. On a superficial level students are learning skills and techniques, but on a deeper level participating in the visual arts involves:

  • Envisioning: Students are asked to envision what they cannot observe directly.
  • Observing: The skill of careful observation is taught.
  • Reflecting: Teachers often encourage reflection by asking open-ended questions about the work. Students are therefore encouraged to develop metacognitive awareness of their work.
  • Engaging and persisting: Students tackle projects which engage them, and they need to persist through frustration as they refine and develop their skill with the medium.
  • Stretching and exploring: Students stretch themselves and take risks in producing their work.

So the visual arts are a powerful context in which to teach the basic problem-solving tools.


This post constitutes a summary of what I learned form the PISA report. It has given me a lot to think about as I evaluate my teaching practice. 21st Century skills, like problem-solving, are the key to future success for our students. Effectively teaching them is as challenging as it is worthwhile and work by PISA, ATC21S and 21CLD are showing the way for classroom teachers like yours truly.



Evaluating a learning activity under 21CLD

21CLD evaluation tool

21st Century Learning Design (21CLD) is a Microsoft supported project to promote the teaching of 21st century skills. They have produced an evaluation tool and a Windows 8 app called ’21st century learning design’. These are intended to help teachers assess their learning activities and develop learning tasks that focus more effectively on 21st century skills. In simple terms a teacher can select one or more skills dimensions and follow the tool to evaluate their activity. For the purpose of this post I have chosen to look at the dimension they define as ‘Knowledge Construction’. This is a key aspect of my innovation strand. The app takes the teacher through a series of questions about the activity, to place it on one of 4 levels of increasing effectiveness. This can be seen in the following screen shot from the app. (I trust Microsoft will not mind me reproducing it.) 21cld knowledge construction This is only the landing page for this evaluation, once the assessment has started more information and exemplars are provided at each of the 4 stages.

Evaluating an activity

Armed with the 21CLD app I decided to evaluate a task of mine. I have run this task for a couple of years, and it is fair to say I was quite proud of it.

Description of the activity

In my Computer Graphics and Design class I wanted to cover theory and highlight connections between concepts, while at the same time give students ownership of their learning. To achieve this I decided to give the class the task of developing a wiki. After some instruction on how to use the wiki software I handed out suitable page topics at random to the students and set them to work researching and writing a page. Each lesson I would give the students another randomly assigned topic and they either started a page or contributed to the one started. The lessons are quite long so only the first 20 minutes was allocated to this task before they went back to their practical assignments. More recently I developed a digital badge to reward students for significantly contributing to the wiki pages. Overall the task went well. Students were engaged and covered a lot of theory. (If you are interested the wiki lives at On the face of it this activity ticks a lot of boxes. It is well differentiated, as weaker students can contribute at their own level and be supported by the contributions of stronger students. The students are interdependent, connecting with each other’s work. They are investigating and constructing information and via the wiki links they see connections. It also forced students to think about copyright and intellectual property in a practical way.

Evaluation against 21CLD

The process of evaluation, as you will guess from the diagram above, is to work through a series of questions connected to levels of effectiveness under that dimension. It is possible to evaluate an activity against several of the dimensions, but thinking that this activity was strong in the area of knowledge construction I applied the 21CLD evaluation tool for that dimension.

  • The first question was “Do learners engage in meaningful knowledge construction?”
    • Reading through the documentation and exemplars provided, this activity is not ideal in this respect. Students are constructing knowledge, but in this case the knowledge is not meaningful in the sense that it is not connected with existing knowledge or experience. By distributing the topics at random I was actually breaking this connection for the students. I was asking them to work backwards and attach meaning to isolated concepts.
    • My task could be stronger if I had managed to make the learning more meaningful. Perhaps if I had started with the students selecting a favourite CD cover, book or other exemple and explore that. researching the techniques used and the designer that developed it. Working from that starting point to populate the wiki. My approach was content driven, and therefore not as effective.
  • The second question was “Do learners work with significant ideas, topics, questions and thinking?”
    • Once again my task was hindered by the wide selection of topics. There are certain key concepts in design: the design principles, design elements and design process. While these are covered in the early part of the course they don’t shine through in this activity. It was not easy for the students to connect the learning in this activity back to the overarching concepts.
    • Perhaps I could improved this activity by posting a few key wiki pages on those big ideas with links in place branching out into finer detail. Allowing students to populate those branches and work ‘outwards’ would help them connect their work to the bigger picture.
  • Question 4 asked “Do learners make important connections and identify patterns?”
    • While I was hoping that my students would see patterns and connections as they developed the wiki, I was not making this easy for them.
    • Once again building the wiki from the big picture down to the details rather then from the details up would help strngthen the activity in this area.
  • Finally “Do learners apply knowledge to new contexts?”
    • As it stands this activity goes nowhere. Having covered the theory students have nothing to do with it. The final stage is to encourage the use the information in the wiki to address design problems.

So in summary, by applying the 21CLD tool I have been able to find weaknesses in my learning design and from there formulate ways in which I can make it richer and more effective in teaching knowledge construction. In this way the 21CLD tool can be used to significantly improve learning design in all the 21st century skill dimensions. Finally, my digital badge for this task was issued for significant contribution to the wiki by adding and editing pages. In retrospect this was a poorly designed badge as it doesn’t credential the aim of the learning activity. I will need to think more about the design of that badge and develop one which credentials knowledge design, or more generally innovation.    

Gamification? Hmmm let me think about that…

There has been a lot in the blogosphere lately about gamification. In the context of this post it involves using techniques common to computer games to promote engagement in school classrooms. This is distinct from game-based learning, which uses computer games as the medium to deliver the curriculum.

Gamification is defined as:

Gamification is the concept of applying game mechanics and game design techniques to engage and motivate people to achieve their goals.

So how do we gamify our classroom learning. The web site list the following methods for gamifying activities:

  • Add points to tasks that need to be completed
  • Define badges/rewards to be given out after a criteria is met
  • Create a Leaderboard to show top performers
  • Define levels to repeat tasks or to perform harder tasks
  • Earning of badges can be tied to unlocking higher levels

In a modern classroom we usually try to avoid an excessively competitive environment. As a result the practical application of gamification in a classroom generally involves establishing an ecology of digital badges which students can work towards. These badges are connected to learned competencies and students can progress through them and often combine them to achieve ‘higher’ badges. The badges can be collected and shared.

This presents a picture of students engaged in their learning as they collect badges and show them proudly to their peers and family. Activity doesn’t necessarily equate to productivity, however, and I can see some potential dangers in this.

The first issue is touched on in the last words of the definition above. “…motivate people to achieve their goals.” I love learning. I am 56 years old (shhhh…) and I still get joy out of learning new things. I wish my students were the same. When I am speaking to a disengaged student about their progress I often ask,

“What is your aim in maths class? What are you here to achieve?”

Typically the answer is,

“I want to pass maths.”

“Wrong”, I say, “if your aim is to pass then no wonder you are bored and struggling. Your aim should be to improve your maths, then if you pass or fail you are a winner and you will likely pass anyway.”

Unfortunately our system has made collectors out of many students. They collect subjects and if they can collect them easily with a minimum of learning then that is fine. If we gamify the classroom a student with that attitude will simply start collecting badges. They may collect a lot of them, but they will do this by completing badges quickly and moving on without necessarily retaining much along the way.

Now to my second concern. During my teacher training I learned about “token economies”. These are Pavlovian systems where students are rewarded for a desired behaviour with a token or reward. In this way the desired behaviour becomes more frequent. The problem with this is that sooner or later the rewards have to stop and if the underlying reason for the undesireable behaviour isn’t addressed, the student will revert back again.

Looking at gamification with badges, we need to ask ourselves what happens when the student stops getting badges, or the law of diminishing returns nullifies the reward? Will they just stop learning? In  other words the gamification has helped the student through a few years of education, but it has not addressed their underlying attitude to learning and they have not become independent learners in the sense of 21st century skills.

In conclusion, I don’t want you to think I am against digital badging or gamification. I just think that we need to be very careful how we implement it. Gamification is effective only when the game is learning, if the game becomes badge collecting then the badges become a distraction from deep learning. How do we ensure this? Well I did say I needed to think about it.

What are 21st Century skills?

In previous posts I have side-stepped giving an exact definition of 21st Century skills. To be fair on myself, there is some debate about what they are and definitions like this are difficult to pin down. It is a bit like trying to define a dog, they are all different, but you just know one when you see one.

For my purposes it is important to find a definition which is broad enough to include current thinking from a range of sources, but clear enough to be useful.

One definition which is gaining support at the moment is provided by ATC21S, and Microsoft have also produced a list of skill areas as a basis for their 21st Century Learning Design initiative (previously called Leap21).


21CLD lists 6 dimensions of 21st century learning.

  • Collaboration

Working in collaboration with others and in teams.

  • Knowledge construction

The ability to go beyond what has been learned to generate ideas.

  • Self regulation

Taking responsibility for their life and their on-going learning

  • Real-world problem-solving and innovation

Adaptability, creativity and balancing requirements and constraints to solve problems.

  • Use of ICT for learning

The ability to leverage ICT to enhance life and learning.

  • Skillful communication

Actively participate in society and learning through enriching communication.

The descriptors are my abreviations and you should refer to the site for a more detailed explanation.

21CLD has provided sample learning activities and a rubric to evaluate classroom activities against these 21st century competencies. This information is available on the web site above, or through the 21st Century Learning Design app (available from the Microsoft appstore).


This project defined the following list of 10 21st century skills.

  • Critical think and problem solving
  • Communication
  • Collaboration
  • Information literacy
  • ICT literacy
  • Citizenship
  • Life and career
  • Personal and social responsibility
  • Creativity and innovation
  • Learning to learn

The ATC21S group is looking at how these skills can be taught and assessed. For this purpose they are combining the skills into broader areas. To date they have done extensive work on teaching and assessing collaborative problem solving, which is a combination of the first three skills in their list.


Looking at the two lists above it is obvious that they cover the same broad skill set, only differing in the way in which they divide up the skills. This is true of all the major attempts to define 21st century skills.

The ATC21S list of 10 skills is more granular than it needs to be. The skills are so inter-related that attempting to assess them all in isolation would be difficult, complex and ultimately unnecessary. For instance how would we separate personal and social responsibility from citizenship, and do we really need to in developing a learning activity? This has been recognized by the ATC21S group and they have incorporated their skills into sets of related skills, for instance critical thinking/problem solving, communication and collaboration, as inter-related skills, have been placed under the umbrella collaborative problem solving.

Looking at the lists of skills and extending from the groups of skills used by ATC21S, I can see 4 broad areas which could be credentialed as a series of badges:

Collaboration skills
  • critical thinking/problem solving (ATC21S)
  • communication (ATC21S) (21CLD)
  • collaboration (ATC21S) (21CLD)
  • real-world problem-solving and innovation (21CLD)
Information skills
  • information literacy (ATC21S)
  • ICT literacy (ATC21S)
  • use of ICT for learning (21CLD)
Community skills
  • personal and social responsibility (ATC21S)
  • life and career (ATC21S)
  • citizenship (ATC21S)
  • self regulation (21CLD)
Innovation skills
  • creativity and innovation (ATC21S)
  • learning to learn (ATC21S)
  • knowledge construction (21CLD)

As a practicing teacher I am excited by the prospect of using the work of the ATC21S group to move on in teaching and assessing 21st century skills, with the 21CLD resources to assist with designing rich learning activities to support that learning. All this could be overlaid with a robust and criterion referenced digital badge ecology, so that students can earn credentials under these 4 broad areas, and take them out into the world.

Digital badges for collaborative problem solving

In my last post I suggested that although open badges are a powerful credential and those working with them are making excellent use of them, badges will struggle to find traction as a replacement for existing reporting systems.

I went on to suggest that badges were ideally suited to credential skills currently classified as 21st century skills. In this way open badges can form a parallel system to the reporting systems currently in use in most jurisdictions. While an assessment for maths provides evidence on the level of numeracy a student has reached, a badge would provide evidence of  global citizenship, research skills or collaborative problem solving. Arguably, these 21st century skills are rarer and say more about a graduate than a traditional school report.

In order to receive broad acceptance, however, the badges need to be based on an objective measure of the skill it is awarded for and in the case of 21st century skills this is difficult. Fortunately work is being done on the assessment of 21st century skills by the ATC21S project, headquartered at Melbourne University. In March of this year ATC21S released an empirical progression for collaborative problem solving, which can be the basis of an assessment rubric.

Based on the collaborative problem solving empirical progression I have developed a series of open badges, from basic (blue) through bronze, silver and gold, to recognize the level of development a student has achieved. These badges provide evidence of a student’s skill level and also identify a zone of proximal development for the student. For instance a student at the bronze level can see from the  silver level criteria what their area of development is.

Each of these levels is supported by more detailed descriptors in the ATC21S documents, but I believe the badges have more utility if a simplified version is attached to the badge, while still maintaining clear developmental stages. An assessor would refer to the more detailed descriptors when awarding a badge. In the empirical progression the basic badge corresponds to level C, the bronze badge level D, the silver badge is level E and the gold badge is at level F.

Collaborative problem solving badges

Basic badge


When solving a non-trivial problem in a collaborative context the recipient

  • Recognizes the role of others in solving problem
  • Shares resources
  • Communicates strategies to achieve a common understanding of the problem

Bronze badge


When solving a non-trivial problem in a collaborative context the recipient

  • Shows perseverance and commitment to solving the problem together with peers
  • Approaches the problem systematically, setting goals and evaluating different strategies
  • Can make connections between different pieces of information
  • Is aware of the performance of their peers, and can see their own performance objectively

Silver badge


When solving a non-trivial problem in a collaborative context the recipient

  • Acts with planning and purpose, drawing on prior knowledge and experience
  • Can adapt and change with new information
  • Initiates interactions and responds to contributions from peers but may not resolve differences or change plans

Gold badge


When solving a non-trivial problem in a collaborative context the recipient

  • Assumes group responsibility for the task
  • Works through the problem efficiently using only relevant resources
  • Tailors communication and incorporates input from peers, changing plans and resolving conflict as necessary
  • Can reorganize the problem in an attempt to find a new solution path

There still needs to be some thought put into refining these badges and techniques for assessment must be developed, but I believe that there is enormous potential for open badges in this area. As the work of developing empirical progressions for other 21st century skill areas continues, this badge system can be expanded.

Digital Badges and 21st Century Skills

Open badges

Open badges offer a method of credentialing specific skills in a number of contexts. They can be used to support professional learning, they can be used to evidence skills developed in extracurricular activities (such as holiday activities) and finally they can be used to reward classroom achievement. In the educational system I work in, however, assessment is already criterion based. It is, therefore, already possible to lay a school report alongside the subject curriculum documents to identify the level of competence a student has achieved for a particular skill. In this situation any curriculum-based badges will, to some extent, be doubling up on the existing assessment. I still issue badges for work within the curriculum and I still see value in it, but many would argue convincingly that these badges represent an unnecessary doubling up of credentials.

On the other hand curriculum documents tend to be relatively narrow and subject specific. Students in the classroom can be awarded badges for co-curricular achievement, in the sense that they credential valuable skills which are not part of the formal curriculum, but underpin the curriculum. Some might refer to these as ‘meta-skills’, such as collaboration, research and time management etc.

The problem with these co-curricular badges is that they can easily lose their value if the criteria are not well thought out. In addition, badges issued through different sources will not necessarily be comparable. For example if I issue a badge to a student for “Internet search skills” the student and I will know what that involves, but will a third party understand it the same way? Finally how does my badge compare with a simular one from the school down the road?

21st Century skills

I would like to move on from badges for a while and consider 21st century skills. The term 21st century skills is used to refer to a collection of skills which will be of particular use in a post-industrial world, where routine skills and factual recall is of less value. There is as yet no definitive list of 21st century skills, but there is a consensus that they include such things as collaboration, digital literacy, problem solving and global citizenship.

These skills are universally valued by educators, but in general they are not well served by school curriculums and assessment methods. In addition, there is currently a political agenda pushing for increased accountability for schools and a need to measure success. This is evident in the common core in the USA and NAPLAN in Australia. While accountability is important, the danger is that this emphasis on numeric measures of success, comparing students and schools, can emphasise those elements of the curriculum which are most easily quantified. In other words test marks can overshadow learning processes, so collaboration, creativity and ethics (21st century skills) end up being neglected.

Bringing this all together

In summary we have a set of 21st century skills which are universally valued, but are in danger of being neglected because they are not easily quantified. We also have a credentialing system in open badges, which recognizes competency without seeking to make comparisons. I suggest we bring these two ideas together. Instead of seeking to replace traditional school assessments with digital badges, badges provide a parallel system to provide recognition for those skills which are ultimately more important, but not easily measured in a traditional testing regime. 21st century skills can be recognized and encouraged through the badges, while the existing assessment of the subject curriculum, NAPLAN and common core testing continue to serve  their purpose, resulting in a balanced and effective education system.

The issue that remains to be addressed is the assessment of 21st century skills. I commented above that for badges to be effective they need to have well defined competencies. Fortunately there is work under way to provide an objective measure of competency for 21st century skills. A leader in this field is the ATC21S project headquartered at Melbourne University. Their particular emphasis at this stage is on the key 21st century skills of collaborative problem solving and learning in digital networks.

ATC21S have developed an empirical progression to define levels of competence at the developmental stages in collaborative problem solving. It is now possible to use this empirical progression to develop a series of digital badges recognizing competence in collaborative problem solving across the developmental stages.

So we have the tools, in the empirical progression and open badges, to begin to develop a robust credentialing system for 21st century skills. The next task is to work on defining the open badges for the existing empirical progression and as the work on teaching and assessing 21st century skills progresses, develop a badge ecology along side to assess and reward those skills.

Teaching and assessing collaboration skills using Etherpad

I am currently coming to the end of a MOOC offered through called Assessment and Teaching 21st Century Skills. I can recommend it to anyone interested in this field.

The main focus is on teaching and assessing collaborative problem solving as a key 21st century skill.

The ability to collaborate effectively has always been valuable, but in the 21st century it has become vital. The problems we are asked to deal with are frequently multidisciplinary and in an age of specialization it is necessary to bring a number of specialists together to solve most complex problems. For diverse groups to work together in this way high level collaborative skills are required.

When employers are asked what they want in their employees they typically ask for the initiative and the ability to collaborate to solve problems.

The question for teachers and educators is: How do we teach these skills and how do we assess them?

In this post I will propose a style of activity, using an online tool called Etherpad, for educators to use in teaching and assessing these skills.

What is collaborative problem solving?

The materials in the MOOC define collaborative problem solving (CPS) as:

Working together to solve a common challenge, which involves the contribution and exchange of ideas, knowledge or resources to achieve the goal.

The first 2 weeks of the MOOC were devoted to fully developing this concept, so it is not convenient to fully expand on CPS in this short post. Broadly speaking, however, a CPS scenario involves:

  1. A problem with an element of ambiguity in it.
  2. No individual participant has the resources necessary to complete the task and completion of the task is dependent on all participants.
  3. There are a number of possible paths to a solution and often a number of solutions.

In schools most teachers are familiar with group work and often associate this with CPS. Group work differs, however, in that each member of the group typically has access to the same resources, making it possible for each member of the group to work relatively independently on their assigned task(s) in solving the problem. In a true CPS scenario it is not possible for an effective solution without contributions from each participant throughout the process.

Assessing collaborative problem solving

Empirical progressions

Before we can teach CPS we need to know how to measure it.

ATC21S breaks down CPS into cognitive and social skills. These are further broken down into 5 strands, each strand breaking down into a number of skill elements. This results in a extremely granular instrument for evaluating CPS skills. Probably too detailed for teachers to use in a class of 30 students, or even 20, without extensive training and practice. I suggest that teachers get started by using a simple one dimensional empirical progression which ATC21S provide as an overview. This merges all the dimensions and strands into 6 levels of increasing proficiency. Teachers can start with this simpler rubric and progress on to the more complex and accurate tools as they become more familiar with CPS.

Zone of proximal development

The zone of proximal development (ZPD) represents the level in the empirical progression where a student can operate with assistance. It is in the ZPD where effective learning takes place. The teacher needs to evaluate the CPS skills of students and then intervene at their ZPD to help them progress.

For example suppose a group of students have a background in group work, so they are proficient at breaking up a project into sub-tasks and allocating these tasks to group members. The teacher might notice, however, that they tend to get on with their individual tasks and only refer back to the group when they are stuck. This corresponds to level B on the one dimensional progression. This indicates that the ZPD is level C, as they are capable of working together when they need to, but it is not something they do readily without prompting. In this example the teacher should intervene to increase their awareness of their partner and proactively share resources.

Teaching collaborative problem solving

In the MOOC reference is made to a series of tasks developed for this purpose. These tasks involve students solving sample problems collaboratively online. These are powerful tasks, but there is a finite number of them and it would not be easy for a classroom teacher to develop more of them. I wanted to be able to quickly develop my own tasks related to specific curriculum areas. I also wanted to be able to both assess and teach CPS using these tasks. With that in mind I turned to a free package called Etherpad.


Etherpad provides a web interface with a text area that participating students can work in, editing their own and their partner’s work. There is also a chat window which allows for direct communication. The entries by each student are colour coded so that each student’s contribution is clear. Finally there is a time slider which allows movement back and forward through the pad to see how it developed through the activity.

The Etherpad instance I use is at . Registration is free and once registered pads can be created as needed. It is possible to make the pads public (guest access and editing), private or password protected. The user interface looks like this:


Sample CPS activity

The ZPD in this example is level C or D in the one dimensional progression mentioned above. This is typical for high school students who are experienced with group work and are able to effectively assign tasks and complete projects. So in completing the activity students will need to be encouraged to be more aware of their partner(s) and share resources to complete the task.

In this activity a pair of students will collaboratively investigate the issues around abortion. Their task is to work together to develop a joint response to some questions. Each student will have a document briefly covering 5 arguments for and 5 against abortion. The two documents are different, so that each student is seeing 10 different arguments, but they are not told that. The students are separated and can only communicate through the computer system. (Click here to download the documents they are given.) The interface they see and the questions they need to answer are shown above.

On completion of the activity the teacher can study the chat session to assess:

  • What level of awareness do they have for each other?
    • Are they discussing/suggesting before adding to the text area?
    • At what point, if at all, do they realize that they have different documents?
    • Do they seek permission before editing the partner’s contribution?
  • Is there evidence of compromise?
    • Acceptance of the partner’s point of view.
    • Willingness to concede a point.
    • Willingness to allow the partner to edit their contribution.

By looking at the text area the teacher can assess:

  • How equal is the contribution by each?
    • Are the colours of the entries well balanced, or is one colour dominant?

By playing through the time slider the teacher can:

  • Evaluate how efficiently they move towards the conclusion of the task.
  • Identify which student is initiating and which is responding, or are they equal?

These are all indicators that the participants are beginning to show awareness of each other and cooperative planning, which is the ZPD for this group.

Once the teacher has been through the evaluation it can be discussed with the students. They can look through the time slider together to review the activity and discuss how they might have approached things differently at key points in the process. Discussion would focus around the ZPD for the students, making explicit the approaches they could have taken which would be more collaborative. These points can then be taken into the next cycle of activities.


  • Collaborative problem solving is a valuable skill, but as a process it is difficult to assess effectively from the end product. The use of Etherpad allows the teacher to effectively capture the process the students go through for later analysis.
  • These sort of activities are easy to set up and run in a classroom.
  • They represent a realistic CPS scenario, including the three components mentioned above.
  • The scenarios can be as complex or simple as needed, to fit the skills and maturity of the students.
  • Students who don’t normally feel comfortable contributing to class discussion are empowered through these sort of activities.
  • The activity can be subject specific and is not an intrusion into the curriculum.

If you have any suggestions or experiences of your own to bring to this topic feel free to comment.