9 trends in compliance learning, training and assessment

John Kleeman HeadshotThis version is a re-post of a popular blog by John Kleeman

Where is the world of compliance training, learning and assessment going?

I’ve collaborated recently with two SAP experts, Thomas Jenewein of SAP and Simone Buchwald of EPI-USE, to write a white paper on “How to do it right – Learning, Training and Assessments in Regulatory Compliance[Free with registration]. In it, we suggested 9 key trends in the area. Here is a summary of the trends we see:

1. Increasing interest in predictive or forward-looking measures

Many compliance measures (for example, results of internal audits or training completion rates) are backwards looking. They tell you what happened in the past but don’t tell you about the problems to come. Companies can see clearly what is in their rear-view mirror, but the picture ahead of them is rainy and unclear. There are a lot of ways to use learning and assessment data to predict and look forward, and this is a key way to add business value.

2. Monitoring employee compliance with policies

A recent survey of chief compliance officers suggested that their biggest operational issue is monitoring employee compliance with policies, with over half of organizations raising this as a concern. An increasing focus for many companies is going to be how they can use training and assessments to check understanding of policies and to monitor compliance.

3. Increasing use of observational assessments

Picture of observational assessment on smartphoneWe expect growing use of observational assessments to help confirm that employees are following policies and procedures and to help assess practical skills. Readers of this blog will no doubt be familiar with the concept. If not, see Observational Assessments—why and how.

4. Compliance training conducted on mobile devices

The world is moving to mobile devices and this of course includes compliance training and assessment.

5. Informal learning

You would be surprised not to see informal learning in our list of trends. Increasingly we are all understanding that formal learning is the tip of the iceberg and that most learning is informal and often on the job.

6. Learning in the extended enterprise

Organizations are becoming more interlinked, and another important trend is the expansion of learning to the extended enterprise, such as contractors or partners. Whether for data security, product knowledge, anti-bribery or a host of other regulatory compliance reasons, it’s becoming crucial to be able to deliver learning and to assess not only your employees but those of other organizations who work closely with you.

7. Cloud

There is a steady movement towards the cloud and SaaS for compliance learning, training, and assessment – with the huge advantage of delegating all of the IT to an outside party being the strongest compelling factor.  Especially for compliance functions, the cloud offers a very flexible way to manage learning and assessment without requiring complex integrations or alignments with a company’s training departments or related functions.

8. Changing workforce needs

The workforce is constantly changing, and many “digital natives” are now joining organizations. To meet the needs of such workers, we’re increasingly seeing “gamification” in compliance training to help motivate and connect with employees. And the entire workforce is now accustomed to seeing high-quality user interfaces in consumer Web sites and expects the same in their corporate systems.

9. Big Data

E-learning and assessments are a unique way of touching all your employees. There is huge potential in using analytics based on learning and assessment data. We have the potential to combine Big Data available from valid and reliable learning assessments with data from finance, sales, and HR sources.  See for example the illustration below from SAP BusinessObjects showing assessment data graphed against performance data as an illustration of what can be done.

data exported using OData from Questionmark into SAP BusinessObjects

For information on these trends, see the white paper written with SAP and EPI-USE: “How to do it right – Learning, Training and Assessments in Regulatory Compliance”, available free to download with registration.

If you have other suggestions for trends, feel free to contribute them below.

Measuring the Effectiveness of Social and Informal Learning

Posted by Julie Delazyn

How you can use assessments to measure the effectiveness of informal learning?  If people are learning at different times, in different ways and without structure, how do you know it’s happening? And how can you justify investment in social and informal learning initiatives?

The 70+20+10 model of learning – which explains that we learn 70% on-the-job, 20% from others and 10% from formal study – brings out the importance of informal learning initiatives. But the effectiveness of such initiatives needs to be measured, and there needs to be proof that people are performing better as a result of their participation in social and informal learning.

This SlideShare presentation:  Measuring the Impact of Social and Informal Learning, explains various approaches to testing and measuring learning for a new generation of students and workers.  We hope you will use it to gather some new ideas about how to answer these important questions about learning:  Did they like it? Did they learn it? Are they doing it?

 

 

Should we formalize informal learning?

Doug Peterson HeadshotPosted By Doug Peterson

Charles Jennings a champion of the 70:20:10 learning model. He explains it in this video: 70% of learning takes place through on-the-job experience, 20% of learning is through informal relationships such as mentoring and conversations with co-workers, and only 10% of learning takes place in formal settings like classrooms and eLearning courses.

I recently read an article in Learning Solutions Magazine entitled Stop Trying to Formalize Informal Learning! The author, Stephanie Ivec, makes some good points about what I would call “organic learning” (she emphasizes unofficial, unscheduled, and impromptu) and how informal learning can possibly be negatively impacted by trying to turn it into a formal process. Ms. Ivec’s position seems to be one of “let formal be formal, and let informal be informal,” which is something with which I agree – in general.

The article got me thinking about one of my favorite sayings: “Everything in Balance.” The article made me ponder what might be the right balance between formal and informal learning.

Should we try to take every informal learning experience, codify it, and teach it in a classroom or e:Learning course? Absolutely not. As a matter of fact, it would be impossible to do so in some circumstances: There are some things you are only going to learn and get better at by doing them on the job.

So then we should just let formal be formal, and informal be informal, and never the twain shall meet, right? Well, that’s where I think we need to find balance.

Consider Henry, the sales person who has been on the job for 30 years. He’s the top sales person in the company, and for good reason: He knows the company’s products like the back of his hand, he understands the company’s customer, and he has great people skills. Henry is more than willing to work with the other members of the sales team, and he provides great advice and insights to anyone who asks. He’s even taken some of the new hires under his wing for their first couple of months to “teach them the ropes.” There is some SERIOUS informal learning going on, know what I mean?

But Henry will retire in a couple of years, and he’s taking all of that (informal) knowledge with him. Somehow, that knowledge needs to be preserved within the company. This is where a bit of formalizing would be appropriate. What if Henry were asked to write a series of blog articles on his sales techniques, what he looks for in customers, how he customizes his pitch – and then new hires were required to read those articles (and take a brief knowledge check) as part of their onboarding?

Or maybe the training department could record a series of interviews with Henry, asking the right questions and capturing the right knowledge. The results could be made available in a podcast format. This means that rather than knowledge transfer taking place randomly (a junior sales person happens to be in the break room with Henry and has the courage to ask the venerable gentleman the right question), we can take some of that informal learning and add just enough of a formal wrapper to it so that we can make sure everyone benefits. We can track the learning as well as assess it.

Another concern I have about informal learning is that we don’t always know what’s being taught. Let’s say I work in a food processing plant, and at the end of each day we go through a cleaning and sanitizing process. If I mess it up, a lot of people could end up very sick. Human nature being what it is, people tend to look for shortcuts and simple ways to do things. So proper cleaning and sanitizing training is not necessarily something that should be left to on-the-job training (70%) or learning from a co-worker (20%).

Suppose my co-worker has a shortcut that appears benign and even saves time. He believes it works just as well as the proper procedures … but it doesn’t, and it’s not something that should be propagated through the workforce and to new hires. In this situation, the company might want to use observational assessments and Job Task Analysis surveys to understand what is really happening “out on the floor.” Then Formal training could incorporate the good practices and eliminate the poor ones. An alternative might be to run high-performing employees through formal training, certify their knowledge, and then designate them as “official informal trainers” back on the job: a formal/informal training hybrid.

I think there will always be, and should always be, formal, semi-formal, and completely informal learning taking place. Training organizations should support all of these, and they can help serve the company’s goals by keeping things in balance: determining what informal training should be formalized or at least captured in a formalized wrapper as a valuable learning resource.

Podcast: Charles Jennings on measuring informal and workplace learning

Posted by Joan Phaup

I’m eagerly looking forward to the keynote presentation Charles Jennings will deliver at the Questionmark 2013 Users Conference  on The Challenge of Measuring Informal and Workplace Learning.

Charles Jennings

Charles is one of the world’s leading thinkers and practitioners in learning and development — currently head of Duntroon Associates and previously Chief Learning Officer for Reuters and Thomson Reuters.He will be talking at the conference about how the 70:20:10 learning framework — based on studies that show high performers learn approximately 70% from experience, 20% from others and 10% from formal study – is being adopted by many organizations around the world.

The keynote will address how this framework serves as a strategy for extending development beyond formal, structured learning to include informal and experiential learning opportunities.

I spoke with Charles recently and asked him for some details about his presentation. For example:

  • How would you describe the 70:20:10 framework?
  • What are the key challenges of measurement and evaluation within that framework?
  • How will your conference presentation address those kinds of challenge?
  • What advice would you give to organizations that want to use online assessments to measure the effectiveness of informal and experiential learning?

If you’d like to find out how he answered, listen to this podcast or read the transcript. There will be much, much more, of course, in his keynote address and in the conference program, which we are busy planning right now.

Early-bird registration savings are available through November 16 — so keep an eye on the conference website and be sure to register soon! We’ll look forward to seeing you in Baltimore, Maryland, March  3 – 6 at this terrific learning and networking event.

 

 

Feeding back from eAssessment Scotland

 Posted by Steve Lay

eAssessment Scotland is an annual event hosted by the University of Dundee in Scotland.

This year’s conference had a very clear theme: Feeding Back, Forming the Future. I have to say that the programme was managed very well to fit with this theme and that the theme also fits well with the current mood of the wider community. For example, in the UK as a whole the JISC have an ongoing programme on assessment and feedback, and this event provided an opportunity for some of those projects to report on their progress.

I do find that ”feedback’ can be a very general term. In the opening keynote, Professor David Boud, University of Technology Sydney provided an analysis of the subject through a 3-generation model of feedback. At one point he encouraged us to “position feedback as part of learning and not as an adjunct to assessment”.

I sensed that assessment was being used in an Assessment of  Learning sense here. This contrasts with “Assessment for Learning”, these phrases are simpler ways of expressing the basic idea behind summative and formative assessment respectively. It is the latter which generates the type of feedback that could potentially meet the challenge posed by Dr Steve Draper, University of Glasgow: What If Feedback Only Counted When it Changed the Learner?

From the tone of the discussion at the conference, I do sense that the higher-education community is trying hard to adapt to the new perceptions of formal, informal and experiential learning reflected in the 70:20:10 model of education and development — by continuing to embrace the value of formal learning while adopting other modes of learning.

The 10% is sometimes summarised as being the part of our learning effected by formal courses (and reading). Feedback is reserved for the 20% where we learn from our peers. Many of the presentations were about embracing social systems to attempt to exploit these modes of feedback.

Clearly, assessment can have an important role to play in assessment for learning but I took away the impression that this community sometimes needs reminding that understanding the purpose of an assessment is vital to its success. Combining assessment for learning and assessment of  learning may not be fruitful.

Charles Jennings on Measuring Informal and Workplace Learning: Questionmark 2013 Users Conference Keynote

Posted by Joan Phaup

We are delighted to announce that  Charles Jennings, one of the world’s leading thinkers and practitioners in learning and development, will deliver the keynote address at the Questionmark 2013 Users Conference, which will take place in Baltimore, Maryland, March 3 – 6.

His presentation,  “Meeting the Challenge of Measuring Informal and Workplace Learning,” will focus on the widespread adoption of the 70:20:10 framework —  based on studies that show  high performers learn approximately 70% from experience, 20% from others and 10% from formal study.

Charles Jennings

Charles will show how the 70:20:10 framework serves as a strategy for extending development beyond formal, structured learning to include informal and experiential learning opportunities. He will pose some important questions about which approaches support informal and workplace learning most effectively and how to effectively measure the success of those approaches.

As head of Duntroon Associates, Charles helps clients with learning and performance strategy, change management and with implementing improved approaches to workforce and leadership development. Previously, he served as Chief Learning Officer for Reuters and Thomson Reuters, where he led a team of 350 learning professionals for the firm’s workforce of 55,000.

This will be our 11th annual North American conference and, incidentally, a celebration of Questionmark’s 25th anniversary! We are busy planning the conference program and are welcoming case study and peer discussion proposals from experienced Questionmark users.

If you have a success story to tell or would like to get together with your colleagues to discuss a topic that concerns you, please check out our call for proposals and send in your ideas by September 14. Even if you are not yet sure you’ll be at the conference, that’s okay. We’d still like to hear from you!