Putting coaching reports front and center

Posted by Joan Phaup

As we approach our first early-bird registration deadline for the Questionmark 2013 Users Conference, I’m checking in with people who will be presenting various sessions during our three days together in Baltimore March 3 – 6.

Abdulquader Kinariwala

It was great to talk the other day with Accenture Certification Manager Abdulquader Kinariwala, who will deliver a case study presentation with his colleague John Kessler, about the use of coaching reports not only to help people succeed in certification exams but also to improve their overall work performance.

Could you briefly describe the role of Questionmark assessments in Accenture’s certification program?

Assessment are very important part of our evaluation mechanism.  When people claim that they are certified – – whether it’s a professional certification or a technology certification  — they must have a credential to prove their qualifications. And to be credentialed, he or she must participate in an assessment.  Assessments are an integral part of our certification process.

I understand you will be zeroing in on the use of coaching reports in your conference presentation. Why are coaching reports important to you?

When we are talking about certifications or learning in general, our assessment framework emphasizes the value of how we coach people to be successful. If we have an employee who is not successful in a certain assessment in a certification program, we share with them the coaching report of their previous performance in an assessments, and we have a discussion about it. With the employee’s permission, we will share the report with his or her career counselor, who will work with the person before they take the assessment again.

Coaching reports are often regarded as a peripheral mechanism for improving performance in an assessment, but we see coaching as a core element of learning.   We use coaching reports not only for failed outcomes but also for partial successes. Someone may have passed an assessment, but there might be a section or two where they could have done better. Our coaching reports provide details about individual topics, so we can pinpoint specific areas where there is room for improvement.

What makes Accenture’s use of coaching reports unique?

We have a customized version of the Questionmark Coaching Report – which we worked on with Questionmark consulting — which allows us to download an Excel extract of the assessment repository. We put that file into one of our custom communication tools which we have created in-house. This tool creates an email that highlights to the person their scores in particular sections and calls out areas where they can improve.

What do you hope people will gain from attending your case study presentation?

We’d like them to get an understanding of how we are using coaching reports,– even for successful candidates — and gain some insights on the custom solution we worked on with Questionmark. We will also have a discussion about how other people are currently using coaching reports, so everyone should have an opportunity to learn from one another.

You’ve attended several Questionmark Users Conference. What do you find most valuable about them?

The ability to network and interact with the Questionmark teams: especially product development, consulting and customer services. I also get a lot out of the keynotes as well as a lot of customer presentations. I like to hear people who have been using Questionmark and consider how to adapt their ideas to our own situation.


Note: The  early-bird Questionmark Users Conference registration discount of $200 ends today!  Check out other conference sessions and click here to sign up.

Questionmark’s first Assessment Innovation Award goes to…

Posted by Joan Phaup

Having admired for many years the creativity and can-do spirit of our customers, we’re pleased to announce the first-ever Questionmark Assessment Innovation Award, which we’ll present at the  Questionmark Users Conference in Los Angeles next month.

For this year’s honor, we’ve chosen Accenture for their work with more than 30 internal certification programs that combine training, experience  and (of course!) assessments – mostly medium- to high- stakes online tests as well as work product evaluations and interviews.

Allison Horn

Over the past three years, Accenture has moved ahead on many fronts to grow their internal assessment development and delivery capabilities to meet the increased demand for knowledge and skill assessments while upholding high quality standards. Moving to a three-tier assessment model, the company  has modified their test development processes so that the amount of time, money and effort invested in each test is determined by its overall stakes. Foundation level programs, for instance, now require significantly less  subject matter expert time, beta testing and item analysis than master level programs. In addition to more efficient development processes, the company is using new tools to automate item and test analysis more than ever before.

On the delivery side, Accenture has begun using testing kiosks for smaller locations and adopting standard models for reporting results. They’ve also enhanced the use of their Questionmark/LMS integration to give certification candidates a more consistent, familiar end-user experience.

Allison Horn,  Senior Manager – Global HR  at Accenture, will accept the award on Accenture’s behalf  in addition to co-presenting a case study on Enabling Self-Service Reporting. Two of her colleagues will be talking at the conference about Accenture’s Tiered Assessment Model for Internal Certification.

You can see the entire list of case study presentations, peer discussions, demos, best practice presentations, product training breakouts and general sessions, by clicking here. And you can register online now!

Conference Close-up: A Tiered Assessment Model for Internal Certification

Posted by Joan Phaup

One of the things I most look forward to at the Questionmark Users Conference is the opportunity to hear customers’ case studies.  Accenture Certification Specialist Neetu Jaiswal spent a few minutes telling me about one of her session about a Tiered Assessment Model for Internal Certification, that she will co-present with Accenture Certifications Analyst Supriya Shetty.

Neetu Jaiswal

Q: Tell me a little about your company.

A:  Accenture is a global management consulting, technology services and outsourcing company. We have about 204,000 employees serving clients in more than 120 countries. Our clients turn to us for industry and business process expertise to help them improve their performance. Our Business Process Outsourcing business involves 33,000 people serving more than 150 clients around the world.

Q: What’s your role in Accenture?

A: As a Certification Specialist, I focus on our certification program for Accenture employees which is organized in three tiers or levels, which build on assessment complexity and rigor. I am in charge of all the Tier 1 or Curriculum Level assessments that we offer in Accenture for various industries groups. I drive all activities associated with curricula and assessment design and development, including job task analysis and/or item writing workshops. I work with solution management and functional architecture teams to develop, deploy and support a cost effective, sustainable solution for the clients. I also oversee the administrative functions including help desk/e-mail support, reporting, and other functions as defined and agreed upon with the client.

Q: How are you using Questionmark?

A: We use almost all the features of Questionmark to support the three-tiered assessment model for Accenture’s internal certification efforts. A few years ago, we started with technical Certification exams in Questionmark and now we use Questionmark tools for technical certifications and other industry assessments, surveys, tests, scoring rubrics, end of program evaluations, etc.   We leverage several Questionmark tools – including Questionmark Live and the Results Management System – to support the development process.

Q: What you will be talking about during your conference presentation?

A:  The demand for employee testing and certification across Accenture has increased dramatically in the past two years.  Accenture’s certification teams had no choice but to look for new, innovative ways to develop tests in less time and with less money than in years past, while still maintaining a sufficient level of quality.

This presentation will describe the three-tiered assessment model used to support Accenture’s internal certification efforts. The three types of assessments are distinguished by the nature of the knowledge and skills being assessed, the types of questions used, the number of forms of each assessment, and the extent of psychometric analysis conducted throughout the development process.   Through the session, we will highlight how several Questionmark tools – including Questionmark Live and the Results Management System – are leveraged to support the development process.

Q: What are you looking forward to at the conference?
A: I want to find out what Perception version 5 offers and how we can use it to our benefit.  Also, I saw that there are many good papers to be presented, so I’m looking forward to attending those sessions. And I want to learn more detail about the psychometric analysis. Questionmark puts on a great conference with many networking opportunities, so I’m looking forward to being with other Perception users to learn about various best practices that they follow. I am also looking forward to meeting the Questionmark people with whom we interact but rarely meet face to face.

Indeed, the conference is a great place to meet face to face with Questionmark staff, industry experts and other Questionmark users. Sign up soon and join us in Los Angeles March 15 -18.