International University gains scalability, advanced authoring, single sign-on and more!

ChloePosted by Chloe Mendonca

Earlier this month I had the pleasure of speaking with The International University of Languages and Media (IULM) based out of Milan, Italy. IULM has been using Questionmark’s Assessment Management System for almost 10 years, so I was also keen to see what prompted their recent move from managing their assessment program on-premise to Questionmark OnDemand (Questionmark’s SaaS cloud-based solution).iulm

IULM serves approximately 5,000 students enrolled across a range of postgraduate and undergraduate courses including interpretation, translation and cultural studies; communication, public relations and advertising, and arts and tourism.

Patrizia Lettieri, the digital platform administrator in the university’s IT department explains why after 10 years, the IULM decided to make the move:  “Our teachers liked using Questionmark; we didn’t want to change that, but there was definitely a need for a more scalable and innovative solution that would match the direction the university was headed in terms of growth. Security and reporting were also very important to us and there is nothing on par with Questionmark OnDemand in regards to security and reporting.”

Here are some highlights from our conversation. You can read the entire case study here.

Advanced authoring
A big draw for IULM in using Questionmark was the ease of use when it came to assessment authoring and improving the way items are written and also the efficiency with which they can manage their item bank. Patrizia elaborates: “The variety of item types and ability to embed images, videos and links into our questions and feedback enable us to further develop the way we are helping our students to learn. Meta tagging of items is also very simple. Before using OnDemand we had to separate these into different folders and our teachers used to delete items that were invalid. Now we can easily assign the appropriate meta tags and modify the status of questions to retired if they are invalid.”

Scalability and save-as-you-go
iulm_campusBefore using Questionmark OnDemand IULM had a few challenges when it came to assessing large numbers of students.  With OnDemand, IULM can reliably deliver their assessments to multiple classes of students. And in the event that something does go wrong — such as a power cut or other technical failure — Questionmark’s save-as-you-go is there to ensure student answers are automatically saved at regular intervals. “My priority is the students; they are our customers and I want them to be comfortable,” adds Patrizia. “With OnDemand everything runs smoothly and our students are more comfortable and confident when taking their assessments.”

Single Sign-On
It used to be that every day, staff and students at the university were having to interact with a variety of applications requiring individuals to remember different credentials for the various applications.  “With Questionmark OnDemand, we were able to take advantage of single sign-on (SSO). For a forward-thinking technological university, improving access for both students and staff just made sense for us.” SSO also saves IULM time on the administrative side. Once students have completed their studies and leave the university, staff no longer have to remove their access to each application but only need to remove each student’s identity provider account. Without that account, the student can no longer log on to any of the linked applications.

Multilingual delivery
Being an international university, having students who speak different languages is a common scenario. Questionmark’s multilingual delivery helps IULM accommodate students whose first language is not Italian or English by enabling them to change the delivery interface to their preferred language. Having the navigation controls in a participant’s own language can often make students feel more at ease during the test taking experience.

If you’re interested in trying out Questionmark OnDemand, request a demo today and one of our team will show you how simple assessment management can be.

Predicting Success at Entel Begins with Trustworthy Assessment Results [Case Study]

Julie ProfilePosted by Julie Delazyn

Entel is one of Chile’s largest telecommunications firms, serving both the consumer and commercial sectors. With more than 12,000 employees across its extended Entelenterprise, Entel provides a broad range of mobile and fixed communications services, IT and call center outsourcing and network infrastructure services.

The Challenge

Success in the highly competitive mobile and telecommunications market takes more than world-class infrastructure, great connectivity, an established dealer network and extensive range of retail location. Achieving optimal on-the-job performance yields a competitive edge in the form of satisfied customers, increased revenues and lower costs. Yet actually accomplishing this objective is no small feat – especially for an industry job role notorious for high turnover rates.

With these challenges in mind, Entel embarked on an innovative new strategy to enhance the predictability of the hiring, onboarding, training and developing practices for its nationwide team of 6,000+ retail store and call center representatives.

Certification as a Predictive Metric

Entel conducted an exhaustive analysis – a “big data” initiative that mapped correlations between dozens of disparate data points mined from various business systems, HR systems as well as assessment results – to develop a comprehensive model of the factors contributing to employee performance.

Working with Questionmark OnDemand enabled Entel to create the valid and reliable tests and exams necessary to measure and document representatives’ knowledge, skills and abilities.

Find out more about Entel’s program planning and development, which helped identify and set benchmarks for required knowledge and skills, optimal behaviors and performance metrics, its use of SAP SuccessFactors to manage and monitor performance against many of the key behavioral aspects of the program, as well as the growing role their trustworthy assessment results are having on future product launches and the business as a whole.

Click here to read the full case study.

The Importance of Safety in the Utilities Industry: A Q&A with PG&E

Headshot Julie

Posted by Julie Delazyn

Wendy Lau is a Psychometrician at Pacific Gas and Electric Company (PG&E). She will be leading a discussion at Questionmark Conference 2016 in Miami, about Safety and the Utilities Industry: Why Assessments Matter.

WendyLau_Q&A

Wendy Lau, Psychometrician, PG&E

Wendy’s session will describe a day in the life of a psychometrician in the utilities industry. It will explore the role assessments play at PG&E, and how Questionmark has helped the company focus on safety and train its employees.

I recently asked her about her session:

Tell me about PG&E and its use of assessments:

PG&E is a utilities company that provides natural gas and electricity to most of the northern two-thirds of California. Over the years, we have evolved into a more data-driven company, and Questionmark has been a part of that for the past 7 years. Having assessments readily available and secured within a platform that we can trust is very important to PG&E. We are also glad to have found a testing tool that offers such a wide variety of question types.

Why is safety important in the utilities industry?

Depending on the activity that our employees perform, most of the work has serious safety implications — whether it is a lineman climbing up a  pole to perform liveline work or a utility worker digging near a major gas pipeline. Our technical training must have safety in mind and, more importantly, it must ensure that after going through training, employees are competent to perform their tasks safely and proficiently. In order to ensure workforce capability, we rely heavily on testing to prove that our workforce is in fact safe and proficient and that the community we serve and our employees are safe and receiving reliable services.

What role does Questionmark play in ensuring that safety?

Questionmark helps us focus on safety-related questions by allowing special assessment strategies such as identifying critical versus coachable assessment items and identifying cutscores for each accordingly. Questionmark also allows a secured platform so that we can ensure our test items are never compromised and that our employees are truly being assessed under fair circumstances.

To find out more about the role of Questionmark plays in ensuring safety, you’ll just have to attend my session at Questionmark Conference 2016 in Miami!

What are you looking forward to at the conference?

I am very much looking forward to ‘talking shop’ with other Psychometricians and sharing best practices with others in the utilities industry and other companies alike!

Thank you, Wendy for taking time out of your busy schedules to discuss your session with us!

palm tree emoji 2If you have not already done so, you still have a chance to attend this important learning event. Click here to register.

 

Performance testing is to certifications as simulation is to learning

Howard Eisenberg HeadshotPost by Howard Eisenberg

I just attended the Performance Testing Council Summit. Performance testing is “testing by doing.”  Exam developers create performance items that require candidates to actually perform real-world, authentic task not multiple-choice questions that have only one best answer or allow a low-ability candidate to guess the correct answer.  The outcome of the task is then evaluated to determine a score, or how well the candidate performed.

All but one of the attendees of this meeting representing certification programs were from software/IT companies. The IT domain lends itself very well to the adoption of performance testing. Advances in virtualization and software-as-a-service make it possible to provision “testing labs” with specific characteristics and traits in minutes and at low costs.  Moreover, as these labs can be hosted in the cloud nowadays, there’s no need for a candidate to travel to a specific location to take an exam.  This means that IT, performance-based certifications can and indeed ARE being delivered online and on-demand, with the help of remote proctoring and other technology-enabled security controls.

sunWe’d like to hear from you! The call for proposals is officially open for the Questionmark 2016 Users Conference. If you have an experience you would like to share with the Questionmark community, please submit a presentation proposal here: click here.

These labs are the real-world context in which an IT professional works.  It’s not a simulation of the software, tools, network connections, etc.: it is the real thing.  As such, it’s arguably a more valid methodology for assessing an IT professional’s ability to perform the tasks required by the job.  So using a performance exam for an IT certification makes sense.

Alas, not all certification exams and the professional domains they represent are as well suited to performance testing. It’s not as easy to recreate the environment in which a registered nurse performs his or her daily duties, for example.  In other domains where technology is not center-stage, Questionmark’s customers have historically done the next best thing.  What’s that, you ask?  Well, it’s simulating the performance environment within the test. And if high-fidelity simulation is not cost-effective to develop, then it’s using real-world exhibits, artifacts, and scenarios expressed through multimedia to bring as much of the performance context/environment to the test as is feasible and cost-effective.

Performance testing is to certifications as simulation is to learning.  It’s that “holy grail.”  If we can make the exam look and feel like the job, then it will have the greatest potential to be the truest measure of ability.  If we can make the training look and feel like the job, then it will have the greatest potential to adequately prepare the employee.  (I say “potential” only because the instrument or the simulation must still be well-designed).

I know that many Questionmark customers have struggled to attain this ideal. That is the reality of working with budgets, timelines and other limited resources.  But I’m willing to bet that many customers have creatively worked around these challenges to create valid tests and exams that provide solid measurement value to the programs in which they are used.

sunIf you have a story to tell about such challenges and solutions, then please share them with the Questionmark community at the Questionmark 2016 Users Conference. Click here to submit your presentation proposal. *Submission deadline is December 4. Slots are limited.

Q&A: Pre-hire, new-hire and ongoing assessments at Canon

HollyGroder

Holly Groder

Headshot JuliePosted by Julie Delazyn

Holly Groder and Mark Antonucci are training developers for Canon Information Technology Services, Inc. (Canon ITS). During their case study presentation at the Questionmark 2015 Users Conference in Napa Valley March 10-13, they will talk about Leveraging Questionmark’s Reports and Analytics Tools for Deeper Insight.

Their session will explore Canon’s use of assessments in hiring, training, continuing job skills assessment and company-wide information gathering via surveys.

I asked them recently about their case study:

Why did you start using Questionmark? 

The primary reason for seeking a new assessment tool was our desire to collect more information from our assessments, quicker. Questionmark offered the flexibility of web-based question creation and built-in reports. Questionmark also offered the ability to add jump blocks and a variety of templates. The survey capabilities were just a bonus for us. We were able to streamline our survey process to one point of contact and eliminate an additional software program.

What kinds of assessments do you use?

MarkAntonucci1

Mark Antonucci

The principal function is split between four business needs: Pre-hire employment assessments, new-hire or cross-training assessments, continuing job knowledge assessments, and business information gathering (surveys).

How are you using those tools?

First, potential employees are required to participate in a technical knowledge assessment prior to an offer of employment. Once employment has been offered and accepted, the new employees are assessed throughout the new-hire training period. Annually, all call center agents participate in a job skills assessment unique to their department. And finally, all employees participate in various surveys ranging from interest in community events to feedback on peer performance.

What are you looking forward to at the conference?

We are interested in best practices, insight into psychometrics, and, most important, networking with other users.

Thank you Holly and Mark for taking time out of your busy schedules to discuss your session with us!

***

If you have not already done so, you still have a chance to attend this important learning event. Click here to register.

Big Themes, Big Deadlines: Napa News

Julie Delazyn HeadshotPosted by Julie Delazyn

We have two big deadlines coming up for the Questionmark 2015 Users Conference in Napa!

All case study and presentation proposals are due on December 10, so submit your proposals soon if you want to lock down a chance to be a speaker at the conference. The perks for you? One 50 percent registration discount per case study and a VIP dinner for all presenters.

Early-bird deadline ends December 17. Register now o save $200. Want to save more? Bring your colleagues and take advantage of our group discounts! There is so much to learn at the conference that many of our customers take on the “divide and conquer” approach by attending different concurrent sessions and comparing notes later.

grape iconThis year, we’ll focus on:

Hackers, attackers and your assessments: Protecting your assessment data

We’ll explore some of the developments and emerging threats to data security and their implications.

Can you trust your assessment results?

The conference will explore what makes assessment results “trustable” any why trustable results matter.

Checking knowledge or checking a box: Assessments and Compliance

Regulatory compliance is a fact of life – one that drives training and the need for trustable, defensible assessment.

grape icon

We look forward to seeing you in Napa—where you will also have a chance to:

  • Get vital info and training on the latest assessment technologies and best practices
  • Network with fellow assessment and learning professional
  • Learn about the Questionmark product roadmap

Oh, and did I mention this will all take place in the heart of the beautiful California wine country? We look forward to learning with you there!