International University gains scalability, advanced authoring, single sign-on and more!

ChloePosted by Chloe Mendonca

Earlier this month I had the pleasure of speaking with The International University of Languages and Media (IULM) based out of Milan, Italy. IULM has been using Questionmark’s Assessment Management System for almost 10 years, so I was also keen to see what prompted their recent move from managing their assessment program on-premise to Questionmark OnDemand (Questionmark’s SaaS cloud-based solution).iulm

IULM serves approximately 5,000 students enrolled across a range of postgraduate and undergraduate courses including interpretation, translation and cultural studies; communication, public relations and advertising, and arts and tourism.

Patrizia Lettieri, the digital platform administrator in the university’s IT department explains why after 10 years, the IULM decided to make the move:  “Our teachers liked using Questionmark; we didn’t want to change that, but there was definitely a need for a more scalable and innovative solution that would match the direction the university was headed in terms of growth. Security and reporting were also very important to us and there is nothing on par with Questionmark OnDemand in regards to security and reporting.”

Here are some highlights from our conversation. You can read the entire case study here.

Advanced authoring
A big draw for IULM in using Questionmark was the ease of use when it came to assessment authoring and improving the way items are written and also the efficiency with which they can manage their item bank. Patrizia elaborates: “The variety of item types and ability to embed images, videos and links into our questions and feedback enable us to further develop the way we are helping our students to learn. Meta tagging of items is also very simple. Before using OnDemand we had to separate these into different folders and our teachers used to delete items that were invalid. Now we can easily assign the appropriate meta tags and modify the status of questions to retired if they are invalid.”

Scalability and save-as-you-go
iulm_campusBefore using Questionmark OnDemand IULM had a few challenges when it came to assessing large numbers of students.  With OnDemand, IULM can reliably deliver their assessments to multiple classes of students. And in the event that something does go wrong — such as a power cut or other technical failure — Questionmark’s save-as-you-go is there to ensure student answers are automatically saved at regular intervals. “My priority is the students; they are our customers and I want them to be comfortable,” adds Patrizia. “With OnDemand everything runs smoothly and our students are more comfortable and confident when taking their assessments.”

Single Sign-On
It used to be that every day, staff and students at the university were having to interact with a variety of applications requiring individuals to remember different credentials for the various applications.  “With Questionmark OnDemand, we were able to take advantage of single sign-on (SSO). For a forward-thinking technological university, improving access for both students and staff just made sense for us.” SSO also saves IULM time on the administrative side. Once students have completed their studies and leave the university, staff no longer have to remove their access to each application but only need to remove each student’s identity provider account. Without that account, the student can no longer log on to any of the linked applications.

Multilingual delivery
Being an international university, having students who speak different languages is a common scenario. Questionmark’s multilingual delivery helps IULM accommodate students whose first language is not Italian or English by enabling them to change the delivery interface to their preferred language. Having the navigation controls in a participant’s own language can often make students feel more at ease during the test taking experience.

If you’re interested in trying out Questionmark OnDemand, request a demo today and one of our team will show you how simple assessment management can be.

Security, Reporting and Online Testing in Academia: A Q&A with Don Kassner

Julie ProfileI recently spoke with Don Kassner, who has joined Questionmark as vice president of academic markets. I wanted to take a moment to welcome Don and to ask him a few questions about his extensive background, his new job role and his hopes for the future. Here’s a snippet of our conversation:

Don Kassner, VP of academic markets, Questionmark

You have had an extensive career. What is your background, and how has it influenced the insight you have on the learning and assessment world?

At heart, I am an entrepreneur.  I started my first business when I was 8 years old and have been starting and building things ever since. In the last 15 years or so I have focused on technology, training and education. During that time, I have served as the CFO for an auto dealership training company and as president of a small university in addition to founding and building the largest online proctoring company. I have also served on the faculty of San Jose State University (economics) and have been an active member of multiple onsite accreditation evaluation teams.

Now that you are part of the Questionmark team, can you share a little about your new role and the goals you have for this position?

I am excited to reach out to the academic markets and address ongoing concerns around testing.

Colleges and universities want to deliver tests and exams that are consistent, fair, reliable and defensible. They want to deliver and monitor course evaluations, identify knowledge gaps and place students in appropriate courses, and be able to analyze questions to determine which ones are valid and fair. They want to do this while increasing student satisfaction and reducing cheating.

With Questionmark’s extensive security, reporting and analytics tools, that’s all possible.  My focus is to leverage my insight and experience and put this secure and easy-to-use tool in the right hands.

Can you share some of the important topics surrounding the academic markets that are on your radar?

In academia, there are real issues related to online testing. These are the top two:

  • Is the student doing the work? – With technology comes flexibility and as the stakes become higher, students will naturally look for ways to enhance their scores.
  • Are the exam results fair?  Did all the students face the same conditions with the same opportunities and were the exam rules followed?

Questionmark OnDemand, especially its analytics and reporting tools, can provide reports that can uncover the meaning hidden within assessment results, such as an item or assessment’s reliability and defensibility. In regards to security, Questionmark Secure’s lock-down browser plays a huge role in helping organizations provide a secure environment in which to deliver high stakes assessments. This can significantly help reduce the risk of cheating when deployed along with other defenses to combat impersonation and content theft.

We’re very excited to have you on board! What are your plans for keeping in touch with us on The Questionmark Blog?

Once a teacher, always a teacher! I look forward to turning back the clock to my university days and speaking on the hot-button topics that surround academic testing. My experience in education and entrepreneurship will help me as I share my thoughts on current market trends and the future of assessment and testing.

Thank you Don for taking the time to speak to us today!

You can follow Don on Twitter and connect with him on LinkedIn

Online Proctoring: FAQs

John Kleeman HeadshotPosted by John Kleeman

Online proctoring was a hot-button topic at Questionmark’s annual Users Conference. And though we’ve discussed the pros and cons in this blog and even offered an infographic highlighting online versus test-center proctoring, many interesting questions arose during the Ensuring Exam Integrity with Online Proctoring  session I presented with Steve Lay at Questionmark Conference 2016.

I’ve compiled a few of those questions and offered answers to them. For context and additional information, make sure to check out a shortened version of our presentation. If you have any questions you’d like to add to the list, comment below!

What control does the online proctor have on the exam?

With Questionmark solutions, the online proctor can:

  • Converse with the participant
  • Pause and resume the exam
  • Give extra time if needed
  • Terminate the exam

What does an online proctor do if he/she suspects cheating?

Usually the proctor will terminate the exam and file a report to the exam sponsor.

What happens if the exam is interrupted, e.g. by someone coming in to the room?

This depends on your security protocols. Some organizations may decide  to terminate the exam and require another attempt. In some cases, if it seems an honest mistake, the organization may decide that the proctor can use discretion to permit the exam to continue.

Which is more secure, online or face-to-face proctoring?online proctoring

On balance, they are about equally secure.

Unfortunately there has been a lot of corruption with face-to-face proctoring, and online proctoring makes it much harder for participant and proctor to collude as there is no direct contact, and all communication can be logged.

But if the proctors are honest, it is easier to detect cheating aids in a face-to-face environment than via a video link.

What kind of exams is online proctoring good for?

Online proctoring works well for exams where:

  • The stakes are high and so you need the security of a proctor
  • Participants are in many different places, making travel to test centers costly
  • Participants are computer literate – have and know how to use their own PCs
  • Exams take 2-3 hours or less

If your technology or subject area changes frequently, then online proctoring is particularly good because you can easily give more frequent exams, without requiring candidates to travel.

What kind of exams is online proctoring less good for?

Online proctoring is less appropriate for exams where:

  • Exams are long and participants needs breaks
  • Exams where participants are local and it’s easy to get them into one place to take the exam
  • Participants do not have access to their own PC and/or are not computer literate

How do you prepare for online proctoring?

Here are some preparation tasks:

  • Brief and communicate with your participants about online proctoring
  • Define clearly the computer requirements for participants
  • Agree what happens in the event of incidents – e.g. suspected cheating, exam interruptions
  • Agree what ID is acceptable for participants and whether ID information is going to be stored
  • Make a candidate agreement or honor code which sets out what you expect from people to encourage them to take the exam fairly

I hope these Q&A and the linked presentation are interesting. You can find out more about Questionmark’s online proctoring solution here.

How to transform recruitment and hiring with online testing

Julie Profile

Posted by Julie Delazyn

Did you know that 80 percent of employee turnover is a result of poor hiring practices? With turnover costs as high as $20,000 per exiting employee, improving the screening and testing process is a priority for every HR professional.

Online testing is making the hiring process easier and more effective. In this webinar,  “How to transform recruitment and hiring with online testing,” we’ll also share best practices and a case study from Canada Post Corporation.

Glen Budgell_large

Dr. Glen Budgell, PhD and senior strategic HR advisor at Human Resource Systems Group (HRSG)

 Join us on June 14 at 12:00 p.m. EDT. Click here to reserve your spot!

The webinar is designed to give you an up-close look at the latest assessment technologies. To learn more, I spoke to Dr. Glen Budgell, PhD and senior strategic HR advisor at Human Resource Systems Group (HRSG), who will be leading the session. Glen has over thirty years of experience managing the promotion, development and delivery of consulting services in the area of assessment and testing.

Here is a snippet from my conversation with Glen:

How can online testing transform recruitment and hiring?

Online testing has transformed the recruitment and hiring landscape in many ways. The days in which employers were limited to local talent is over: online tests can be administered at the click of a button to job candidates in remote regions, thereby increasing both the diversity of the talent pool and the inclusiveness of the work place. Standardized testing (whether paper-pencil or computer-based) provides job candidates with a sense of fairness in the hiring process.  When you combine this sense of fairness with an innovative online test that is both job relevant and engaging, you really “wow!” the job candidate, and that is one of our goals at HRSG: to wow our clients and their job candidates with innovative employment tests.

Online testing platforms such as Questionmark provide employers and firms like HRSG the tools to efficiently develop and deliver assessments. At HRSG, we use Questionmark to develop employment tests for our clients so that they can identify the right talent faster, and at a lower cost.

What are some best practices that are important to developing a strong assessment process?

Traditional testing best practices are always relevant, and we strongly urge our clients to conduct the required job/task analysis before choosing or building an employment test. In addition, and especially if the test is being built in house, it should be ensured that the test is both job relevant and linked to the abilities/skills identified in the job analysis.

Of course, online / computer-based testing has introduced a myriad of other challenges. Companies must ensure that the testing software they use is secure and candidate data is protected. We rely on the expertise of Questionmark to provide our clients with the required security and data protection features mandated by the ISO and ICT standards. Furthermore, we share with our client’s methods and best practices for cheating prevention and analysis, monitoring the web for leaked test content, and most importantly, providing candidate’s with technical support and accessibility features when required.

What tools will attendees walk away from the webinar with:

We’ll also discuss:

  • When to buy an off the shelf assessment when to build in house
  • How to enhance the candidate experience with online testing
  • How to develop and use innovative item types
  • Importance of candidate perceptions of online testing
  • Best practices for unproctored internet testing
  • Case study which describes best practices with high volume testing

Thank you, Glen, for taking the time to speak to us!

We look forward to seeing you at this complimentary webinar on Tuesday, June 14 at 12:00 p.m. EDT.

Click here to register and save your spot.

Establishing a data-driven assessment strategy – A Q&A with Amazon

Headshot Julie

Posted by Julie Delazyn

Jason Sunseri is a senior Program Manager – Learning Technology at Amazon. He will be leading a discussion at Questionmark Conference 2016 in Miami, about Creating a Global Knowledge and Skills Assessment Program for Amazon Sellers.

Jason Sunseri, Program Manager – Learning Technology, Amazon

Jason’s session will look at how Amazon Seller Support and Questionmark OnDemand have partnered to deliver a world-class solution. Jason will illustrate how Amazon has used the OnDemand platform to deliver a robust, data-driven assessment strategy.

I recently asked him about his session:

Tell me about Amazon and its use of assessments:

Amazon Seller Support engages with the 2.5 million+ global sellers represented on the Amazon platform. Due to rapid global expansion across the platform, the Amazon Seller Support needed to find a technology and assessment partner that could support both its knowledge and skill acquisition assessment strategies.

How does Amazon use data to drive strategy?

Assessments play a huge role at Amazon. We have really evolved into a data-driven culture and we use assessments in surveys and inside curriculum to assess training and performance, and to identify early issues and trends in order to tweak training content and fix errors.

What role does Questionmark play in that strategy?

We rely heavily on reports — Survey Matrix, Job Task Analysis and other report functions — to assess performance. We’re able to leverage the tool by having individual training centers analyze learning and training gaps and pass on those results. It allows us to see how and why a site is succeeding; where that behavior stems from — it’s really cool to see.

What are you looking forward to at the conference?

It’s Miami, so…the weather, for sure! In all seriousness, I look forward to learning about how other Questionmark users utilize the same tools and how their approach varies from ours.

Thank you, Jason for taking time out of your busy schedules to discuss your session with us!

The Importance of Safety in the Utilities Industry: A Q&A with PG&E

Headshot Julie

Posted by Julie Delazyn

Wendy Lau is a Psychometrician at Pacific Gas and Electric Company (PG&E). She will be leading a discussion at Questionmark Conference 2016 in Miami, about Safety and the Utilities Industry: Why Assessments Matter.

WendyLau_Q&A

Wendy Lau, Psychometrician, PG&E

Wendy’s session will describe a day in the life of a psychometrician in the utilities industry. It will explore the role assessments play at PG&E, and how Questionmark has helped the company focus on safety and train its employees.

I recently asked her about her session:

Tell me about PG&E and its use of assessments:

PG&E is a utilities company that provides natural gas and electricity to most of the northern two-thirds of California. Over the years, we have evolved into a more data-driven company, and Questionmark has been a part of that for the past 7 years. Having assessments readily available and secured within a platform that we can trust is very important to PG&E. We are also glad to have found a testing tool that offers such a wide variety of question types.

Why is safety important in the utilities industry?

Depending on the activity that our employees perform, most of the work has serious safety implications — whether it is a lineman climbing up a  pole to perform liveline work or a utility worker digging near a major gas pipeline. Our technical training must have safety in mind and, more importantly, it must ensure that after going through training, employees are competent to perform their tasks safely and proficiently. In order to ensure workforce capability, we rely heavily on testing to prove that our workforce is in fact safe and proficient and that the community we serve and our employees are safe and receiving reliable services.

What role does Questionmark play in ensuring that safety?

Questionmark helps us focus on safety-related questions by allowing special assessment strategies such as identifying critical versus coachable assessment items and identifying cutscores for each accordingly. Questionmark also allows a secured platform so that we can ensure our test items are never compromised and that our employees are truly being assessed under fair circumstances.

To find out more about the role of Questionmark plays in ensuring safety, you’ll just have to attend my session at Questionmark Conference 2016 in Miami!

What are you looking forward to at the conference?

I am very much looking forward to ‘talking shop’ with other Psychometricians and sharing best practices with others in the utilities industry and other companies alike!

Thank you, Wendy for taking time out of your busy schedules to discuss your session with us!

palm tree emoji 2If you have not already done so, you still have a chance to attend this important learning event. Click here to register.

 

Next Page »