Calling all Speakers: Share your story in Santa Fe & become a VIP

Posted by Julie Delazyn

We have officially announced the Questionmark Conference 2017 March 21-14, and we look forward to seeing you at the beautiful Inn and Spa at Loretto for this important learning event.

In order to create a rich and varied conference program, we have opened a call for proposals and invite you to submit your idea for case study presentation or a peer discussion soon.

How do you know if you should submit a proposal? If you can answer yes to any of the questions below, we want to hear from you! Click here to find out more and download the proposal forms!

  • Your experience with Questionmark technologies will help others
  • You have found innovative ways to use online assessments
  • You can explain how you organized your assessment program and what you learned
  • You have gained a lot from previous conferences and want to contribute in 2015
  • You are using assessments to support organizational goals
  • You have a unique application of online or mobile assessments
  • You have integrated Questionmark with another system

We are accepting proposals from now until December 1, so consider what you’d like to contribute.

The Inn and Spa at Loretto

Aside from helping your fellow Questionmark users by sharing your story, please note that presenters will also get some perks:

Presenters and discussion leaders will receive some red carpet treatment — including a special dinner in their honor on Wednesday, March 21. And we award one 50% registration for each case study presentation.

Click here to start the process of becoming a speaker!

Even if you are not sure you’ll attend the conference, we would like to hear from you! And whether you plan to present or not, plan now to have the conference in your budget for 2017.

Single sign-on: secure and easy access

Bart Hendrickx Small

Posted by Bart Hendrickx

It’s Tuesday morning. You have just started your computer. Now it’s time to open your day-to-day tools: email, chat/phone, tasks and so on. As you go through your tasks, you realize you need to take that data security test you’ve been postponing.

Every day, you interact with various applications. Some are installed on your personal computer, others on servers managed either by your organization by vendors.  Your applications might come from a multitude of service providers, in-house or on the Cloud.

For many of those applications, you need to authenticate—to tell the applications who you are—so that the app can present the information that pertains to you. Sometimes this happens automatically. When your email client connects to the mail server, you read your emails, not those of your co-workers. Your email client has authenticated you against your mail server because you entered a username or email address, a password and some other data, way back when.

You often need to use different sign-ins for different apps. When you log in tor you Questionmark OnDemand portal, for instance, you enter a different username and password than the one you used to unlock your computer earlier today, (Your organization’s data security policy does not allow you to  store your organizational password in other systems.)

Want to learn more? I’ll be discussing this topic and more at the Questionmark Conference 2016 in Miami, April 12-15. Register before March 3 to take advantage of our final early-bird discounts.

Problem: Unrecognized username or password

You’ve logged in for your exam, but you get an error message. Maybe you mistyped the password? Second try. Nope; same results. You must have forgotten your password. You start an instant message window with your internal IT help desk. “Sorry, we don’t manage Questionmark OnDemand. Can you use its password reset function?”

You go back to your Questionmark login page, get a secure on-time login and establish a new, permanent password that complies with the data security policy — “I better not forget my password this time,” you say to yourself as you finally start your data security test. “Isn’t there something more convenient?”

Solution: Single sign-on

We all find ourselves in similar situations, but with Single sign-on (SSO) we can avoid them.

Since, there are several definitions of SSO, here’s how I’ll define it in the context of this blog:

Single Sign-On (SSO) for software is the ability for one application, the identity provider, to tell another application, the service provider, who you are.

By identity provider, I mean a system that contains digital identity information—also known as people data—on users, For example, think of social network sites or Active Directory from Microsoft.

The service provider is the system that users work with to do something—say Questionmark OnDemand, in the case of your data security test.

With SSO, a user does not log on directly to the service provider. Instead, they log on to an identity provider, which then tells the service provider who the user is. The identity provider and service provider have been configured to trust each other. So when the identity provider says: “This is Jane Doe,” the service provider will trust and accept that.

It is important to note that SSO is therefore not about creating accounts with the same usernames and passwords—a prevalent mechanism for different service providers. SSO is about making those service providers accept what an identity provider says about a user.

Why SSO?

SSO comes with several advantages. Users can access all applications that are linked to their identity providers—using one username and password for multiple systems. Depending on the capabilities of the applications and how things have been set up, the authentication can be seamless. You might log on to your identity provider when you start your computer, and the other applications (service providers) you access during the day will automatically check with your identity provider without you having to enter your username and password again.

SSO makes password management easier for IT administrators. Having an employee leave an organization might mean having to decommission access to dozens of service providers. If the authentication to those service providers has been set up with SSO, then an IT administrator only needs to decommission the employee’s identity provider account. Without that account, the employee can no longer log on to any of the linked applications.

There is one disadvantage to SSO: If the account at the identity provider is hacked, all linked applications can be compromised. It is therefore imperative the account is properly secured. How can you set up SSO to ensure its security and effectiveness? Watch for more posts on this subject, which will include information about our newly added support for a popular technique called SAML.

If you would like to learn more, attend my session: Secure Authentication: Accessing Questionmark OnDemand with SSO at the Questionmark Conference 2016, April 12-15. Register before March 3 to take advantage of our final early-bird discounts.

Standard Setting: A Keystone to Legal Defensibility

Austin Fossey-42Since the last Questionmark Users Conference, I have heard several clients discuss new measures at their companies requiring them to provide evidence of the legal defensibility of their assessment. Legal defensibility and validity are closely intertwined, but they are not synonymous. An assessment can be legally defensible, yet still have flaws that impact its validity. The distinction between the two is often the difference between how you developed the instrument versus how well you developed the instrument.

Regardless of whether you are concerned with legal defensibility or validity, careful attention should be paid to the evaluative component of your assessment program. What if  someone asks, “What does this score mean?” How do you answer? How do you justify your response? The answers to these questions impact how your stakeholders will interpret and use the results, and this may have consequences for your participants. Many factors go into supporting the legal defensibility and validity of assessment results, but one could argue that the keystone is the standard-setting process.

Standard setting is the process of dividing score scales so that scores can be interpreted and actioned (AERA, APA, NCME, 2014). The dividing points between sections of the scales  are called “cut scores,” and in criterion-referenced assessment, they typically correspond to performance levels that are defined a priori. These cut scores and their corresponding performance levels help test users make the cognitive leap from a participant’s response pattern to what can be a complex inference about the participant’s knowledge, skills, and abilities.

In their chapter in Educational Measurement (4th Ed.), Hambleton and Pitoniak explain that standard-setting studies need to consider many factors, and that they also can have major implications for participants and test users. For this reason, standard-setting studies are often rigorous, well-documented projects.

At this year’s Questionmark Users Conference, I will be delivering a session that introduces the basics of standard setting. We will discuss standard-setting methods for criterion- referenced and norm-referenced assessments, and we will touch on methods used in both large-scale assessments and in classroom settings. This will be a useful session for anyone who is working on documenting the legal defensibility of their assessment program or who is planning their first standard-setting study and wants to learn about different methods that are available. Participants are encouraged to bring their own questions and stories to share with the group.

Register today for the full conference, but if you cannot make it, make sure to catch the live webcast!

Napa Conference Overview & Special Live Webcast

Headshot JuliePosted by Julie Delazyn

The Questionmark Users Conference in Napa Valley March 10-13 is officially less than a month away!2015-napa-01

We are looking forward to seeing you there for three full days of learning and special events.

Experiencing the complete conference offers by far the best way to learn about assessment-related best practices and about how to make the most of Questionmark technologies. Participants will hear customer case studies and learn about the unique ways in which other organizations are using assessments. They will also weigh in on the product road map, connect with Questionmark staff and network with their peers while enjoying California’s beautiful wine country.

But here’s a great opportunity for Questionmark users who can’t travel to the conference this year: live webcasts of selected conference sessions.

These select sessions will be broadcast at no charge March 11-12:

  • Opening General Session
  • Introduction to Item Development in Large-Scale Test Development
  • Hacking yourself first: Protecting your assessment data with penetration testing
  • Conference Keynotecollage
  • Overview of Setting Performance Standards: Making the cognitive leap from scores to interpretations
  • Writing Performance-Based Test Items

Make sure to sign up for conference webcasts

Aside from getting to take advantage of beautiful California wine country, there’s much more in store for conference participants:

Register today for the full conference, but if you cannot make it, make sure to catch the live webcast

See you in Napa Valley for the 2015 Users Conference

Julie Delazyn HeadshotPosted by Julie Delazyn

We’re very excited to announce plans for the 2015 Questionmark Users Conference.Fall_in_Napa_Valley

Questionmark users will get together to learn best practices and discover new uses for online assessments from …Drumroll please…March 10 to 13 in the Napa Valley Marriott Hotel & Spa in the heart of California Wine Country.

Mark your calendar now for this important learning event, and register as soon as you can.

grape iconHere’s what you can expect during this gathering:

  • Real-world case studies by Questionmark users
  • Introductions to new solutions and featurescollage
  • Sessions explaining Questionmark features & functions
  • Presentations about testing and assessment best practices
  • Opportunities to influence future solutions
  • One-on-one meetings with Questionmark technicians
  • Plenty of time to network with your peers

grape iconHere’s some of the feedback we’ve received from people who joined us at this year’s conference:

“Excellent conference every year.”

“Really good sessions. I learned a lot of things that will help me improve our operation.”

“I got all the answers to all my questions and met the right people within my first day here. This conference is an experience I would do again in a heartbeat.”

“I learned so much! … The conference is three days of fun, learning, and meeting people who are going through the same things that you are.”

Early-bird registration discounts are available until January 29th 2015, so sign up soon and start making your plans for Napa Valley.

South African Users Conference Programme Takes Shape

Chloe MendoncaPosted by Chloe Mendonca

In just five weeks, Questionmark users and other learning professionals will gather in Midrand for the first South African Questionmark Users Conference.

Delegates will enjoy a full programme, from case studies to features and functions sessions on the effective use of Questionmark technologies.

There will also be time for networking during lunches and our Thursday evening event.

Here are some of the sessions you can look forward to:

  • Case Study: Coming alive with Questionmark Live: A mind shift for lecturers – University of Pretoria
  • Case Study: Lessons and Discoveries over a decade with Questionmark – Nedbank
  • Case Study: Stretching the Boundaries: Using Questionmark in a High-Volume Assessment Environment – University of Pretoria
  • Features and Functions: New browser based tools for collaborative authoring – Overview and Demonstrations
  • Features and Functions: Analysing and Sharing Results with Stakeholders: Overview of Key New Questionmark Reporting and Analytics features
  • Features and Functions: Extending the Questionmark Platform: Updates and overviews of APIs, Standards Support, and integrations with third party applications
  • Customer Panel Discussion: New Horizons for eAssessment

You can register for the conference online or visit our website for more information.

We look forward to seeing you in August!

Chloe-Banner-SA-Conference-2014