Final Early-Bird Discount Ends Next Week!

Register for Questionmark Conference 2017 by Wednesday, February 22, for a chance to take advantage of of our final early-bird discount.

Register now and save!

Here are some highlights from the agenda:
  • Assessments as Career Development Tools at Caterpillar Inc.
  • BYOL: Next Generation Assessment Authoring Using Demographics and Jump Blocks
  • Secure Authentication: Single sign-on for Questionmark
  • Staying Ahead of Evolving Security Threats
  • Setting a Fair, Defensible Cut (Passing) Scores

Don’t miss out on the opportunity to:

  • Maximize your Assessment Value
  • Harness the Power of your Results
  • Innovate with us on Future Assessment Technologies

Register now for early-bird savings
Book your room at the Inn and Spa at Loretto before Thursday, Feb. 16 to take advantage of our special rates

We look forward to seeing you in Santa Fe, New Mexico, March 21-24 for our most important learning event of the year!

Seven tips to recruit and manage SMEs for technology certification exams

imagePosted by John Kleeman

How do you keep a certification exam up to date when the technology it is assessing is changing rapidly?

Certifications in new technologies like software-as-a-service and cloud solutions have some specific challenges. The nature of the technology usually means that questions often require very specialist knowledge to author. And because knowledge of the new technology is in short supply, subject matter experts (SMEs) who are able to author and review new items will be in high demand within the organization for other purposes.

Cloud technological offerings also change rapidly. It used to be that new technology releases came out every year or two, and if you were writing certification exams or other assessments to test knowledge and skill in them, you had plenty of notice and could plan an update cycle. But nowadays most technology organizations adopt an agile approach to development with the motto “release early, release often”. The use of cloud technology makes frequent, evolutionary releases – often monthly or quarterly – normal.

So how can you keep an exam valid and reliable if the content you are assessing is changing rapidly? Here are seven tips that could help – a few inspired by an excellent presentation by Cisco and Microsoft at the recent European Association of Test Publishers conference.

  1. Try to obtain item writing SMEs from product development. They will know what is coming and what is changing and will be in a good position to write accurate questions. 
  2. Also network for SMEs outside the organization – at technology conferences, via partners and resellers, on social media and/or via an online form on your certification website. A good source of SMEs will be existing certified people.
  3. Incentivize SMEs – what will work best for you will depend on your organization, but you can consider free re-certifications, vouchers, discounts off conferences, books and other incentives. Remember also that for many people working in technology, recognition and appreciation are as important as financial incentives. Appreciate and recognize your SMEs. For internal SMEs, send thank you letters to their managers to appreciate their effort.
  4. Focus your exam on underlying key knowledge and skills that are not going to become obsolete quickly. Work with your experts to avoid items that are likely to become obsolete and seek to test on fundamental concepts not version specific features.
  5. When working with item writers, don’t be frightened to develop questions based on beta or planned functionality, but always do a check before questions go live in case the planned functionality hasn’t been released yet.
  6. Analyze, create, deliverSince your item writers will likely be geographically spread and will be busy and tech-literate, use a good collaborative tool for item writing and item banking that allows easy online review and tracking of changes. (See https://www.questionmark.com/content/distributed-authoring-and-item-management for information on Questionmark’s authoring solution.)
  7. In technology as in other areas, confidentiality and exam security are crucial to ensure the integrity of the exam. You should have a a formal agreement with internal and external SMEs  who author or review questions to remind them not to pass the questions to others. Ensure that your HR or legal department are involved in the drafting of these so that they are enforceable.

Certification of new technologies helps adoption and deployment and contributes to all stakeholders success. I hope these tips help you improve your assessment programme.

Sneak Peek: Conference Agenda & Special Workshop in Santa Fe

Posted by Julie Delazyn

Certified Performance Technologist Jim Parry

Questionmark Conference 2017 is the place to be for vital info and training on the latest assessment technologies and best practices. This event will give you the tools you need to take your assessments to the next level and harness the power of your results.

Register by February 22 — your final chance to save for early-bird discounts.

This year, we’re offering a special pre-conference session. Advanced Test Item Writing Workshop – Learn how to test more than just knowledge. Join Certified Performance Technologist Jim Parry – an expert user of Questionmark technologies – in this fast paced, high-powered workshop.

What’s on the agenda? Here’s a sneak-peek of what you can expect in Santa Fe March 21-24:

  • Explore the Advanced Editor in Next Gen Authoring
  • Avionics: A hands-on, performance-based certification using Questionmark at SpaceTEC
  • Certifying Services-Oriented Associates at PetSmart
  • Setting a Fair, Defensible Cut (Passing) Scores
  • Staying Ahead of Evolving Security Threats

FBI and Homeland Security advice on trumping cybersecurity attacks

Posted by John Kleeman

There’s a lot in the news recently about possible cybersecurity attacks on the political process. Here are some thoughts on how we can learn from this and apply it to assessment security.

One of the most interesting documents I’ve read on this subject is the Department of Homeland Security and FBI’s joint analysis report  JAR-16-20296 titled GRIZZLY STEPPE – Russian Malicious Cyber Activity.  This presents evidence on how a cybersecurity attack was made on a US political party in 2016 and gives some practical advice on how others can set up their systems to avoid such attacks.

Whoever the attack was performed by (and there has been some debate about this), the practical advice is useful to anyone who wants to improve their security. I was particularly struck by a section in the report which offered questions to ask your organization to see if they have good cybersecurity practices. I’ve taken the liberty of including the questions in the graphic below:

See Grizzly Steppe report for text here

I’ve shared various sets of security questions in this blog, including Eight ways to check if security is more than skin deep and 24 midsummer questions to ask your assessment software provider, but here are some questions from a very credible source!

I’d encourage you to pose these questions within your organization and with your suppliers to check that you are well protected in case of a cyberattack. Questionmark, like all sensible organizations, believes in continuous improvement in our security, and listening to sources like this analysis informs our improvement.

I hope highlighting the report and these questions helps strengthen your defenses against cybersecurity and acts as a guide in choosing your vendors.

Seven New Year’s Resolutions to Keep Your Assessments Safe

Paper with "Resolutions" written on it implying one is about to write some resolutions downJohn Kleeman HeadshotPosted by John Kleeman

Many blogs at this time of year seek to predict the year ahead, and many of them foresee more data breaches and security incidents in 2017.  But I’m a great believer that the best way to predict the future is to create or change it yourself. So if you want to reduce the chances of your assessment data security being breached in 2017, make some of the things you’ve talked about happen.

Here are some possible New Year’s resolutions that could help keep your assessments safe and secure.

1. Audit your user accounts. Go through each of your systems that hold or give access to assessment data, and check there are no accounts for ex-employees or ex-contractors. Make sure there are no generic or test accounts that do not belong to a current individual. Dormant accounts like this are a common route to a breach. Also check that no one who has changed role has the privileges of their old role.

2. Run an incident response table-top practice exercise. This is a session where you gather together those responsible for security, pretend there is a breach or other incident and work through verbally how you’d deal with it as a team. You can do this in a couple of hours with good preparation, and it allows you to check your procedures and ensure people know what to do. It will often give useful insight into improving your preparedness.  As Benjamin Franklin once said “An ounce of prevention is worth a pound of cure”.

3. Start testing your personnel on security procedures. One of the biggest security risks for any organization is staff mistakes and accidents that compromise credentials or data. Security awareness training makes an important difference. And if you test your personnel on security after the training, you verify that people understand the training and you identify areas of weakness. This makes it more likely that your personnel become more aware and follow better security practices. If you have access to an online assessment tool like Questionmark, it’s very, very easy to do.

Photo of doctor stethoscope on computer keyboard4. Review some of your key vendors. A risk for most organizations is weaknesses in suppliers or subcontractors that have access to your data. Ask suppliers to share information on their technical and organizational measures for security and what they are doing to ensure that your data is not breached. Any reputable organization will be willing and able to provide this under NDA. See 24 midsummer questions to ask your assessment software provider on this blog for some of the questions you can ask.

 

5. Conduct a restore test from backups. How do you know your backups work? Over the years, I’ve come across a few organizations and teams who’ve lost their data because their backups didn’t work. The only way to be sure is to test restoring it from backup and check data is there. If you don’t already run restore tests, organize a restore test in 2017 (ideally once a quarter, but once is better than not at all). You shouldn’t need to do this if you use a cloud service like Questionmark OnDemand as the vendor should do it for you.

6. Run a pilot for online proctoring. Microsoft do it. SAP do it. Why shouldn’t you do it? If you run a certification program that uses physical test centers, consider whether online proctoring might work for you. Not only will it reduce the risk of collusion with proctors helping candidates cheat, but it will also be a huge boon to your candidates who will no longer need to travel to test centers.

TheCadetHonorCodeMonument7. Put in place a code of conduct for your participants. This is a simple thing to do and can make a big difference in reducing cheating by encouraging test-takers to stay honest.  See Candidate Agreements: Establishing honor codes for test takers and What is the best way to reduce cheating? on this blog for tips on how and why to do this. If you are looking for inspiration, at famous code of conduct is that of the U.S. Army West Point Military Academy which simply says: “A cadet will not lie, cheat, steal, or tolerate those who do.” Of course you need to communicate and get buy-in for your code of conduct, but if you do, it can be very effective.

Many of you will already be doing all of these things, but if you’re not, I hope one or more of these resolutions help you improve your assessment security in 2017.

And here’s a bonus New Year’s resolution to consider. Questionmark Information Security Officer David Hunt and I are giving a session on Staying Ahead of Evolving Security Threats at the Questionmark conference in March in Santa Fe. Make a New Year’s resolution to come to the conference, and learn about security and assessment!

A quiz for the Holidays?

The holidays are here; it’s time for cheer!

No matter if you’ve been naughty or nice, take this quiz and check it twice.

Scan the QR Code or click here to get started:

Wishing you a joyous holiday season and happy new year!

happyholidays_questionmark

 

« Previous PageNext Page »